Information processing device, endoscopic device, information processing method, and program

Abstract

This information processing device comprises a processor. The processor: acquires multiple first endoscopic examination images, which have been obtained by capturing images of multiple regions in a first endoscopic examination which is an endoscopic examination performed previously, from a storage which has stored therein the multiple first endoscopic examination images; and causes a display unit to display, as a reference image, at least one first endoscopic examination image that satisfies a specific condition from among the multiple first endoscopic examination images, during a period in which a second endoscopic examination being the current endoscopic examination is performed.

Description

Information processing device, endoscope device, information processing method, and program

[0001] The technology disclosed herein relates to an information processing device, an endoscope device, an information processing method, and a program.

[0002] Japanese Patent No. 6284439 discloses a medical information processing system including an endoscopic system for performing endoscopic examinations on a subject and an examination information management system for managing past examination information. The examination information management system includes an examination information storage unit that stores past examination information, an examination information extraction unit that extracts the subject's past examination information from the examination information storage unit, and a transmission unit that transmits the extracted past examination information of the subject to the endoscopic system. The endoscopic system includes a receiving unit that receives the subject's past examination information from the examination information management system, an image acquisition unit that acquires observation images of the subject captured by an imaging device, a display control unit that displays the observation images acquired by the image acquisition unit and information about the observation site included in the examination information received by the receiving unit on a display device, a completion information receiving unit that receives site observation completion information indicating that observation of the observation site corresponding to the information displayed on the display device has been completed, and a site information registration unit that registers the site observation completion information. When the completion information receiving unit receives the area observation completion information, the area information registration unit registers the area observation completion information in association with information regarding the observed area, and when the area information registration unit registers the area observation completion information, the display control unit displays information regarding the next observation area to be observed on the display device.

[0003] International Publication No. 2019 / 049451 discloses a video processor including: a time measurement unit that measures the elapsed time from a reference timing in an endoscopic examination; a memory unit that stores a first endoscopic observation image acquired in a first endoscopic examination and the elapsed time from the reference timing in the first endoscopic examination at the time the first endoscopic observation image was acquired, in association with each other; and a display control unit that controls the simultaneous display of a second endoscopic observation image acquired in a second endoscopic examination performed after the first endoscopic examination and the first endoscopic observation image stored in the memory in association with the same elapsed time from the reference timing in the second endoscopic examination at the time the second endoscopic observation image was acquired.

[0004] One embodiment of the technology disclosed herein provides an information processing device, an endoscopic device, an information processing method, and a program that allow a user to understand changes in the area that is the main target of an endoscopic examination.

[0005] A first aspect of the technology disclosed herein is an information processing device that includes a processor, which acquires multiple first endoscopic examination images from a storage device in which multiple first endoscopic examination images obtained by imaging multiple areas in a first endoscopic examination, which is an endoscopic examination performed previously, are accumulated, and displays at least one first endoscopic examination image from the multiple first endoscopic examination images that satisfies specific conditions on a display device as a reference image during the period in which the current endoscopic examination, which is a second endoscopic examination, is performed.

[0006] A second aspect of the technology disclosed herein is an information processing device according to the first aspect, in which the processor displays a reference image on the display device at a timeout stage that occurs during the implementation period of the second endoscopic examination.

[0007] A third aspect of the technology disclosed herein is an information processing device relating to the second aspect, in which a processor displays on a display device first information obtained during the implementation period of the first endoscopic examination at a timeout stage that occurs during the implementation period of the second endoscopic examination.

[0008] A fourth aspect of the technology of the present disclosure is an information processing device according to any one of the first to third aspects, in which a processor acquires second information required for a timeout that occurs during the period in which the second endoscopic examination is performed from a device in which the second information is stored, and displays the acquired second information on a display device.

[0009] A fifth aspect of the technology disclosed herein is an information processing device according to the fourth aspect, in which the processor stores the second information in storage when a timeout that occurs within the period during which the second endoscopic examination is performed is completed.

[0010] A sixth aspect of the technology of the present disclosure is an information processing device according to any one of the first to fifth aspects, in which the first endoscopic examination image is associated with information obtained upon timeout of the first endoscopic examination.

[0011] A seventh aspect of the technology of the present disclosure is an information processing device according to any one of the first to sixth aspects, in which the second endoscopic examination image obtained by imaging during the second endoscopic examination is associated with information obtained during the timeout of the second endoscopic examination.

[0012] An eighth aspect of the technology of the present disclosure is an information processing device according to any one of the first to seventh aspects, in which the specific condition is a condition selected in accordance with instructions accepted by the accepting device.

[0013] A ninth aspect of the technology of the present disclosure is an information processing device according to any one of the first to seventh aspects, in which the specific condition is selected by performing image recognition processing on a plurality of first endoscopic examination images and / or information processing on metadata of a plurality of first endoscopic examination images.

[0014] A tenth aspect of the technology of the present disclosure is an information processing device relating to any one of the first to seventh aspects, in which the specific condition is selected by performing image recognition processing on a plurality of first endoscopic examination images and / or information processing on metadata of a plurality of first endoscopic examination images, and is selected in accordance with instructions accepted by a reception device.

[0015] An eleventh aspect of the technology of the present disclosure is an information processing device according to the ninth or tenth aspect, in which the metadata includes endoscopic examination information obtained within the period in which the first endoscopic examination was performed.

[0016] A twelfth aspect of the technology of the present disclosure is an information processing device according to the eleventh aspect, in which the endoscopic examination information includes information obtained upon timeout of the first endoscopic examination.

[0017] A thirteenth aspect of the technology of the present disclosure is an information processing device relating to any one of the first to twelfth aspects, in which a processor performs a first notification process to notify when a location corresponding to a characteristic part in the reference image is captured in a second endoscopic examination image obtained by capturing an image in a second endoscopic examination.

[0018] A fourteenth aspect of the technology of the present disclosure is the information processing device according to the thirteenth aspect, in which the first notification process includes a process of changing a display mode of the reference image.

[0019] A fifteenth aspect of the technology of the present disclosure is an information processing device according to any one of the first to fourteenth aspects, in which a processor changes the display mode of the reference image depending on the positional relationship between a characteristic part in the reference image and a corresponding location that appears in a second endoscopic examination image obtained by capturing an image during a second endoscopic examination.

[0020] A sixteenth aspect of the technique of the present disclosure is the information processing device according to the fifteenth aspect, wherein the processor performs a second notification process of issuing a notification when the positional relationship becomes a default positional relationship.

[0021] A 17th aspect of the technology of the present disclosure is an information processing device relating to any one of the first to sixteenth aspects, in which a processor performs support processing to support adjusting the imaging conditions of the second endoscopic examination to the imaging conditions of the first endoscopic examination based on a reference image and a second endoscopic examination image obtained by capturing an image in the second endoscopic examination.

[0022] An 18th aspect of the technology of the present disclosure is an information processing device according to the 17th aspect, in which the support processing includes an output processing for outputting support information required to match the imaging conditions of the second endoscopic examination with the imaging conditions of the first endoscopic examination.

[0023] A 19th aspect of the technology of the present disclosure is an information processing device according to the 18th aspect, in which the support information is derived based on the result of comparing a first feature part shown in the reference image with a second feature part shown in the second endoscopic examination image.

[0024] A twentieth aspect of the technology of the present disclosure is an information processing device relating to any one of the first to nineteenth aspects, in which the support processing includes a third notification processing that notifies when the imaging conditions of the second endoscopic examination and the imaging conditions of the first endoscopic examination match.

[0025] A 21st aspect of the technology of the present disclosure is an information processing device according to any one of the first to twentieth aspects, in which a processor changes the composition of the reference image according to a second endoscopic examination image obtained by capturing an image in a second endoscopic examination.

[0026] A 22nd aspect of the technology of the present disclosure is an information processing device relating to any one of the 1st to 21st aspects, in which a processor determines whether a common lesion is visible in the reference image and the second endoscopic examination image based on the reference image and the second endoscopic examination image obtained by capturing the second endoscopic examination.

[0027] A 23rd aspect of the technology of the present disclosure is an information processing device according to the 20th aspect, in which the processor performs a fourth notification process to notify that the determination result has been confirmed when the processor determines that a common lesion is present in the reference image and the second endoscopic examination image and an instruction to confirm the determination result is received by the reception device.

[0028] A 24th aspect of the technology of the present disclosure is an information processing device relating to any one of the first to 23rd aspects, in which a processor generates lesion-related information regarding a lesion when a common lesion is captured in a reference image and a second endoscopic examination image obtained by capturing an image in a second endoscopic examination.

[0029] A twenty-fifth aspect of the technique of the present disclosure is the information processing device according to the twenty-fourth aspect, in which the lesion-related information includes size-related information relating to the size of the lesion.

[0030] A twenty-sixth aspect of the technology of the present disclosure is the information processing device according to the twenty-fifth aspect, in which the size-related information includes transition specifying information that can specify a transition of size over time.

[0031] A 27th aspect of the technology of the present disclosure is an information processing device according to the 26th aspect, in which the transition identification information is derived based on the size of the lesion shown in the reference image and / or the size of the lesion shown in the second endoscopic examination image.

[0032] A 28th aspect of the technology of the present disclosure is an information processing device according to the 27th aspect, in which the transition identification information is derived based on the size of the lesion shown in the reference image and / or the size and type of the lesion shown in the second endoscopic examination image.

[0033] A 29th aspect of the technology of the present disclosure is an information processing device according to any one of the 24th to 28th aspects, in which the lesion-related information includes information that can identify the type of lesion, information that can identify the number of lesions, and / or information that can identify the type of lesion.

[0034] A 30th aspect of the technology of the present disclosure is an information processing device according to any one of the 24th to 29th aspects, in which the lesion-related information is associated with a reference image showing the lesion and / or a second endoscopic examination image showing the lesion.

[0035] A 31st aspect of the technology of the present disclosure is an information processing device according to any one of the 24th to 30th aspects, in which a processor creates a report recording lesion-related information, a reference image showing the lesion, and / or a second endoscopic examination image showing the lesion.

[0036] A 32nd aspect of the technology of the present disclosure is an information processing device relating to any one of the 1st to 31st aspects, in which a processor displays on a display device a reference image and a second endoscopic examination image obtained by capturing an image in a second endoscopic examination so that they can be compared.

[0037] A thirty-third aspect of the technology of the present disclosure is an endoscopic device comprising an information processing device according to any one of the first to third aspects and an endoscope that captures images of multiple areas during an endoscopic examination.

[0038] A 34th aspect of the technology of the present disclosure is an information processing method comprising acquiring a plurality of first endoscopic examination images from a storage in which a plurality of first endoscopic examination images obtained by imaging a plurality of regions in a first endoscopic examination, which is an endoscopic examination performed previously, are accumulated, and displaying at least one first endoscopic examination image among the plurality of first endoscopic examination images that satisfies specific conditions as a reference image on a display device during the period in which a second endoscopic examination, which is the current endoscopic examination, is performed.

[0039] A 35th aspect of the technology of the present disclosure is a program for causing a computer to execute processing including acquiring a plurality of first endoscopic examination images from a storage in which a plurality of first endoscopic examination images obtained by imaging a plurality of parts in a first endoscopic examination, which is an endoscopic examination performed previously, are accumulated, and displaying at least one first endoscopic examination image from the plurality of first endoscopic examination images that satisfies specific conditions as a reference image on a display device during the period in which the current endoscopic examination, which is a second endoscopic examination, is performed.

[0040] 1 is a conceptual diagram showing an example of an aspect in which an endoscopic system is used. FIG. 1 is a conceptual diagram showing an example of the overall configuration of an endoscopic system. FIG. 1 is a conceptual diagram showing an example of an aspect in which an insertion portion of an endoscope is inserted into the large intestine of a subject. FIG. 2 is a block diagram showing an example of the hardware configuration of an endoscopic processing device. FIG. 3 is a block diagram showing an example of the hardware configuration of a control device. FIG. 4 is a block diagram showing an example of the hardware configuration of a server. FIG. 5 is a block diagram showing an example of main functions of a processor of the control device. FIG. 6 is a block diagram showing an example of main functions of a processor of the server. FIG. 7 is a conceptual diagram showing an example of processing content for displaying an endoscopic image on a display device. FIG. 8 is a conceptual diagram showing an example of an aspect in which a report is exchanged between the server and the control device. FIG. 9 is a conceptual diagram showing an example of the contents of a report. FIG. 10 is a conceptual diagram showing an example of an aspect in which the server transmits a report to the endoscopic device in response to a request from the endoscopic device. FIG. 11 is a conceptual diagram showing an example of processing content for generating a past examination image screen and a subject information screen based on a report. FIG. 12 is a conceptual diagram showing an example of an aspect in which subject identification information is displayed on a display device, and an example of an aspect in which a past examination image screen and a subject information screen are selectively displayed on a touch panel display of a tablet terminal. FIG. 13 is a conceptual diagram showing an example of an aspect in which an examination result image selected in accordance with an image selection instruction is selected as a reference image and stored in a reference image storage area. 1 is a conceptual diagram showing an example of an aspect in which information obtained upon timeout of the second endoscopic examination is recorded in a report and displayed on a touch panel display of a tablet terminal. FIG. 1 is a conceptual diagram showing an example of an aspect in which an image recognition unit performs image recognition processing on a reference image and compares a feature portion extracted by the image recognition processing with an endoscopic image. FIG. 1 is a conceptual diagram showing an example of an aspect in which a reference image is displayed on a past examination image screen when a reference image display instruction is received by a reception device, and an example of an aspect in which the reference image is enlarged and displayed on the past examination image screen when a corresponding feature portion is captured in the endoscopic image. FIG. 1 is a conceptual diagram showing an example of an aspect in which support information is derived based on an endoscopic image and a reference image, and the derived support information is displayed on the screen of a display device. FIG. 1 is a conceptual diagram showing processing contents when the imaging conditions for the second endoscopic examination match the imaging conditions for the first endoscopic examination.26B is a conceptual diagram showing an example of processing for determining whether an endoscopic image contains a lesion that is common to both the reference image and the endoscopic image, and for notifying the user when the endoscopic image contains a lesion that is common to both the reference image and the endoscopic image. FIG. 26C is a conceptual diagram showing an example of processing for determining whether an endoscopic image contains a lesion that is common to both the reference image and the endoscopic image, and for notifying the user when the endoscopic image contains a lesion that is common to both the reference image and the endoscopic image. FIG. 26D is a conceptual diagram showing an example of processing for determining whether an endoscopic image contains a lesion that is common to both the reference image and the endoscopic image, and for notifying the user when the endoscopic image contains a lesion that is common to both the reference image and the endoscopic image. FIG. 26E is a conceptual diagram showing an example of processing for determining whether an endoscopic image contains a lesion that is common to both the reference image and the endoscopic image, and for notifying the user when the endoscopic image contains a lesion that is common to both the reference image and the endoscopic image. FIG. 26F is a conceptual diagram showing an example of processing for determining whether an endoscopic image contains a lesion that is common to both the reference image and the endoscopic image. 1 is a conceptual diagram showing an example of a manner in which a reference image is selected by performing image recognition processing on a plurality of inspection result images, and a conceptual diagram showing an example of a manner in which a reference image is selected based on the results of information processing performed on a plurality of metadata corresponding to a plurality of inspection result images.

[0041] Hereinafter, exemplary embodiments of an information processing device, an endoscope device, an information processing method, and a program according to the techniques of the present disclosure will be described with reference to the accompanying drawings.

[0042] First, the terms used in the following description will be explained.

[0043] CPU is an abbreviation for "Central Processing Unit." GPU is an abbreviation for "Graphics Processing Unit." RAM is an abbreviation for "Random Access Memory." NVM is an abbreviation for "Non-volatile memory." EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory." ASIC is an abbreviation for "Application Specific Integrated Circuit." PLD is an abbreviation for "Programmable Logic Device." FPGA is an abbreviation for "Field-Programmable Gate Array." SoC is an abbreviation for "System-on-a-chip." SSD is an abbreviation for "Solid State Drive." USB is an abbreviation for "Universal Serial Bus." HDD is an abbreviation for "Hard Disk Drive." EL is an abbreviation for "Electro-Luminescence." CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor." CCD is an abbreviation for "Charge Coupled Device." LAN is an abbreviation for "Local Area Network." WAN is an abbreviation for "Wide Area Network." AI is an abbreviation for "Artificial Intelligence." BLI is an abbreviation for "Blue Light Imaging." LCI is an abbreviation for "Linked Color Imaging." In this embodiment, "match" refers to a perfect match, as well as a match that includes an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and that does not contradict the spirit of the technology of the present disclosure.

[0044] 1 , an endoscopic system 10 includes an endoscopic device 12. The endoscopic device 12 is used by a doctor 14 during an endoscopic examination. At least one assistant staff member 16 (e.g., a nurse) assists the doctor 14 in the endoscopic examination. Hereinafter, when it is not necessary to distinguish between the doctor 14 and the assistant staff member 16, they will also be referred to as "users" without being assigned reference numerals.

[0045] The endoscopic device 12 includes an endoscope (endoscopic scope) 18 and is a device for performing medical treatment on the inside of a subject 20 (e.g., a patient) via the endoscope 18. The endoscopic device 12 is an example of an "endoscopic device" according to the technology of the present disclosure. The endoscope 18 is an example of an "endoscope" according to the technology of the present disclosure.

[0046] The endoscope 18 captures images of the inside of the body of the subject 20, thereby acquiring and outputting images showing the state of the inside of the body. The example shown in Fig. 1 shows a mode in which the endoscope 18 is inserted into a body cavity from the anus of the subject 20. Note that in the example shown in Fig. 1, the endoscope 18 is inserted into the body cavity from the anus of the subject 20, but this is merely one example, and the endoscope 18 may also be inserted into the body cavity from the mouth, nostril, perforation, or the like of the subject 20, and the location where the endoscope 18 is inserted is determined by the type of endoscope 18, the surgical procedure, etc.

[0047] For ease of explanation, the endoscopic examination previously performed on the subject 20 will be referred to as the "first endoscopic examination." The endoscopic examination previously performed on the subject 20 refers to the endoscopic examination performed on the subject 20 before the previous one. An example of the endoscopic examination previously performed on the subject 20 is the previous endoscopic examination performed on the subject 20, but this is merely an example and may refer to an endoscopic examination performed on the subject 20 multiple times before. For ease of explanation, the current endoscopic examination on the subject 20 will be referred to as the "second endoscopic examination." In the following, when there is no need to distinguish between the first endoscopic examination and the second endoscopic examination, they will simply be referred to as "endoscopic examinations." In the following, it is assumed that multiple locations within the subject 20's body (e.g., the inner wall of the large intestine) are imaged by the endoscope 18 in each of the first endoscopic examination and the second endoscopic examination. Here, the multiple locations within the subject 20's body are an example of "multiple regions" according to the technology of the present disclosure.

[0048] The endoscopic device 12 includes an endoscopic processing device 22, a light source device 24, a control device 28, a display device 30, and a tablet terminal 32. The endoscopic processing device 22, the light source device 24, the control device 28, the display device 30, and the tablet terminal 32 are mounted on an armed wagon 34. The armed wagon 34 has a wagon 34A and an arm 34B. The wagon 34A has multiple stands arranged vertically, and the control device 28, endoscopic processing device 22, the light source device 24, and the display device 30 are mounted from the lower stand to the upper stand.

[0049] An arm 34B is attached to the wagon 34A. The arm 34B extends laterally from the side of the wagon 34A and is bent upward in an L-shape. A holder 34B1 is provided at the tip of the arm 34B. The holder 34B1 holds the tablet terminal 32 by clamping the end of the tablet terminal 32. The tablet terminal 32 can be removed from the holder 34B1 by releasing the clamping state of the holder 34B1. By holding the tablet terminal 32 with the arm 34B, the tablet terminal 32 and the display device 30 on the wagon 34A are arranged side by side.

[0050] The display device 30 displays various information including images. Examples of the display device 30 include a liquid crystal display and an EL display. The display device 30 displays a plurality of screens arranged side by side. In the example shown in FIG. 1 , screens 36 and 38 are shown as examples of the plurality of screens.

[0051] An endoscopic image 40 is displayed on the screen 36. The endoscopic image 40 is an image acquired by imaging an observation target region within the body cavity of the subject 20 using the endoscope 18. The observation target region may be the inner wall of the large intestine. The inner wall of the large intestine is merely one example, and the observation target region may be the inner or outer wall of another region, such as the small intestine, duodenum, or stomach.

[0052] The endoscopic image 40 displayed on the screen 36 is one frame included in a moving image that includes multiple frames. In other words, the screen 36 displays multiple frames of the endoscopic image 40 at a predetermined frame rate (e.g., 30 frames / second or 60 frames / second).

[0053] Subject identification information 42 is displayed on the screen 38. The subject identification information 42 is information relating to the subject 20. The subject identification information 42 includes, for example, the name of the subject 20, the age of the subject 20, an identification number that can identify the subject 20, and information to be careful of when performing a procedure on the subject 20 using the endoscope 18.

[0054] The tablet terminal 32 is equipped with a touch panel display 44. The touch panel display 44 has a display (e.g., a liquid crystal display or an EL display) and a touch panel. For example, the touch panel display 44 is formed by overlaying a touch panel on a display. An example of the touch panel display 44 is an out-cell type touch panel display in which the touch panel is overlaid on the surface of the display area of ​​the display. Note that this is merely one example, and the touch panel display 44 may also be an on-cell type or an in-cell type touch panel display, for example.

[0055] Various screens are displayed on the touch panel display 44. The various screens displayed on the touch panel display 44 and the screens 36 and 38 displayed on the display device 30 are visually contrasted by the user.

[0056] An example of a screen displayed on the touch panel display 44 is a previous examination image screen 46. The previous examination image screen 46 displays a list of multiple examination result images 50, and also displays subject identification information 52 arranged next to the multiple examination result images 50. Each of the multiple examination result images 50 is an endoscopic image 40 obtained by capturing images of multiple locations inside the body of the subject 20 (e.g., multiple locations on the inner wall of the large intestine) using the endoscope 18 during a first endoscopic examination of the subject 20. The subject identification information 52 is the same information as the information included in the subject identification information 42.

[0057] As shown in FIG. 2 as an example, the endoscope 18 includes a control section 54 and an insertion section 56. The insertion section 56 is tubular. The insertion section 56 has a distal end portion 58, a bending section 60, and a flexible section 62. The distal end portion 58, the bending section 60, and the flexible section 62 are arranged in this order from the distal end to the proximal end of the insertion section 56. The flexible section 62 is formed of a long, flexible material and connects the control section 54 and the bending section 60. The bending section 60 partially bends and rotates around the axis of the insertion section 56 when the control section 54 is operated. As a result, the insertion section 56 is advanced into the deeper part of the hollow organ while bending and rotating around the axis of the insertion section 56 in accordance with the shape of the hollow organ (e.g., the shape of the duct of the large intestine).

[0058] The distal end portion 58 is provided with an illumination device 64, a camera 66, and a treatment tool opening 68. The illumination device 64 has illumination windows 64A and 64B. The illumination device 64 emits light through the illumination windows 64A and 64B. Examples of the light emitted from the illumination device 64 include visible light (e.g., white light), invisible light (e.g., near-infrared light), and / or special light. Examples of special light include light for BLI and / or light for LCI. The camera 66 captures images of the inside of a tubular organ using an optical method. An example of the camera 66 is a CMOS camera. The CMOS camera is merely an example, and other types of cameras such as a CCD camera may also be used.

[0059] The treatment tool opening 68 is an opening that allows a treatment tool 70 to protrude from the distal end portion 58. The treatment tool opening 68 also functions as a suction port that sucks blood, internal waste, and the like. A treatment tool insertion port 72 is formed in the operation portion 54, and the treatment tool 70 is inserted into the insertion portion 56 from the treatment tool insertion port 72. The treatment tool 70 passes through the insertion portion 56 and protrudes to the outside from the treatment tool opening 68. In the example shown in FIG. 2, a puncture needle is shown as the treatment tool 70. Other examples of the treatment tool 70 include a wire, a scalpel, grasping forceps, a guide sheath, and an ultrasonic probe.

[0060] The endoscope device 12 includes a universal cord 74 and a reception device 76 .

[0061] The universal cord 74 has a base end 74A, a first tip end 74B, and a second tip end 74C. The base end 74A is connected to the operation unit 54. The first tip end 74B is connected to the endoscope processing device 22. The second tip end 74C is connected to the light source device 24.

[0062] The reception device 76 receives instructions from the user and outputs the received instructions as an electrical signal. Examples of the reception device 76 include a foot switch, a keyboard, a mouse, a touch panel, and a microphone.

[0063] A reception device 76 is connected to the endoscope processing device 22. The endoscope processing device 22 exchanges various signals with the camera 66 and controls the light source device 24 in accordance with instructions received by the reception device 76. The endoscope processing device 22 causes the camera 66 to capture an image, and acquires and outputs an endoscopic image 40 (see FIG. 1 ) from the camera 66. The light source device 24 emits light under the control of the endoscope processing device 22 and supplies the light to the illumination device 64. A light guide is built into the illumination device 64, and the light supplied from the light source device 24 is emitted from illumination windows 64A and 64B via the light guide.

[0064] The control device 28 controls the entire endoscopic device 12. The control device 28 is connected to the endoscopic processing device 22, the display device 30, the tablet terminal 32, and the reception device 76. The control device 28 controls the display device 30 and the tablet terminal 32 in accordance with instructions received by the reception device 76. The control device 28 acquires an endoscopic image 40 from the endoscopic processing device 22 and causes the display device 30 to display a screen 36 including the acquired endoscopic image 40, and causes the display device 30 to display a screen 38 including subject identification information 42 (see FIG. 1 ). The control device 28 also selectively displays multiple screens, including a previous examination image screen 46 (see FIG. 1 ), on the touch panel display 44 of the tablet terminal 32. The control device 28 is an example of an "information processing device" according to the technology disclosed herein.

[0065] The endoscope system 10 includes a server 78. The server 78 has a computer 80, which is the main body of the server 78, a display device 82, and a reception device 84. The computer 80 and the control device 28 are communicatively connected via a network 86. An example of the network 86 is a LAN. Note that the LAN is merely an example, and the network 86 may be configured with at least one of a LAN, a WAN, etc.

[0066] The control device 28 is positioned as a client terminal for the server 78. Therefore, the server 78 executes processing in response to a request provided from the control device 28 via the network 86, and provides the processing results to the control device 28 via the network 86.

[0067] The display device 82 and the reception device 84 are connected to the computer 80. The display device 82 displays various information under the control of the computer 80. Examples of the display device 82 include a liquid crystal display and an EL display. The reception device 84 receives instructions from a user of the server 78, etc. Examples of the reception device 84 include a keyboard and a mouse. The computer 80 executes processing in accordance with the instructions received by the reception device 84.

[0068] 3 , the insertion section 56 of the endoscope 18 is inserted from the anus of the subject 20 into the large intestine 88. The camera 66 captures images of the inside of the large intestine 88 to generate an endoscopic image 40. The endoscopic image 40 is generated as an image showing the appearance of an inner wall 88A. For example, the camera 66 inserted into the large intestine 88 moves from the entrance side to the back side of the large intestine 88, and captures images of the inner wall 88A from the entrance side to the back side of the large intestine 88 at a predetermined frame rate. As a result, a moving image is generated that includes multiple frames of endoscopic images 40 showing the appearance of the inner wall 88A from the entrance side to the back side of the large intestine 88.

[0069] 4 , the endoscope processing device 22 includes a computer 90 and an input / output interface 92. The computer 90 includes a processor 94, a RAM 96, and an NVM 98. The input / output interface 92, the processor 94, the RAM 96, and the NVM 98 are connected to a bus 100.

[0070] For example, the processor 94 has a CPU and a GPU, and controls the entire endoscope processing device 22. The GPU operates under the control of the CPU and is mainly responsible for executing image processing. Note that the processor 94 may be one or more CPUs that have integrated GPU functionality, or one or more CPUs that do not have integrated GPU functionality.

[0071] The RAM 96 is a memory that temporarily stores information and is used as a work memory by the processor 94. The NVM 98 is a non-volatile storage device that stores various programs, various parameters, etc. Examples of the NVM 98 include a flash memory (e.g., an EEPROM) and / or an SSD. Note that the flash memory and the SSD are merely examples, and the NVM 98 may be another non-volatile storage device such as an HDD, or may be a combination of two or more types of non-volatile storage devices.

[0072] The input / output interface 92 is connected to the reception device 76, and the processor 94 receives instructions received by the reception device 76 via the input / output interface 92 and executes processing in accordance with the received instructions. The input / output interface 92 is also connected to the camera 66. The processor 94 controls the camera 66 via the input / output interface 92 and receives, via the input / output interface 92, endoscopic images 40 obtained by the camera 66 capturing images of the interior of the body of the subject 20. The light source device 24 is also connected to the input / output interface 92. The processor 94 controls the light source device 24 via the input / output interface 92 to supply light to the illumination device 64 and adjust the amount of light supplied to the illumination device 64. The input / output interface 92 is also connected to the control device 28. The processor 94 exchanges various signals with the control device 28 via the input / output interface 92.

[0073] 5, the control device 28 includes a computer 102 and an input / output interface 104. The computer 102 includes a processor 106, a RAM 108, and an NVM 110. The input / output interface 104, the processor 106, the RAM 108, and the NVM 110 are connected to a bus 112.

[0074] The processor 106 controls the entire control device 28. Note that the multiple hardware resources included in the computer 102 shown in Fig. 5 (i.e., the processor 106, the RAM 108, and the NVM 110) are the same type as the multiple hardware resources included in the computer 90 shown in Fig. 4, and therefore a description thereof will be omitted here.

[0075] The input / output interface 104 is connected to the reception device 76, and the processor 106 acquires instructions accepted by the reception device 76 via the input / output interface 104 and executes processing according to the acquired instructions. The input / output interface 104 is also connected to the endoscope processing device 22, and the processor 106 exchanges various signals with the processor 94 (see FIG. 4) of the endoscope processing device 22 via the input / output interface 104.

[0076] The display device 30 is connected to the input / output interface 104, and the processor 106 controls the display device 30 via the input / output interface 104 to display various information on the display device 30. For example, the processor 106 acquires an endoscopic image 40 (see FIG. 1 ) from the endoscopic processing device 22 and causes the display device 30 to display the endoscopic image 40.

[0077] The endoscope device 12 includes a communication module 114. The communication module 114 is connected to the input / output interface 104. The communication module 114 is an interface including a communication processor, an antenna, etc. The communication module 114 is connected to the network 86 and controls communication between the processor 106 and the computer 80 of the server 78.

[0078] The endoscope device 12 includes a wireless communication module 116. The wireless communication module 116 is connected to the input / output interface 104. The wireless communication module 116 is an interface including a wireless communication processor, an antenna, and the like. The wireless communication module 116 is communicably connected to the tablet terminal 32 via a wireless LAN or the like, and controls communication between the processor 106 and the tablet terminal 32. Note that, although an example in which communication is performed wirelessly between the control device 28 and the tablet terminal 32 is given here, this is merely an example, and communication between the control device 28 and the tablet terminal 32 may also be performed wired.

[0079] The computer 102 is an example of a "computer" according to the technology of the present disclosure. The processor 106 is an example of a "processor" according to the technology of the present disclosure. The display device 30 and the tablet terminal 32 are examples of a "display device" according to the technology of the present disclosure.

[0080] 6, the server 78 includes a computer 80, a display device 82, and a reception device 84, as well as an input / output interface 118 similar to the input / output interface 104 (see FIG. 5), and a communication module 120 similar to the communication module 114 (see FIG. 5). The computer 80 includes a processor 122 similar to the processor 106 (see FIG. 5), a RAM 124 similar to the RAM 108 (see FIG. 5), and an NVM 126 similar to the NVM 110 (see FIG. 5). The input / output interface 118, the processor 122, the RAM 124, and the NVM 126 are connected to a bus 128.

[0081] The display device 82 is connected to the input / output interface 118, and the processor 122 controls the display device 82 via the input / output interface 118 to cause the display device 82 to display various information.

[0082] The input / output interface 118 is connected to the reception device 84, and the processor 122 acquires instructions received by the reception device 84 via the input / output interface 118 and executes processing according to the acquired instructions.

[0083] A communication module 120 is connected to the input / output interface 118. The communication module 120 is connected to the network 86 and, in cooperation with the communication module 114, manages communication between a processor 122 of the server 78 and the processor 106 of the control device 28.

[0084] Incidentally, a plurality of endoscopic images 40 obtained by imaging the subject 20 with the endoscope 18 during the first endoscopic examination using the endoscope 18 are recorded in a report or the like. While viewing the plurality of endoscopic images 40 recorded in the report or the like, the doctor 14 identifies a location (e.g., a lesion) within the body of the subject 20 where treatment (e.g., therapy) will be performed during the second endoscopic examination.

[0085] When performing the second endoscopic examination on the subject 20, it is important for all of the physicians 14 and the assistant staff 16 to understand how much the area of ​​the subject 20 that will be treated in the second endoscopic examination (hereinafter referred to as the "main examination target area") has changed since the time of the first endoscopic examination. One method for achieving this is to use a timeout period (i.e., a meeting held before or during the second endoscopic examination) to have all of the physicians 14 and the assistant staff 16 understand the main examination target area. In this case, it is preferable to select an appropriate endoscopic image 40 (i.e., an endoscopic image 40 that shows the main examination target area) from multiple endoscopic images 40 recorded in a report or the like, and present the selected endoscopic image 40 to all of the physicians 14 and the assistant staff 16.

[0086] In view of these circumstances, in this embodiment, endoscopic image display processing and endoscopic operation support processing are performed by the processor 106 of the control device 28 (see Figure 7), and report management processing is performed by the processor 122 of the server 78 (see Figure 8).

[0087] 7 , an endoscopic image display program 130 is stored in the NVM 110. The processor 106 reads the endoscopic image display program 130 from the NVM 110 and executes the read endoscopic image display program 130 on the RAM 108. The endoscopic image display process is realized by the processor 106 operating as a first control unit 106A in accordance with the endoscopic image display program 130 executed on the RAM 108.

[0088] An endoscope operation support program 132 is stored in the NVM 110. The processor 106 reads the endoscope operation support program 132 from the NVM 110 and executes the read endoscope operation support program 132 on the RAM 108. The endoscope operation support process is realized by the processor 106 operating as a second control unit 106B, a first transmission / reception unit 106C, and an image recognition unit 106D in accordance with the endoscope operation support program 132 executed on the RAM 108. The endoscope operation support program 132 is an example of a "program" according to the technology of the present disclosure.

[0089] 8 as an example, a report management program 134 is stored in the NVM 126. The processor 122 reads the report management program 134 from the NVM 126 and executes the read report management program 134 on the RAM 124. The report management process is realized by the processor 122 operating as a second transmission / reception unit 122A and a third control unit 122B in accordance with the report management program 134 executed on the RAM 124.

[0090] In the following, when the endoscopic image display processing and the endoscopic operation support processing are collectively referred to, they will be referred to as "controller processing." Furthermore, for ease of explanation, when the endoscopic image display processing, the endoscopic operation support processing, and the report management processing are collectively referred to as "endoscopic system processing." Furthermore, for ease of explanation, when the endoscopic image display program 130, the endoscopic operation support program 132, and the report management program 134 are collectively referred to as "endoscopic system programs."

[0091] As an example, as shown in FIG. 9 , in the endoscopic device 12, the first control unit 106A causes the display device 30 to display a screen 36. The first control unit 106A acquires endoscopic images 40 from the camera 66 at a predetermined frame rate and sequentially displays the acquired endoscopic images 40 on the screen 36. As a result, a moving image consisting of multiple endoscopic images 40 is displayed on the screen 36. The endoscopic images 40 displayed on the screen 36 are images obtained in the second endoscopic examination. Note that, hereinafter, when it is necessary to distinguish the endoscopic images 40 obtained in the second endoscopic examination from the endoscopic images 40 obtained in the first endoscopic examination, they will be referred to as "endoscopic images 40A." The endoscopic image 40A is an example of a "second endoscopic examination image" according to the technology of the present disclosure.

[0092] As an example, as shown in FIG. 10 , a plurality of reports 136 are stored in the NVM 126 of the server 78. For example, a report 136 is created for each different subject 20. The report 136 includes information about the subject 20 and information about the results of the first endoscopic examination performed on the subject 20. The third control unit 122B stores the report 136 in the NVM 126 and acquires the report 136 from the NVM 126. The second transceiver 122A transmits the report 136 acquired from the NVM 126 by the third control unit 122B to the control device 28. The report 136 transmitted by the second transceiver 122A is received by the first transceiver 106C of the control device 28. The first transceiver 106C of the control device 28 transmits the report 136 to the server 78. The report 136 transmitted by the first transceiver 106C is received by the second transceiver 122A. The report 136 received by the second transmitting / receiving unit 122A is stored in the NVM 126 by the third control unit 122B. The NVM 126 is an example of the "storage" and "device" according to the techniques of the present disclosure.

[0093] 11 as an example, a report 136 has recorded therein subject identification information 138 and endoscopic examination information 140. The subject identification information 138 is information capable of uniquely identifying the subject 20. The subject identification information 138 includes a subject number, which is a number uniquely assigned to the subject 20, the name, sex, and age of the subject 20, etc. The endoscopic examination information 140 is information relating to the first endoscopic examination performed on the subject 20 identified from the subject identification information 138. The endoscopic examination information 140 includes an examination number, which is a number uniquely assigned to the first endoscopic examination, the examination date and time when the first endoscopic examination was performed, the name of the doctor who performed the first endoscopic examination, and findings (e.g., the area where the first endoscopic examination was performed, the type of lesion discovered by the first endoscopic examination, the size of the lesion, and the location of the lesion, etc.).

[0094] The endoscopic examination information 140 also includes timeout information 142 and a plurality of examination result images 50. The timeout information 142 is information obtained during a timeout that took place during the implementation period of the first endoscopic examination identified from the examination number recorded in the report 136. The information obtained during the timeout includes information that can identify the participants in the timeout (such as their names or identification numbers), the matters confirmed during the timeout, the date and time when the timeout took place, and information indicating the location where the timeout took place.

[0095] The examination result image 50 is an endoscopic image 40 obtained in the first endoscopic examination identified from the examination number recorded in the report 136. Metadata 50A is associated with the examination result image 50 as data accompanying the examination result image 50. The metadata 50A includes, for example, various information related to the examination result image 50 (e.g., the date and time when the examination result image 50 was obtained, and the results of image recognition processing performed on the examination result image 50), as well as the same information as the timeout information 142.

[0096] As an example, as shown in FIG. 12 , when a timeout starts, the reception device 76 receives timeout start information. The timeout start information is information indicating that a timeout has started. Furthermore, when a timeout ends, the reception device 76 receives timeout end information. The timeout end information is information indicating that a timeout has ended. The reception device 76 receives timeout start information from the user when a timeout starts, and receives timeout end information from the user when a timeout ends. When the reception device 76 receives timeout start information, the processor 106 recognizes that a timeout has started. Furthermore, when the reception device 76 receives timeout end information, the processor 106 recognizes that a timeout has ended.

[0097] When the timeout start information is received by the reception device 76, the first transmission / reception unit 106C transmits, to the server 78, request information 144, which is information requesting the server 78 to transmit the report 136. The request information 144 includes information that can uniquely identify the report 136. An example of information that can uniquely identify the report 136 is the subject number and / or the examination number.

[0098] The request information 144 transmitted by the first transceiver 106C is received by the second transceiver 122A of the server 78. When the request information 144 is received by the second transceiver 122A, the third control unit 122B acquires a report 136 corresponding to the request information 144 (e.g., a report 136 identified from the subject number and / or examination number) from the NVM 126. The second transceiver 122A transmits the report 136 acquired from the NVM 126 by the second control unit 122B to the endoscopic device 12. The report 136 transmitted by the second transceiver 122A is received by the first transceiver 106C of the endoscopic device 12. Note that the reception of the report 136 by the first transceiver 106C is an example of "acquiring a plurality of first endoscopic examination images" according to the technique of the present disclosure.

[0099] 13 , in the control device 28, the second control unit 106B generates a past examination image screen 46 and a subject information screen 146 based on the report 136 received by the first transmission / reception unit 106C. The past examination image screen 46 includes a plurality of examination result images 50 and subject identification information 52.

[0100] The multiple examination result images 50 in the past examination image screen 46 are multiple examination result images 50 (see FIG. 11 ) recorded in the report 136 received by the first transmission / reception unit 106C as multiple endoscopic images 40 obtained for multiple regions (e.g., multiple locations in the large intestine 88) during the first endoscopic examination. Metadata 50A is also associated with the examination result images 50 in the past examination image screen 46. The subject identification information 52 is information (e.g., subject number, name, gender, age, etc.) in the subject identification information 138 recorded in the report 136 received by the first transmission / reception unit 106C. The subject identification information 52 is an example of "second information" according to the technology of the present disclosure, and is used as information required for a timeout performed during the implementation period of the second endoscopic examination.

[0101] The subject information screen 146 includes subject identification information 146A and timeout information 146B. The subject identification information 146A is information that can uniquely identify the subject 20. The subject identification information 146A is an example of "second information" according to the technology of the present disclosure and is used as information required for a timeout that occurs during the implementation period of the second endoscopic examination. An example of the subject identification information 146A is the same information as the subject identification information 138 included in the report 136 received by the first transceiver 106C. The timeout information 146B is information obtained during a timeout that occurs during the implementation period of the first endoscopic examination, which is identified from the examination number recorded in the report 136 received by the first transceiver 106C. An example of the timeout information 146B is the same information as the timeout information 142 included in the report 136 received by the first transceiver 106C. The timeout information 146B is an example of "first information" according to the technology of the present disclosure.

[0102] 14 as an example, the second controller 106B causes the display device 30 to display a screen 38 alongside the screen 36. The second controller 106B acquires subject identification information 138 from a report 136 shown in FIG. 13 and displays the acquired subject identification information 138 on the screen 38 as subject identification information 42.

[0103] Furthermore, the second control unit 106B causes the touch panel display 44 of the tablet terminal 32 to display the past examination image screen 46 or the subject information screen 146. The reception device 76 accepts a screen switching instruction, which is an instruction to switch the display between the past examination image screen 46 and the subject information screen 146. In response to the screen switching instruction accepted by the reception device 76, the second control unit 106B causes the touch panel display 44 to selectively display the past examination image screen 46 or the subject information screen 146.

[0104] As an example, as shown in FIG. 15 , with a previous examination image screen 46 displayed on the touch panel display 44, a user issues an image selection instruction to the touch panel display 44 to select one examination result image 50 as a reference image 150. The reference image 150 refers to an image referenced by the user. The example shown in FIG. 15 illustrates a mode in which an examination result image 50 containing a characteristic portion 152, which is a portion indicating a lesion identified from the endoscopic examination information 140 (see FIG. 11 ) of the report 136, is selected as the reference image 150. The characteristic portion 152 is an example of a "characteristic portion in a reference image" and a "first characteristic portion" according to the technology of the present disclosure. The image selection instruction is an example of an "instruction accepted by the reception device" according to the technology of the present disclosure.

[0105] In the tablet terminal 32, one test result image 50 is selected as the reference image 150 in accordance with the image selection instruction. In the control device 28, the second control unit 106B acquires, from the tablet terminal 32, the reference image 150 selected by the image selection instruction and metadata 50A associated with the test result image 50 selected as the reference image 150. The second control unit 106B stores the reference image 150 and metadata 50A acquired from the tablet terminal 32 in the reference image storage area 108A in the RAM 108 in an associated state.

[0106] As an example, as shown in FIG. 16 , timeout information 146C, which is information obtained upon timeout of the second endoscopic examination, is received by the reception device 76. Then, in the control device 28, the second control unit 106B records the timeout information 146C received by the reception device 76 in the report 136. An example of "recording" is adding an append. By recording the timeout information 146C in the report 136 in this way, the content of the report 136 is updated. Furthermore, the second control unit 106B displays the timeout information 146C received by the reception device 76 on the subject information screen 146 of the touch panel display 44. On the subject information screen 146, the timeout information 146C is displayed alongside the subject identification information 146A and the timeout information 146B.

[0107] As an example, as shown in FIG. 17 , in the endoscopic device 12, the reception device 76 receives a reference image display instruction. The reference image display instruction is an instruction to display a reference image 150 on the touch panel display 44. The reference image display instruction is issued by the user during the timeout period of the second endoscopic examination. When the reception device 76 receives the reference image display instruction, the image recognition unit 106D acquires the reference image 150 from the reference image storage area 108A. The image recognition unit 106D extracts a characteristic portion 152 from the reference image 150 by performing image recognition processing such as an AI method and / or a template matching method (hereinafter simply referred to as "image recognition processing") on the reference image 150. The image recognition unit 106D acquires an endoscopic image 40A from the camera 66. Then, the image recognition unit 106D compares the endoscopic image 40A acquired from the camera 66 with the characteristic portion 152 extracted from the reference image 150. The image recognition unit 106D determines whether the corresponding characteristic portion 154 is shown in the endoscopic image 40A by comparing the endoscopic image 40 with the characteristic portion 152. The corresponding characteristic portion 154 is an example of a "location corresponding to a characteristic portion" and a "second characteristic portion" according to the technology of the present disclosure.

[0108] On the other hand, when the reception device 76 accepts a reference image display instruction, the examination result image 50 selected in accordance with the image selection instruction shown in FIG. 15 is confirmed as the reference image 150. That is, when the reception device 76 accepts a reference image display instruction, the examination result image 50 that satisfies a specific condition (here, as an example, the image selected in accordance with the image selection instruction) is confirmed as the reference image 150. When the reception device 76 accepts a reference image display instruction, the second control unit 106B displays the reference image 150 and other examination result images 50 in a distinguishable manner on the past examination image screen 46, as shown in FIG. 18 . The reference image 150 is displayed on the past examination image screen 46 at the timeout stage during the second endoscopic examination. For example, the second control unit 106B displays the reference image 150 in a distinguishable manner by graying out the examination result images 50 other than the reference image 150 (see the upper diagram in FIG. 18 ), displaying them semi-transparently, or hiding them.

[0109] The condition that an image selection instruction is accepted by the reception device 76 and a reference image display instruction is accepted is an example of a "specific condition" and a "condition of being selected in accordance with an instruction accepted by the reception device" related to the technology disclosed herein.

[0110] As an example, as shown in FIG. 18 , when the image recognition unit 106D determines that the corresponding feature portion 154 is present in the endoscopic image 40A, the second control unit 106B performs processing to notify the user that the corresponding feature portion 154 is present in the endoscopic image 40A by enlarging and displaying the reference image 150 on the past examination image screen 46. The processing to notify the user that the corresponding feature portion 154 is present in the endoscopic image 40A is an example of a “first notification processing” according to the technology disclosed herein. While an example of enlarging and displaying the reference image 150 is given here, this is merely an example. For example, the user may be notified that the corresponding feature portion 154 is present in the endoscopic image 40A by highlighting the outline of the reference image 150 or by displaying a message or mark that identifies the presence of the corresponding feature portion 154 in the endoscopic image 40A.

[0111] Here, a support process for supporting adjustment of the imaging conditions for the second endoscopic examination to those for the first endoscopic examination will be described. One example of the support process is a process for adjusting the scale of the endoscopic image 40A to that of the reference image 150 when the scale (i.e., the angle of view) of the reference image 150 and the scale of the endoscopic image 40A are different. Therefore, the process for adjusting the scale of the endoscopic image 40A to that of the reference image 150 will be described with reference to FIGS. 19 and 20 .

[0112] There are two possible assumptions for adjusting the scale of the endoscopic image 40A to that of the reference image 150. The first assumption is that the actual size of the characteristic portion 152 in the reference image 150 matches the actual size of the corresponding characteristic portion 154 in the endoscopic image 40A. The second assumption is that the actual size of the characteristic portion 152 in the reference image 150 does not match the actual size of the corresponding characteristic portion 154 in the endoscopic image 40A.

[0113] Under the first premise, as shown in FIG. 19 as an example, the second control unit 106B acquires the pixel count 156 of the characteristic portion 152 from the reference image 150 and acquires the pixel count 158 ​​of the corresponding characteristic portion 154 from the endoscopic image 40A. The second control unit 106B calculates the difference 160 between the pixel count 156 and the pixel count 158. The second control unit 106B derives support information 162 corresponding to the difference 160. The support information 162 is derived using a table 163 in which the difference 160 and the support information 162 are associated with each other. The support information 162 is information required to match the scale of the endoscopic image 40A to the scale of the reference image 150. The information required to match the scale of the endoscopic image 40A to the scale of the reference image 150 refers to, for example, a message guiding the position of the camera 66 to a position where the difference 160 becomes zero. The second control unit 106B outputs the support information 162 corresponding to the difference 160 to the display device 30. As a result, the support information 162 is displayed on the screen 36 of the display device 30. The support information 162 is an example of "support information" according to the technology of the present disclosure. The process in which the second control unit 106B outputs the support information 162 to the display device 30 is an example of "output processing" according to the technology of the present disclosure.

[0114] 20 , each time a new endoscopic image 40A is obtained by the camera 66, the second control unit 106B calculates the difference 160 in the same manner as described above and determines whether the difference 160 has become zero. When the difference 160 has become zero, the second control unit 106B generates a notification message 164 and displays the generated notification message 164 on the screen 36, thereby performing processing to notify the user that the scale of the reference image 150 and the scale of the endoscopic image 40A match. The processing to notify the user that the scale of the reference image 150 and the scale of the endoscopic image 40A match is an example of the “assistance processing” and “third notification processing” according to the technology disclosed herein. The notification message 164 is a message that notifies the user that the scale of the reference image 150 and the scale of the endoscopic image 40A match.

[0115] The notification message 164 is merely an example. For example, the notification may be a voice output from a speaker, or any notification means may be used as long as it can make the user perceive that the scale of the reference image 150 and the scale of the endoscopic image 40A match.

[0116] On the other hand, under the second premise, the second control unit 106B calculates the difference between the value obtained by dividing the number of pixels 156 by the horizontal (or vertical) length of the characteristic portion 152 and the value obtained by dividing the number of pixels 156 by the horizontal (or vertical) length of the corresponding characteristic portion 154, and derives the support information 162 in the same manner as above. In this case, the notification using the support information 162 and the notification that the scale of the reference image 150 and the scale of the endoscopic image 40A match may also be performed in the same manner as above.

[0117] While the process of adjusting the scale of the endoscopic image 40A to that of the reference image 150 has been described above as an example, the technology of the present disclosure is not limited thereto. For example, a process of adjusting the optical characteristics (e.g., color, brightness, etc.) of the endoscopic image 40A to those of the reference image 150 may be performed. In this case, for example, the second control unit 106B may calculate the difference between the optical characteristics of the characteristic portion 152 and the optical characteristics of the corresponding characteristic portion 154 and derive support information (e.g., information indicating specific instructions regarding the type and / or light intensity of the light source) that reduces the calculated difference to zero. In this case, the notification using the support information and the notification that the optical characteristics of the reference image 150 and the optical characteristics of the endoscopic image 40A match may also be performed in a manner similar to that described above.

[0118] Furthermore, in the example shown in Figure 19, an example form has been described in which support information 162 is derived by comparing feature portion 152 with corresponding feature portion 154, but support information 162 may also be derived by comparing corresponding portions of reference image 150 and endoscopic image 40A in areas other than feature portion 152 and corresponding feature portion 154.

[0119] The scale of the reference image 150 and the optical characteristics of the endoscopic image 40A are examples of "imaging conditions for the first endoscopic examination" according to the technology of the present disclosure. The scale of the endoscopic image 40A and the optical characteristics of the endoscopic image 40A are examples of "imaging conditions for the second endoscopic examination" according to the technology of the present disclosure.

[0120] Once the process of adjusting the scale of the endoscopic image 40A to that of the reference image 150 is completed as described above, the processor 106 determines, based on the reference image 150 and the endoscopic image 40A, whether or not a common lesion is captured in the reference image 150 and the endoscopic image 40A. In this case, as shown in FIG. 21 as an example, the image recognition unit 106D performs pattern matching between the reference image 150 and the endoscopic image 40A to identify a corresponding feature portion 154 that matches the feature portion 152. The image recognition unit 106D then performs image recognition processing on the corresponding feature portion 154 to acquire a feature 166 possessed by the corresponding feature portion 154. Examples of the feature 166 include the type of lesion and the size of the lesion.

[0121] The second control unit 106B compares the endoscopic examination information 140 recorded in the report 136 with the features 166 to determine whether the endoscopic examination information 140 contains the same information as the features 166. If the second control unit 106B determines that the endoscopic examination information 140 contains the same information as the features 166, it generates a notification message 168. The notification message 168 is a message notifying that it has been determined that a common lesion is contained in the reference image 150 and the endoscopic image 40A. In other words, the notification message 168 is a message notifying that a lesion identical to the lesion discovered in the first endoscopic examination is likely to have also been discovered in the second endoscopic examination. The second control unit 106B displays the notification message 168 on the screen 36 of the display device 30.

[0122] When the notification message 168 is displayed on the screen 36, the doctor 14 confirms the endoscopic image 40A and the notification message 168. Then, as shown in FIG. 22 as an example, a confirmation instruction from the doctor 14 is received by the reception device 76. The confirmation instruction is an instruction to finalize the determination result that the reference image 150 and the endoscopic image 40A contain a common lesion. When the confirmation instruction is received by the reception device 76, the second control unit 106B generates a notification message 169. The notification message 169 is a message notifying that the determination result that the reference image 150 and the endoscopic image 40A contain a common lesion has been finalized. The second control unit 106B displays the notification message 169 on the screen 36 of the display device 30. This notifies that the determination result that the reference image 150 and the endoscopic image 40A contain a common lesion has been finalized. The process of notifying that the determination result that the reference image 150 and the endoscopic image 40A contain a common lesion has been confirmed is an example of the "fourth notification process" according to the technology of the present disclosure.

[0123] As an example, as shown in FIG. 23 , timeout information 146C obtained upon timeout of the second endoscopic examination is associated with the endoscopic image 40A. For example, the process of associating the timeout information 146C with the endoscopic image 40A is performed when a recording instruction from the user is received by the reception device 76. The recording instruction is an instruction to record the timeout information 146C in the report 136. The second control unit 106B assigns metadata 170 to the endoscopic image 40A that has undergone the image recognition process shown in FIG. 21 . When the reception device 76 receives a recording instruction, the second control unit 106B associates the timeout information 146C with the endoscopic image 40A by including the timeout information 146C in the metadata 170. The second control unit 106B records the endoscopic image 40A and the metadata 170 in the report 136. The report 136 in which the endoscopic image 40A and the metadata 170 are recorded is the report 136 received by the first transmission / reception unit 106C in the example shown in FIG. 12 .

[0124] The report 136, in which the endoscopic image 40A and the metadata 170 are recorded, is transmitted by the first transmitting / receiving unit 106C to the server 78. In the server 78, the second transmitting / receiving unit 122A receives the report 136 transmitted from the first transmitting / receiving unit 106C. The third control unit 122B stores the report 136 received by the second transmitting / receiving unit 122A in the NVM 126.

[0125] As an example, as shown in Fig. 24 , when a lesion common to both the reference image 150 and the endoscopic image 40A is captured, the second controller 106B generates lesion-related information 172 related to the lesion (i.e., the lesion common to both the reference image 150 and the endoscopic image 40A). The lesion-related information 172 is generated based on features 166 obtained by performing the image recognition process shown in Fig. 21 on the endoscopic image 40A, and an image recognition result 174. The image recognition result 174 is the result of the image recognition process performed on the reference image 150 shown in Fig. 17. The image recognition result 174 includes the type of lesion, the size of the lesion, and the like.

[0126] The lesion-related information 174 includes type information 172A, number information 172B, and aspect information 172C. The lesion-related information 174 also includes, as information relating to the size of the lesion, current size information 172D and transition specification information 172E.

[0127] The type information 172A is information that can identify the type of lesion common to the reference image 150 and the endoscopic image 40A. The number information 172B is information that can identify the number of lesions common to the reference image 150 and the endoscopic image 40A. The aspect information 172C is information that indicates the aspect of the lesion common to the reference image 150 and the endoscopic image 40A (e.g., the degree of inflammation, the degree of bleeding, the color of the lesion, and / or the shape of the lesion). The aspect of the lesion refers to, for example, the current aspect of the lesion. The current aspect of the lesion is identified from the feature 166. The current size information 172D is information that indicates the current size of the lesion common to the reference image 150 and the endoscopic image 40A. The current size is identified from the feature 166. The transition identification information 172E is information that can identify the transition over time of the lesion common to the reference image 150 and the endoscopic image 40A.

[0128] The transition identification information 172E is derived based on the size of the lesion shown in the reference image 150 and / or the size of the lesion shown in the endoscopic image 40A. The size of the lesion shown in the reference image 150 is the size of the lesion included in the image recognition result 174, and the size of the lesion shown in the endoscopic image 40A is the size of the lesion included in the feature 166. Examples of the transition identification information 172B include at least one piece of information from the first to fourth examples shown below.

[0129] A first example of the transition specifying information 172E is the ratio of the size of the lesion shown in the endoscopic image 40A to the size of the lesion shown in the reference image 150.

[0130] A second example of the transition identification information 172E is the rate of change in lesion size. The rate of change in lesion size is the ratio of the elapsed time to the value obtained by subtracting the size of the lesion shown in the reference image 150 from the size of the lesion shown in the endoscopic image 40A. The elapsed time refers to, for example, the time elapsed from the date and time when the first endoscopic examination was performed to the date and time when the second endoscopic examination was performed. The date and time when the first endoscopic examination was performed can be identified, for example, from the examination date and time (see FIG. 11 ) included in the endoscopic examination information 140 recorded in the report 136 received by the first transceiver 106C shown in FIG. 10 .

[0131] A third example of the transition identification information 172E is information indicating the size of the lesion several days later. The size of the lesion several days later is identified by regression analysis (e.g., extrapolation) using the size of the lesion captured in the reference image 150 and the size of the lesion captured in the endoscopic image 40A. Alternatively, the size of the lesion several days later may be a value obtained by multiplying the size identified by the regression analysis by a coefficient predetermined for the type of lesion identified from the feature 166 or the image recognition result 174.

[0132] A fourth example of the transition identification information 172E is information indicating the size of the lesion at a specified date and time. The size of the lesion at a specified date and time may be derived using an arithmetic expression in which the size of the lesion shown in the reference image 150 (and / or the size of the lesion shown in the endoscopic image 40A), the type of lesion, and the elapsed time from the examination date and time of the first endoscopic examination (or the second endoscopic examination) to the specified date and time are independent variables, and the size of the lesion at the specified date and time is a dependent variable.

[0133] Lesion-related information 174 is an example of "lesion-related information" according to the technology of the present disclosure. Current size information 172D and transition identification information 172E are examples of "size-related information" according to the technology of the present disclosure. Transition identification information 172E is an example of "transition identification information" according to the technology of the present disclosure. Type identification information 172A is an example of "information that can identify the type of lesion" according to the technology of the present disclosure. Number information 172B is an example of "information that can identify the number of lesions" according to the technology of the present disclosure. Mode information 172C is an example of "information that can identify the mode of a lesion" according to the technology of the present disclosure.

[0134] The second control unit 106B associates the lesion-related information 172 with the reference image 150 and the endoscopic image 40A by including the lesion-related information 172 in the metadata 50A and 170. As described in the example shown in Fig. 23 , the metadata 170 also includes timeout information 146C.

[0135] The second control unit 106B creates a report 136 in which the reference image 150, the endoscopic image 40A, and the lesion-related information 172 are recorded. That is, the second control unit 106B records the reference image 150 and the metadata 50A in the report 136. The report 136 in which the reference image 150 and the metadata 50A are recorded is stored in the NVM 126 of the server 78 in a manner similar to the example shown in FIG. 23 . The second control unit 106B also records the endoscopic image 40A and the metadata 170 in the report 136. The report 136 in which the endoscopic image 40A and the metadata 170 are recorded is stored in the NVM 126 of the server 78 in a manner similar to the example shown in FIG. 23 .

[0136] Although the above description has been given of an example in which the lesion-related information 172 is recorded in the report 136 while being included in the metadata 50A and 170, the technology of the present disclosure is not limited to this. For example, the lesion-related information 172 may be recorded in the report 136 as part of the findings.

[0137] Next, the operation of the endoscope system 10 will be described with reference to FIGS.

[0138] First, an example of the flow of the endoscopic image display process performed by the processor 106 of the control device 28 when the camera 66 is inserted into the large intestine 88 of the subject 20 in the second endoscopic examination will be described with reference to Fig. 25. Note that the description here is based on the premise that the camera 66 captures images at a predetermined frame rate, thereby sequentially acquiring endoscopic images 40A by the processor 106.

[0139] 25, first, in step ST10, the first control unit 106A determines whether or not one frame of image has been captured by the camera 66. If one frame of image has not been captured by the camera 66 in step ST10, the determination is negative, and the endoscopic image display process proceeds to step ST16. If one frame of image has been captured by the camera 66 in step ST10, the determination is positive, and the endoscopic image display process proceeds to step ST12.

[0140] In step ST12, the first control unit 106A acquires the endoscopic image 40A obtained by capturing one frame of image using the camera 66 (see FIG. 9 ). After the processing of step ST12 is executed, the endoscopic image display processing proceeds to step ST14.

[0141] In step ST14, the first control unit 106A displays the endoscopic image 40A acquired in step ST12 on the screen 36 (see FIG. 9). After the processing of step ST14 is executed, the endoscopic image display processing proceeds to step ST16.

[0142] In step ST16, the first control unit 106A determines whether or not a condition for terminating the endoscopic image display processing (hereinafter referred to as the "endoscopic image display processing termination condition") has been satisfied. One example of the endoscopic image display processing termination condition is a condition in which an instruction to terminate the endoscopic image display processing has been accepted by the acceptance device 76. In step ST16, if the endoscopic image display processing termination condition has not been satisfied, the determination is negative, and the endoscopic image display processing proceeds to step ST10. In step ST16, if the endoscopic image display processing termination condition has been satisfied, the determination is positive, and the endoscopic image display processing ends.

[0143] Next, an example of the flow of the endoscope operation support processing performed by the processor 106 of the control device 28 in the second endoscopic examination will be described with reference to Figures 26A to 26D. Note that the flow of the endoscope operation support processing shown in Figures 26A to 26D is an example of an "information processing method" according to the technology of the present disclosure. In addition, the description here is based on the premise that the endoscope operation support processing is performed in parallel with the endoscopic image display processing.

[0144] In the endoscope operation support process shown in Fig. 26A, first, in step ST20, the first transmitter / receiver 106C determines whether or not timeout start information (see Fig. 12) has been received by the reception device 76. If the timeout start information has not been received by the reception device 76 in step ST20, the determination is negative, and the determination of step ST20 is made again. If the timeout start information has been received by the reception device 76 in step ST20, the determination is positive, and the endoscope operation support process proceeds to step ST22.

[0145] In step ST22, the first transmitting / receiving unit 106C transmits the request information 144 to the server 78 (see FIG. 12). After the process of step ST22 is executed, the endoscope operation support process proceeds to step ST24.

[0146] When the request information 144 is transmitted from the first transmitting / receiving unit 106C to the server 78 as a result of the execution of the process of step ST22, in response to this, the process of step ST104 is executed in the report management process shown in Fig. 27. As a result, the report 136 is transmitted from the server 78 to the control device 28 (see Fig. 12).

[0147] Therefore, in step ST24, the second control unit 106B determines whether the report 136 transmitted from the server 78 to the control device 28 has been received by the first transmitting / receiving unit 106C. If the report 136 has not been received by the first transmitting / receiving unit 106C in step ST24, the determination is negative, and the determination of step ST24 is made again. If the report 136 has been received by the first transmitting / receiving unit 106C in step ST24, the determination is positive, and the endoscope operation support processing proceeds to step ST26.

[0148] In step ST26, the second control unit 106B generates a previous examination image screen 46 and a subject information screen 146 based on the report 136 received by the first transmitting / receiving unit 106C (see FIG. 13 ). After the process of step ST26 is executed, the endoscope operation support process proceeds to step ST28.

[0149] In step ST28, the second control unit 106B selectively displays the previous examination image screen 46 generated in step ST26 and the subject information screen 146 on the touch panel display 44 in accordance with the screen switching instruction received by the reception device 76 (see FIG. 14 ). After the processing of step ST28 is executed, the endoscope operation support processing proceeds to step ST30.

[0150] In step ST30, the second control unit 106B determines whether or not an image selection instruction (see FIG. 15 ) has been accepted by the touch panel display 44. If an image selection instruction has not been accepted by the touch panel display 44 in step ST30, the determination is negative, and the determination of step ST30 is made again. If an image selection instruction has been accepted by the touch panel display 44 in step ST30, the determination is positive, and the endoscope operation support processing proceeds to step ST32.

[0151] In step ST32, the second control unit 106B acquires the examination result image 50 selected in accordance with the image selection instruction as the reference image 150. The second control unit 106B also acquires metadata 50A of the examination result image 50 selected in accordance with the image selection instruction. The second control unit 106B then stores the reference image 150 and the metadata 50A in the reference image storage area 108A (see FIG. 15 ). After the processing of step ST32 is executed, the endoscope operation support processing proceeds to step ST34.

[0152] In step ST34, the second control unit 106B records timeout information 146C obtained upon timeout of the second endoscopic examination in the report 136, and also displays it on the subject information screen 146 of the touch panel display 44. After the processing of step ST34 is executed, the endoscope operation support processing proceeds to step ST36.

[0153] In step ST36, the second control unit 106B determines whether or not a reference image display instruction (see FIG. 17 ) has been accepted by the acceptance device 76. If a reference image display instruction has not been accepted by the acceptance device 76 in step ST36, the determination is negative, and the determination of step ST36 is made again. If a reference image display instruction has been accepted by the acceptance device 76 in step ST36, the determination is positive, and the endoscope operation support processing proceeds to step ST38.

[0154] In step ST38, the second control unit 106B displays the reference image 150 on the past examination image screen 46 of the touch panel display 44 in a manner that allows it to be distinguished from the other examination result images 50 (see the upper diagram in FIG. 18 ). This allows the reference image 150 displayed on the past examination image screen 46 of the touch panel display 44 to be compared with the endoscopic image 40A displayed on the screen 36 of the display device 30 by executing the processing of step ST14 shown in FIG. 25 . After the processing of step ST38 is executed, the endoscopic operation support processing proceeds to step ST40 shown in FIG. 26B.

[0155] In step ST40, the second control unit 106B determines whether or not the conditions for acquiring the endoscopic image 40A (hereinafter referred to as the "endoscopic image acquisition conditions") are satisfied. A first example of the endoscopic image acquisition condition is that an instruction to acquire the endoscopic image 40A from the camera 66 has been accepted by the acceptance device 76. A second example of the endoscopic image acquisition condition is that a pre-specified timing for acquiring the endoscopic image 40A from the camera 66 has arrived.

[0156] In step ST40, if the endoscopic image acquisition conditions are not satisfied, the determination is negative, and the process proceeds to step ST82 shown in Fig. 26D. In step ST40, if the endoscopic image acquisition conditions are satisfied, the determination is positive, and the endoscopic operation support process proceeds to step ST42.

[0157] In step ST42, the image recognition unit 106D acquires the endoscopic image 40A from the camera 66 (see FIG. 17). The image recognition unit 106D also acquires the reference image 150 from the reference image storage area 108A (see FIG. 17). After the processing of step ST42 is executed, the endoscope operation support processing proceeds to step ST44.

[0158] In step ST44, the image recognition unit 106D performs image recognition processing on the reference image 150 acquired from the reference image storage area 108A in step ST42, thereby identifying the characteristic portion 152 in the reference image 150. After the processing of step ST44 is executed, the endoscope operation support processing proceeds to step ST46.

[0159] In step ST46, the image recognition unit 106D extracts the characteristic portion 152 identified in step ST44 from the reference image 150. After the processing of step ST46 is executed, the endoscope operation support processing proceeds to step ST48.

[0160] In step ST48, the image recognition unit 106D compares the endoscopic image 40A acquired from the camera 66 in step ST42 with the feature portion 152 extracted from the reference image 150 in step ST46 to determine whether the corresponding feature portion 154 is present in the endoscopic image 40A (see FIG. 17 ). If the corresponding feature portion 154 is not present in the endoscopic image 40A in step ST48, the determination is negative, and the endoscope operation support processing proceeds to step ST78 shown in FIG. 26D . If the corresponding feature portion 154 is present in the endoscopic image 40A in step ST48, the determination is positive, and the endoscope operation support processing proceeds to step ST50.

[0161] In step ST50, the second control unit 106B determines whether or not the reference image 150 is being displayed in an enlarged form on the previous examination image screen 46 of the touch panel display 44. If the reference image 150 is being displayed in an enlarged form on the previous examination image screen 46 of the touch panel display 44 in step ST50, the determination is negative, and the endoscope operation support processing proceeds to step ST54. If the reference image 150 is not being displayed in an enlarged form on the previous examination image screen 46 of the touch panel display 44 in step ST50, the determination is positive, and the endoscope operation support processing proceeds to step ST52.

[0162] In step ST52, the second control unit 106B displays an enlarged reference image 150 on the previous examination image screen 46 of the touch panel display 44 (see the lower diagram in FIG. 18 ). After the processing of step ST52 is executed, the endoscope operation support processing proceeds to step ST54.

[0163] In step ST54, the second control unit 106B generates support information 162 based on the endoscopic image 40A and the reference image 150 acquired in step ST42 (see FIG. 19 ). Then, the second control unit 106B displays the generated support information 162 on the screen 36 of the display device 30. After the processing of step ST54 is executed, the endoscope operation support processing proceeds to step ST56.

[0164] The doctor 14 adjusts the position of the camera 66 while referring to the support information 162 displayed on the screen 36 as a result of the processing of step ST54 being executed.

[0165] In step ST56, the second control unit 106B determines whether or not the scale of the endoscopic image 40A matches the scale of the reference image 150. If the scale of the endoscopic image 40A does not match the scale of the reference image 150 in step ST56, the determination is negative, and the endoscope operation support processing proceeds to step ST82 shown in Fig. 26D. If the scale of the endoscopic image 40A matches the scale of the reference image 150 in step ST56, the determination is positive, and the endoscope operation support processing proceeds to step ST58.

[0166] In step ST58, the second control unit 106B generates a notification message 164 and displays the generated notification message 164 on the screen 36, thereby performing processing to notify that the scale of the reference image 150 and the scale of the endoscopic image 40A match (see FIG. 20 ). After the processing of step ST58 is executed, the endoscope operation support processing proceeds to step ST60 shown in FIG. 26C.

[0167] In step ST60 shown in Fig. 26C, the image recognition unit 106D performs pattern matching between the reference image 150 acquired in step ST42 and the scale-adjusted endoscopic image 40A (see Fig. 21). After the processing of step ST60 is executed, the endoscopic operation support processing proceeds to step ST62.

[0168] In step ST62, the image recognition unit 106D determines, based on the result of the pattern matching performed in step ST60, whether or not a corresponding feature portion 154 that matches the feature portion 152 in the reference image 150 is captured in the endoscopic image 40A. If, in step ST62, the corresponding feature portion 154 that matches the feature portion 152 in the reference image 150 is not captured in the endoscopic image 40A, the determination is negative, and the endoscopic operation support processing proceeds to step ST78 shown in Fig. 26D. If, in step ST62, the corresponding feature portion 154 that matches the feature portion 152 in the reference image 150 is captured in the endoscopic image 40A, the determination is positive, and the endoscopic operation support processing proceeds to step ST64.

[0169] In step ST64, the image recognition unit 106D performs image recognition processing on the endoscopic image 40A used for pattern matching in step ST60 (see FIG. 21). After the processing in step ST64 is executed, the endoscope operation support processing proceeds to step ST66.

[0170] In step ST66, the second control unit 106B compares the features 166 obtained by executing the image recognition process in step ST64 with the endoscopic examination information 140 recorded in the report 136 received by the first transmission / reception unit 106C in step ST24 to determine whether or not the reference image 150 and the endoscopic image 40A contain a common lesion. If the reference image 150 and the endoscopic image 40A do not contain a common lesion in step ST66, the determination is negative, and the endoscopic operation support process proceeds to step ST78 shown in FIG. 26D. If the reference image 150 and the endoscopic image 40A contain a common lesion in step ST66, the determination is positive, and the endoscopic operation support process proceeds to step ST68.

[0171] In step ST68, the second control unit 106B notifies that it has been determined that the reference image 150 and the endoscopic image 40A contain a common lesion by displaying a notification message 168 on the screen 36 of the display device 30. After the processing of step ST68 is executed, the endoscopic operation support processing proceeds to step ST70.

[0172] When the notification message 168 is displayed on the screen 36, the doctor 14 checks the endoscopic image 40A and the notification message 168. Then, in step ST70, the second control unit 106B determines whether a confirmation instruction from the doctor 14 has been accepted by the reception device 76. If the confirmation instruction from the doctor 14 has not been accepted by the reception device 76 in step ST70, the determination is negative, and the endoscope operation support processing proceeds to step ST78 shown in Fig. 26D. If the confirmation instruction from the doctor 14 has been accepted by the reception device 76 in step ST70, the determination is positive, and the endoscope operation support processing proceeds to step ST72.

[0173] In step ST72, the second control unit 106B displays a notification message 169 on the screen 36 of the display device 30 to notify that the determination result that the reference image 150 and the endoscopic image 40A contain a common lesion has been confirmed (see FIG. 22 ). After the processing of step ST72 is executed, the endoscopic operation support processing proceeds to step ST74.

[0174] In step ST74, the second control unit 106B generates lesion-related information 172 based on the features 166 obtained by executing the process of step ST64 and the image recognition result 174 obtained by executing the process of step ST44 (see FIG. 24 ). After the process of step ST74 is executed, the endoscope operation support process proceeds to step ST76.

[0175] In step ST76, the second control unit 106B associates the lesion-related information 172 generated in step ST74 with the reference image 150 and the endoscopic image 40A by including the lesion-related information 172 in the metadata 50A and 170. After the processing of step ST76 is executed, the endoscopic operation support processing proceeds to step ST78 shown in FIG.

[0176] In step ST78, the second control unit 106B determines whether or not a recording instruction (see FIG. 23 ) has been accepted by the acceptance device 76. If the acceptance device 76 has not accepted the recording instruction in step ST78, the determination is negative, and the endoscope operation support process proceeds to step ST82. If the acceptance device 76 has accepted the recording instruction in step ST78, the determination is positive, and the endoscope operation support process proceeds to step ST80.

[0177] In step ST80, the second control unit 106B associates the timeout information 146C obtained upon timeout with the endoscopic image 40A by including the timeout information 146C in the metadata 170 (see FIG. 23 ). The second control unit 106B records the endoscopic image 40A and the metadata 170 in the report 136 received by the first transmitting / receiving unit 106C in step ST24. The second control unit 106B also records the metadata 50A, in which the lesion-related information 172 was added in step ST76, and the reference image 150 in the report 136 received by the first transmitting / receiving unit 106C in step ST24 (see FIG. 24 ). After the processing of step ST80 is executed, the endoscopic operation support processing proceeds to step ST82.

[0178] In step ST82, the second control unit 106B determines whether a condition for terminating the endoscope operation support process (hereinafter referred to as the "endoscope operation support process termination condition") has been satisfied. One example of the endoscope operation support process termination condition is a condition in which an instruction to terminate the endoscope operation support process has been accepted by the acceptance device 76. If the endoscope operation support process termination condition has not been satisfied in step ST82, the determination is negative, and the endoscope operation support process proceeds to step ST40 shown in FIG. 26B. If the endoscope operation support process termination condition has been satisfied in step ST82, the determination is positive, and the endoscope operation support process proceeds to step ST84.

[0179] In step ST84, the first transmitting / receiving unit 106C transmits the report 136 received in step ST24 to the server 78 (see FIG. 23). After the process of step ST84 is executed, the endoscope operation support process ends.

[0180] Next, an example of the flow of a report management process performed by the processor 122 of the server 78 will be described with reference to FIG.

[0181] In the report management process shown in Fig. 27, first, in step ST100, the third control unit 122B determines whether or not the request information 144 transmitted from the endoscope device 12 by executing step ST22 shown in Fig. 26A has been received by the second transmission / reception unit 122A. If the request information 144 has not been received by the second transmission / reception unit 122A in step ST100, the determination is negative, and the report management process proceeds to step ST106. If the request information 144 has been received by the second transmission / reception unit 122A in step ST100, the determination is positive, and the report management process proceeds to step ST102.

[0182] In step ST102, the third control unit 122B acquires the report 136 corresponding to the request information 144 received by the second transmitting / receiving unit 122A in step ST100 from the NVM 126 (see FIG. 12). After the process of step ST102 is executed, the report management process proceeds to step ST104.

[0183] In step ST104, the second transmitter / receiver 122A transmits the report 136 acquired from the NVM 126 in step ST102 to the endoscope device 12 (see FIG. 12). After the process of step ST104 is executed, the report management process proceeds to step ST106.

[0184] In step ST106, the third control unit 122B determines whether the report 136 transmitted from the endoscope device 12 by executing step ST84 shown in Fig. 26D has been received by the second transmitting / receiving unit 122A. If the report 136 has not been received by the second transmitting / receiving unit 122A in step ST106, the determination is negative, and the report management process proceeds to step ST110. If the report 136 has been received by the second transmitting / receiving unit 122A in step ST106, the determination is positive, and the report management process proceeds to step ST108.

[0185] In step ST108, the third control unit 122B stores the report 136 received by the second transmitting / receiving unit 122A in step ST106 in the NVM 126 (see FIG. 23). After the process of step ST108 is executed, the report management process proceeds to step ST110.

[0186] In step ST110, the third control unit 122B determines whether a condition for terminating the report management process (hereinafter referred to as a "report management process termination condition") has been satisfied. One example of the report management process termination condition is that an instruction to terminate the report management process has been accepted by the acceptance device 76 or 84. In step ST110, if the report management process termination condition has not been satisfied, the determination is negative, and the report management process proceeds to step ST100. In step ST110, if the report management process termination condition has been satisfied, the determination is positive, and the report management process ends.

[0187] As described above, in the endoscopic system 10, the control device 28 acquires multiple examination result images 50 from the NVM 126 of the server 78 (see FIGS. 11 and 12 ). Then, during the second endoscopic examination, an examination result image 50 selected from the multiple examination result images 50 in accordance with instructions from the physician 14 is displayed as a reference image 150 on the past examination image screen 46 of the touch panel display 44 (see FIG. 18 ). During the second endoscopic examination, an endoscopic image 40A is displayed on the screen 36 of the display device 30, allowing the user to compare the reference image 150 displayed on the past examination image screen 46 with the endoscopic image 40A displayed on the screen 36 of the display device 30. This allows the user to understand the change from the area that was the main examination target in the first endoscopic examination (e.g., characteristic portion 152) to the area that will be the main examination target in the second endoscopic examination (e.g., corresponding characteristic portion 154).

[0188] Furthermore, in the endoscope system 10, the reference image 150 is displayed on the past examination image screen 46 at the timeout stage during the implementation period of the second endoscopy. Therefore, at the timeout stage during the implementation period of the second endoscopy, the user can understand the change from the area that was the main examination target in the first endoscopy (e.g., characteristic portion 152) to the area that will be the main examination target in the second endoscopy (e.g., corresponding characteristic portion 154).

[0189] Furthermore, in the endoscope system 10, when the second endoscopic examination times out, timeout information 146B is displayed on the subject information screen 146 on the touch panel display 44 (see FIG. 16 ). The timeout information 146B is information obtained during the timeout that occurred during the implementation period of the first endoscopic examination. Therefore, when the second endoscopic examination times out, the user can grasp the timeout information 146B obtained during the timeout that occurred during the implementation period of the first endoscopic examination.

[0190] Furthermore, in the endoscopic system 10, subject identification information 138 is acquired from the server 78 in a state recorded in a report 136 as information required for a timeout that occurs during the implementation period of the second endoscopic examination. Then, subject identification information 146A, which is the same information as the subject identification information 138, is displayed on a subject information screen 146 of the touch panel display 44. Therefore, compared to a case where the subject identification information 146A is manually input into the tablet terminal 32 and then the manually input subject identification information 146A is displayed on the touch panel display 44 of the tablet terminal 32, the subject identification information 138 can be provided to the user more quickly as information required for a timeout that occurs during the implementation period of the second endoscopic examination.

[0191] Furthermore, in the endoscopic system 10, when a timeout that takes place during the implementation period of the second endoscopic examination is completed, the report 136 used in the timeout is sent to the server 78 and stored in the NVM 126 of the server 78. Therefore, in a timeout that takes place during the implementation period of an endoscopic examination that takes place after the second endoscopic examination (e.g., the next endoscopic examination), it is possible to utilize information (e.g., subject identification information 146A) recorded in the report 136 used in the timeout that takes place during the implementation period of the second endoscopic examination.

[0192] Furthermore, in the endoscope system 10, metadata 50A is associated with the inspection result image 50 as data accompanying the inspection result image 50. The metadata 50A includes the same information as the timeout information 142 obtained in the timeout performed during the implementation period of the first endoscopic examination. Therefore, a service (e.g., display of information related to the timeout information 142) using the timeout information 142 obtained in the timeout performed during the implementation period of the first endoscopic examination can be provided to a user who uses the inspection result image 50 (e.g., a user who observes the inspection result image 50) as information related to the inspection result image 50.

[0193] Furthermore, in the endoscopic system 10, metadata 170 is associated with the endoscopic image 40A as data accompanying the endoscopic image 40A. The metadata 170 includes timeout information 146C obtained upon timeout of the second endoscopic examination (see FIG. 23 ). Therefore, a service using the timeout information 146C obtained upon timeout of the second endoscopic examination (e.g., display of information related to the timeout information 146C) can be provided to a user using the endoscopic image 40A (e.g., a user observing the endoscopic image 40A) as information related to the endoscopic image 40A.

[0194] Furthermore, in the endoscope system 10, an examination result image 50 selected from the plurality of examination result images 50 in accordance with an instruction from the doctor 14 is displayed as a reference image 150 on the past examination image screen 46 of the touch panel display 44 (see FIGS. 15 and 18 ). Therefore, the user can observe the examination result image 50 intended by the doctor 14 as the reference image 150.

[0195] Furthermore, in the endoscopic system 10, when the image recognition unit 106D determines that the corresponding feature portion 154 is present in the endoscopic image 40A, the second control unit 106B notifies the user that the corresponding feature portion 154 is present in the endoscopic image 40A by enlarging and displaying the reference image 150 on the past examination image screen 46 (see the lower diagram in FIG. 18 ). This allows the user to perceive that a location corresponding to the feature portion 152 in the reference image 150 is present in the endoscopic image 40A displayed on the screen 36 of the display device 30.

[0196] Furthermore, in the endoscope system 10, the second control unit 106B performs processing to assist in matching the imaging conditions for the second endoscopic examination to those for the first endoscopic examination based on the reference image 150 and the endoscopic image 40A (see FIG. 19 ). This improves the accuracy of comparing the reference image 150 and the endoscopic image 40A compared to when the imaging conditions for the first endoscopic examination and the imaging conditions for the second endoscopic examination are not matched.

[0197] Furthermore, in the endoscopic system 10, support information 162 is output to the display device 30 as information required to adjust the imaging conditions for the second endoscopic examination to those for the first endoscopic examination, and the support information 162 is displayed on the screen 36 of the display device 30 (see FIG. 19 ). This makes it possible to support the task of adjusting the imaging conditions for the second endoscopic examination in order to improve the accuracy of comparing the reference image 150 and the endoscopic image 40A.

[0198] Furthermore, in the endoscope system 10, support information 162 is derived based on the result of comparing the characteristic portion 152 shown in the reference image 150 with the corresponding characteristic portion 154 shown in the endoscopic image 40A (see FIG. 19 ), and the derived support information 162 is displayed on the screen 36 of the display device 30. Therefore, compared to when support information 162 derived without using the characteristic portion 152 and the corresponding characteristic portion 154 is displayed on the screen 36 of the display device 30, the imaging conditions used to image the characteristic portion 152 can be matched with the imaging conditions used to image the corresponding characteristic portion 154 with high accuracy.

[0199] Furthermore, in the endoscopic system 10, a notification is provided when the imaging conditions for the second endoscopic examination match those for the first endoscopic examination. For example, a notification message 164 is displayed on the screen 36 of the display device 30 as a message notifying that the scale of the reference image 150 and the scale of the endoscopic image 40A match (see FIG. 20 ). This allows the user to perceive that the imaging conditions for the second endoscopic examination match those for the first endoscopic examination (for example, that the scale of the reference image 150 and the scale of the endoscopic image 40A match).

[0200] Furthermore, in the endoscopic system 10, it is determined based on the reference image 150 and the endoscopic image 40A whether a common lesion is captured in the reference image 150 and the endoscopic image 40A (see FIG. 21 ). Then, a notification message 168 indicating the determination result is displayed on the screen 36 of the display device 30 (see FIG. 21 ). Therefore, a user or the like (for example, a user and / or various devices such as the control device 28) can perform processing depending on whether a common lesion is captured in the reference image 150 and the endoscopic image 40A.

[0201] Furthermore, in the endoscopic system 10, when it is determined that a common lesion is captured in the reference image 150 and the endoscopic image 40A and a confirmation instruction from the doctor 14 is received by the reception device 76, a notification message 169 is displayed on the screen 36 of the display device 30 to notify the user that the determination result that a common lesion is captured in the reference image 150 and the endoscopic image 40A has been confirmed (see FIG. 22 ). This allows the user to perceive that the determination result that a common lesion is captured in the reference image 150 and the endoscopic image 40A has been confirmed.

[0202] Furthermore, in the endoscopic system 10, when a common lesion is captured in the reference image 150 and the endoscopic image 40A, lesion-related information 172 is generated as information about the common lesion (see FIG. 24 ). This allows the user or the like to perform processing based on the lesion-related information 172 generated as information about the common lesion when a common lesion is captured in the reference image 150 and the endoscopic image 40A.

[0203] Furthermore, in the endoscopic system 10, current size information 172D and transition identification information 172E are included in the lesion-related information 172 as information relating to the size of a lesion common to the reference image 150 and the endoscopic image 40A (see FIG. 24 ). This allows the user, etc., to perform processing based on the current size information 172D and transition identification information 172E generated as information relating to the size of a lesion common to the reference image 150 and the endoscopic image 40A when a lesion common to the reference image 150 and the endoscopic image 40A is captured.

[0204] Furthermore, in the endoscopic system 10, transition identification information 172E is included in the lesion-related information 172 as information that can identify the transition over time of the size of a lesion common to the reference image 150 and the endoscopic image 40A (see FIG. 24 ). This allows the user or the like to perform processing based on the transition identification information 172E that has been generated as information that can identify the transition over time of the size of a lesion common to the reference image 150 and the endoscopic image 40A.

[0205] Furthermore, in the endoscope system 10, the transition specifying information 172E is derived based on the size of the lesion shown in the reference image 150 and / or the size of the lesion shown in the endoscopic image 40A. This allows the transition specifying information 172E to be derived with higher accuracy than when the transition specifying information 172E is derived only from information unrelated to the size of the lesion.

[0206] Furthermore, in the endoscope system 10, the transition identification information 172E is derived based on the size of the lesion shown in the reference image 150 and / or the size and type of the lesion shown in the endoscopic image 40A. This allows the transition identification information 172E to be derived with higher accuracy than when the transition identification information 172E is derived while ignoring the type of lesion.

[0207] Furthermore, in the endoscope system 10, type information 172A, number information 172B, and aspect information 172C are included in the lesion-related information 172 (see FIG. 24 ). This allows the user or the like to perform processing based on the type of lesion, the number of lesions, and the aspect of the lesion when a common lesion is captured in the reference image 150 and the endoscopic image 40A.

[0208] Furthermore, in the endoscopic system 10, lesion-related information 172 is associated with the reference image 150 and the endoscopic image 40A, which show a common lesion (see FIG. 24 ). This allows the user, etc., to perform processing on the reference image 150 based on the lesion-related information 172 associated with the reference image 150. Furthermore, when processing the endoscopic image 40A, the user, etc., can perform processing based on the lesion-related information 172 associated with the endoscopic image 40A.

[0209] Furthermore, the endoscope system 10 generates a report 136 in which the lesion-related information 172, the reference image 150, and the endoscopic image 40A are recorded (see FIGS. 23 and 24 ). This allows the user or the like to identify the lesion-related information 172 and the reference image 150, which correspond to each other, through the report 136. Furthermore, the user or the like can identify the lesion-related information 172 and the endoscopic image 40A, which correspond to each other, through the report 136.

[0210] Furthermore, in the endoscope system 10, the reference image 150 is displayed on the touch panel display 44 of the tablet terminal 32 (see FIG. 18), and the endoscopic image 40A is displayed on the display device 30 (see FIG. 9). The touch panel display 44 and the display device 30 are arranged side by side (see FIG. 1). That is, the reference image 150 and the endoscopic image 40A are displayed so that they can be compared. This allows the user to visually compare the reference image 150 and the endoscopic image 40A.

[0211] [First Modification] In the above embodiment, an example has been described in which the scale of the endoscopic image 40A is adjusted to match the scale of the reference image 150 by adjusting the position of the camera 66 so that the difference 160 becomes zero, but the technology of the present disclosure is not limited to this. For example, as shown in Fig. 28 , the second control unit 106B may change the display mode of the reference image 150 depending on the positional relationship between a characteristic portion 152 shown in the reference image 150 and a corresponding characteristic portion 154 shown in the endoscopic image 40A.

[0212] 28 shows an example in which the scale of the reference image 150 is changed so that the difference 160 becomes zero under the first premise described in the above embodiment. Here, a case in which the difference 160 becomes zero under the first premise is illustrated, but this is merely an example, and the scale of the reference image 150 may be changed so that the difference 160 becomes a value other than zero that is specified in advance. Furthermore, even in the case of the second premise described in the above embodiment, the display mode of the reference image 150 may be changed in accordance with the positional relationship between the feature portion 152 shown in the reference image 150 and the corresponding feature portion 154 shown in the endoscopic image 40A using the difference calculated in the same manner as in the above embodiment.

[0213] 28 , the second control unit 106B generates a notification message 176 when the positional relationship between the feature portion 152 in the reference image 150 and the corresponding feature portion 154 in the endoscopic image 40A becomes a predetermined positional relationship (here, as an example, when the difference 160 becomes zero). The second control unit 106B then displays the generated notification message 176 on the past examination image screen 46. The notification message 176 is a message notifying that the positional relationship between the feature portion 152 in the reference image 150 and the corresponding feature portion 154 in the endoscopic image 40A has become a predetermined positional relationship. The display of the notification message 176 on the past examination image screen 46 notifies the user that the positional relationship between the feature portion 152 in the reference image 150 and the corresponding feature portion 154 in the endoscopic image 40A has become a predetermined positional relationship. The process of notifying that the positional relationship between the characteristic part 152 shown in the reference image 150 and the corresponding characteristic part 154 shown in the endoscopic image 40A has become a predetermined positional relationship is an example of a "second notification process" related to the technology disclosed herein.

[0214] In this way, in this first modified example, the display mode of the reference image 150 is changed according to the positional relationship between the characteristic part 152 shown in the reference image 150 and the corresponding characteristic part 154 shown in the endoscopic image 40A. Therefore, even if the doctor 14 does not operate the camera 66, the characteristic part 152 shown in the reference image 150 and the corresponding characteristic part 154 shown in the endoscopic image 40A can be aligned.

[0215] Furthermore, in the first modified example, the notification message 176 is displayed on the previous examination image screen 46 to notify the user that the positional relationship between the characteristic part 152 in the reference image 150 and the corresponding characteristic part 154 in the endoscopic image 40A has become the default positional relationship. Therefore, the user can perceive that the positional relationship between the characteristic part 152 in the reference image 150 and the corresponding characteristic part 154 in the endoscopic image 40A has become the default positional relationship.

[0216] In the first modified example, the scale of the reference image 150 is changed. However, the optical characteristics of the reference image 150 may be adjusted to match the optical characteristics of the endoscopic image 40A.

[0217] [Second Modification] In the above-described first modification, an example in which the scale of the reference image 150 is changed has been described, but the technology of the present disclosure is not limited to this. For example, as shown in FIG. 29 , the second control unit 106B may change the composition of the reference image 150 (e.g., the size and position of geometric elements included in the image) in accordance with the endoscopic image 40A. In this case, for example, the second control unit 106B compares the endoscopic image 40A with the reference image 150 and, based on the comparison result, changes the composition of the reference image 150 so that the composition of the reference image 150 matches that of the endoscopic image 40A. The reference image 150 with the changed composition is displayed on the past examination image screen 46 by the second control unit 106B.

[0218] Furthermore, when the composition of the reference image 150 matches the endoscopic image 40A, the second control unit 106B generates a notification message 178 and displays the generated notification message 178 on the previous examination image screen 46. The notification message 178 is a message notifying that the composition of the reference image 150 matches the endoscopic image 40A. Therefore, by displaying the notification message 178 on the previous examination image screen 46, it is notified that the composition of the reference image 150 matches the endoscopic image 40A.

[0219] In this way, in the second modified example, the composition of the reference image 150 is changed in accordance with the endoscopic image 40 A. Therefore, compared to when the composition of the reference image 150 is fixed, the user or the like can easily distinguish the difference between the reference image 150 and the endoscopic image 40 A.

[0220] [Third Modification] In the above embodiment, an example was described in which an inspection result image 50 selected in accordance with an image selection instruction is treated as a reference image 150 that satisfies a specific condition. However, the technology of the present disclosure is not limited to this. For example, as shown in FIG. 30 , an inspection result image 50 selected by performing image recognition processing on multiple inspection result images 50 recorded in a report 136 by the image recognition unit 106D may be treated as a reference image 150 that satisfies a specific condition, as in the above embodiment. The example shown in FIG. 30 illustrates a mode in which, among the multiple inspection result images 50, an inspection result image 50 that includes a characteristic portion 152 is selected as a reference image 150 that satisfies a specific condition through image recognition processing and stored in the reference image storage area 108A.

[0221] In this way, in the third modified example, the test result images 50 selected by performing image recognition processing on the multiple test result images 50 are treated as reference images 150, which are images that satisfy specific conditions, in the same manner as in the above embodiment. Therefore, even if the doctor 14 is not required to select the test result image 50 that shows the characteristic portion 152 from the multiple test result images 50, the test result image 50 that shows the characteristic portion 152 is selected as the reference image 150.

[0222] In the third modified example, the test result image 50 selected by performing image recognition processing on the multiple test result images 50 is treated as the reference image 150 that satisfies a specific condition, as in the above embodiment. However, the technology disclosed herein is not limited to this. For example, the test result image 50 selected by performing image recognition processing on the multiple test result images 50 may be displayed on the past test image screen 46 in a manner distinguishable from other test result images 50, and when the distinguishably displayed test result image 50 is selected in accordance with an image selection instruction, the test result image 50 selected in accordance with the image selection instruction may be treated as the reference image 150 that satisfies a specific condition, as in the above embodiment. In this case, the test result image 50 that includes the characteristic portion 152 intended by the doctor 14 is selected as the reference image 150, even if the doctor 14 is not required to select the test result image 50 that includes the characteristic portion 152 from the multiple test result images 50.

[0223] [Fourth Modification] In the above third modification, an example has been described in which an inspection result image 50 selected by performing image recognition processing on a plurality of inspection result images 50 is selected as a reference image 150 that satisfies a specific condition, but the technology of the present disclosure is not limited to this. For example, as shown in Fig. 31 , an inspection result image 50 selected by performing information processing on metadata 50A assigned to each of a plurality of inspection result images 50 recorded in a report 136 and selected in accordance with an image selection instruction may be selected as a reference image 150 that satisfies a specific condition.

[0224] As an example, as shown in FIG. 31 , the metadata 50A of an examination result image 50 containing a characteristic portion 152 includes an identifier 50A1. The identifier 50A1 is information that uniquely identifies the examination result image 50 as containing the characteristic portion 152. The second control unit 106B identifies the metadata 50A containing the identifier 50A1 from the multiple pieces of metadata 50A corresponding to the multiple examination result images 50 recorded in the report 136. The second control unit 106B then displays the examination result image 50 to which the metadata 50A containing the identifier 50A1 is assigned on the past examination image screen 46 in a display manner that allows the examination result image 50 to be distinguished from other examination result images 50. One example of a display manner that allows the examination result image 50 to be distinguished from other examination result images 50 is to highlight the outline of the examination result image 50 to which the metadata 50A containing the identifier 50A1 is assigned. The second control unit 106B also displays a guidance message 180 on the past examination image screen 46. The guidance message 180 is a message that asks the doctor 14 whether or not it is acceptable to use the examination result image 50 displayed on the past examination image screen 46 in a display format that is distinguishable from other examination result images 50 as the reference image 150.

[0225] When an inspection result image 50 displayed on the past inspection image screen 46 in a display manner that is distinguishable from other inspection result images 50 is selected in accordance with an image selection instruction, the inspection result image 50 selected in accordance with the image selection instruction is selected as a reference image 150, which is an image that satisfies specific conditions.

[0226] In this way, in the fourth modified example, the test result images 50 are selected by performing information processing on the metadata 50A assigned to each of the multiple test result images 50, and the test result images 50 selected in accordance with the image selection instruction are selected as the reference images 150, which are images that satisfy specific conditions. Therefore, even if the doctor 14 is not required to select the test result images 50 that show the characteristic portions 152 from the multiple test result images 50, the test result images 50 that show the characteristic portions 152 that the doctor 14 intends are selected as the reference images 150.

[0227] 31 , the test result image 50 selected in accordance with the image selection instruction is selected as the reference image 150, which is an image that satisfies a specific condition. However, this is merely an example. For example, metadata 50A containing an identifier 50A1 may be identified from multiple metadata 50A corresponding to multiple test result images 50 recorded in the report 136, and the test result image 50 to which the identified metadata 50A is assigned may be selected as the reference image 150, which is an image that satisfies a specific condition. In this case, the test result image 50 that includes the characteristic portion 152 is selected as the reference image 150 without requiring the doctor 14 to select the test result image 50 that includes the characteristic portion 152 from the multiple test result images 50.

[0228] The metadata 50A may also include endoscopic examination information 140. In this case, the second control unit 106B determines whether the examination result image 50 contains the characteristic portion 152 by performing information processing on the endoscopic examination information 140 in the metadata 50A. That is, when the endoscopic examination information 140 in the metadata 50A contains information on the lesion corresponding to the characteristic portion 152, the second control unit 106B identifies the examination result image 50 corresponding to the metadata 50A including information on the lesion corresponding to the characteristic portion 152 as an image containing the characteristic portion 152. Here, the examination result image 50 corresponding to the metadata 50A including information on the lesion corresponding to the characteristic portion 152 is selected as the reference image 150. In this case, the examination result image 50 containing the characteristic portion 152 is selected as the reference image 150 without requiring the doctor 14 to select the examination result image 50 containing the characteristic portion 152 from multiple examination result images 50.

[0229] Furthermore, when the second control unit 106B identifies an examination result image 50 corresponding to the metadata 50A including information about the lesion corresponding to the characteristic portion 152, and the identified examination result image 50 is selected in accordance with an image selection instruction, the examination result image 50 corresponding to the metadata 50A including information about the lesion corresponding to the characteristic portion 152 may be selected as the reference image 150. In this case, even if the doctor 14 is not required to select an examination result image 50 including the characteristic portion 152 from multiple examination result images 50, the examination result image 50 including the characteristic portion 152 intended by the doctor 14 is selected as the reference image 150.

[0230] Furthermore, the metadata 50A may include the same information as the timeout information 142 or a part of the timeout information 142. In this case, the second control unit 106B performs information processing on the same information as the timeout information 142 or a part of the timeout information 142, thereby determining whether the test result image 50 contains the characteristic portion 152. That is, when the same information as the timeout information 142 or a part of the timeout information 142 contains information about the lesion corresponding to the characteristic portion 152, the second control unit 106B identifies the test result image 50 corresponding to the metadata 50A containing information about the lesion corresponding to the characteristic portion 152 as an image containing the characteristic portion 152. In this case, the same effect as described above can be obtained.

[0231] [Other Modifications] In the above embodiment, an example was described in which multiple examination result images 50 obtained in the first endoscopic examination are displayed on the previous examination image screen 46. However, the technology of the present disclosure is not limited to this. For example, multiple types of previous examination image screens 46 may be selectively displayed in response to a selection instruction from the user.

[0232] For example, in this case, the user selects the previous endoscopic examination performed on the subject 20 and the endoscopic examination before the previous one (or the endoscopic examination before the previous one) performed on the subject 20. Then, in response to a selection instruction from the user, a past examination image screen 46 including a plurality of examination result images 50 obtained in the previous endoscopic examination performed on the subject 20 and a past examination image screen 46 including a plurality of examination result images 50 obtained in the endoscopic examination before the previous one (or the endoscopic examination before the previous one) performed on the subject 20 are switched and displayed. Switching between the display of the plurality of types of past examination image screens 46 may be performed in response to an operation using voice recognition.

[0233] In addition, multiple test result images 50 obtained in the previous endoscopic examination performed on the subject 20 and multiple test result images 50 obtained in the endoscopic examination performed on the subject 20 two years ago (or the endoscopic examination before that) may be displayed side by side (e.g., in a state where they can be compared).

[0234] In the above embodiment, an example (see FIGS. 1 and 13 to 15 ) in which a plurality of test result images 50 and subject identification information 52 are displayed side by side on the past test image screen 46 has been described, but the technology of the present disclosure is not limited to this. For example, a plurality of test result images 50 and subject identification information 52 may be selectively displayed in response to a selection instruction from a user. In this case, too, the display may be switched in response to an operation using voice recognition.

[0235] In the above embodiment, an example has been described in which the control device 28 acquires the subject identification information 138 and the endoscopic examination information 140 (see FIG. 11 ) from the server 78. However, the technology of the present disclosure is not limited to this. For example, if an instruction from a user is received by the tablet terminal 44 or the like on the endoscopic device 12 before a timeout occurs, the control device 28 may acquire various information (e.g., the subject identification information 138 and the endoscopic examination information 140) from the server 78 and / or the tablet terminal 44 or the like in response to the instruction from the user.

[0236] In the above embodiment, an example is given in which the previous examination image screen 46 is displayed on the tablet terminal 44, but the technology of the present disclosure is not limited to this. For example, a monitor display button may be displayed on the touch panel display 44 of the tablet terminal 44, and when the monitor display button is turned on, a sub-screen corresponding to the screen displayed on the touch panel display 44 may be displayed or the display mode may be changed on the display device 30 in conjunction with a change in the display content of the touch panel display 44.

[0237] For example, the display device 30 may be configured to display a screen equivalent to the past examination image screen 46 (e.g., a screen in which the past examination image screen 46 is reduced) or part of the information included in the past examination image screen 46 (e.g., at least part of the multiple examination result images 50 and / or at least part of the subject identification information 52) on the condition that a monitor display button displayed on the touch panel display 44 of the tablet terminal 44 is turned on. An example of a location where the information is displayed on the display device 30 is an edge of the screen of the display device 30 (e.g., the right edge when viewed from the front).

[0238] Furthermore, when the monitor display button is turned on, at least one of the multiple test result images 50 (e.g., the reference image 150) may be enlarged and displayed on the touch panel display 44 of the tablet terminal 44. In conjunction with this, the display device 30 may also enlarge and display the same test result image 50 as the enlarged test result image 50 displayed on the tablet terminal 44. For example, when multiple test result images 50 are displayed on a sub-screen of the display device 30, the sub-screen may be enlarged. As a result, the multiple test result images 50 are enlarged and displayed on the display device 30. Alternatively, the size of the sub-screen may be left unchanged, and only a specific test result image 50 (e.g., the reference image 140) may be enlarged and displayed on the sub-screen. Furthermore, the sub-screen may include subject identification information 52, etc., and the display mode of the subject identification information 52, etc. may be changed in response to a change in the display mode of the specific test result image 50. In this case, for example, when a specific test result image 50 is enlarged and displayed, information other than the specific test result image 50 (such as the subject identification information 52) may be reduced, enlarged, or hidden accordingly. When the monitor display button is turned off, the original screen (for example, the screen shown in FIG. 1) is displayed on the display device 30 and the touch panel display 44.

[0239] In the above embodiment, an example in which the control device processing is performed by the control device 28 has been described, but the technology of the present disclosure is not limited to this. For example, a device that performs the control device processing may be provided outside the control device 28. An example of a device that is provided outside the control device 28 is a server 78. The server 78 may be realized by cloud computing. Cloud computing is merely an example, and for example, the server 78 may be realized by a mainframe, or by network computing such as fog computing, edge computing, or grid computing.

[0240] Here, the server 78 is given as an example of a device provided external to the control device 28, but this is merely an example, and at least one personal computer or the like may be used instead of the server 78. Furthermore, the control device processing may be performed in a distributed manner by a plurality of devices including the control device 28 and a device provided external to the control device 28. Furthermore, the device provided external to the control device 28 may be the endoscope processing device 22.

[0241] In the above embodiment, an example has been described in which the endoscopic image 40A is displayed on the display device 30 and the previous examination image screen 46 and the subject information screen 146 are displayed on the touch panel display 44 of the tablet terminal 32, but the technology of the present disclosure is not limited to this. For example, the endoscopic image 40A, the previous examination image screen 46, and the subject information screen 146 may be displayed on the display device 30. In this case, the endoscopic image 40A, the previous examination image screen 46, and the subject information screen 146 may be displayed side by side, or the previous examination image screen 46 and the subject information screen 146 may be selectively displayed so as to be contrasted with the endoscopic image 40A.

[0242] The endoscopic image 40A may also be displayed on the touch panel display 44 of the tablet terminal 32. In this case, the endoscopic image 40A, the previous examination image screen 46, and the subject information screen 146 may be displayed side by side, or the previous examination image screen 46 and the subject information screen 146 may be selectively displayed in a state where they can be compared with the endoscopic image 40A.

[0243] Furthermore, the endoscopic image 40A, the previous examination image screen 46, and the subject information screen 146 may be displayed on separate displays.

[0244] In the above embodiment, there is only one reference image 150, but this is merely an example and there may be multiple reference images 150. In this case, for example, multiple reference images 150 are selected from multiple inspection result images 50 in the manner described above.

[0245] In the above embodiment, an example has been described in which the report 136 is stored in the NVM 134 of the server 78, but the technology of the present disclosure is not limited to this. For example, the report 136 may be stored in the NVM 98 of the endoscope processing device 22, the NVM 110 of the control device 28, and / or the memory of the tablet terminal 32.

[0246] In the above embodiment, the subject identification information 138 and the endoscopic examination information 140 are recorded in the report 136, and the processor 106 acquires this information from the report 136. However, the technology of the present disclosure is not limited to this. At least a portion of the subject identification information 138 and / or at least a portion of the endoscopic examination information 140 may be stored in the NVM 98 of the endoscopic processing device 22, the NVM 110 of the control device 28, and / or the memory of the tablet terminal 32. In this case, for example, at least a portion of the subject identification information 138 and / or at least a portion of the endoscopic examination information 140 may be acquired by the processor 106 from the NVM 98 of the endoscopic processing device 22, the NVM 110 of the control device 28, and / or the memory of the tablet terminal 32.

[0247] In the above embodiment, an example in which endoscopic system processing is performed using endoscopic images 40 acquired by the camera 66 has been described, but the technology of the present disclosure is not limited to this. For example, endoscopic system processing may be performed using ultrasound images acquired by an ultrasound endoscope.

[0248] In the above embodiment, an endoscope 18 used for the large intestine 88 is exemplified as a lower gastrointestinal endoscope, but this is merely one example, and the technology of the present disclosure can also be applied to endoscopes used for lower gastrointestinal organs other than the large intestine 88, such as upper gastrointestinal endoscopes, biliary-pancreatic endoscopes, or bronchial endoscopes.

[0249] In the above embodiment, an example has been given in which a location where treatment (e.g., therapy) is to be performed inside the body of the subject 20 during the second endoscopy, but the technology of the present disclosure is not limited to this. For example, in order to confirm the effect of treatment (e.g., therapy) during the second endoscopy, a location where a previous treatment (e.g., therapy) was performed may be identified.

[0250] Examples of the confirmation of the effect of the treatment include confirmation of the effect of polypectomy and / or confirmation of the effect of treatment to remove Helicobacter pylori.

[0251] In the above embodiment, an example was described in which the endoscopic image display program 130 and the endoscopic operation support program 132 are stored in the NVM 110 and the report management program 134 is stored in the NVM 126, but the technology of the present disclosure is not limited to this. For example, the endoscopic system program may be stored in a portable storage medium such as an SSD or USB memory. The storage medium is a non-transitory computer-readable storage medium. The endoscopic system program stored in the storage medium is installed in the computer 80, 90, and / or 102. The processors 94, 106, and / or 122 execute endoscopic system processing in accordance with the endoscopic system program.

[0252] In the above embodiment, the computers 80, 90, and / or 102 are exemplified, but the technology of the present disclosure is not limited to this, and devices including ASICs, FPGAs, and / or PLDs may be applied instead of the computers 80, 90, and / or 102. Furthermore, instead of the computers 80, 90, and / or 102, a combination of hardware and software configurations may be used.

[0253] The hardware resources for executing the endoscope system processing described in the above embodiments can be various processors, as listed below. Examples of processors include general-purpose processors that function as hardware resources for executing endoscope system processing by executing software, i.e., programs. Examples of processors include dedicated electronic circuits, such as FPGAs, PLDs, and ASICs, which are processors with circuit configurations designed specifically for executing specific processes. Each processor has built-in or connected memory, and each processor uses the memory to execute endoscope system processing.

[0254] The hardware resource for executing the endoscope system processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a processor and an FPGA). Also, the hardware resource for executing the endoscope system processing may be a single processor.

[0255] Examples of a system configured with a single processor include: a first configuration in which one or more processors are combined with software to form a single processor, which functions as a hardware resource for executing endoscope system processing; a second configuration in which a processor is used to implement the functions of the entire system, including multiple hardware resources for executing endoscope system processing, on a single IC chip, as typified by SoCs; and endoscope system processing is thus implemented using one or more of the various processors described above as hardware resources.

[0256] Furthermore, the hardware structure of these various processors can be, more specifically, electronic circuits that combine circuit elements such as semiconductor devices. The above-described endoscope system processing is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0257] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure.

[0258] Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or elements may be substituted in the above-described and illustrated contents, without departing from the spirit of the technology of the present disclosure. Furthermore, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, explanations of common general technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure have been omitted from the above-described and illustrated contents.

[0259] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0260] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A processor is provided. The processor, acquiring a plurality of first endoscopic examination images from a storage device in which a plurality of first endoscopic examination images obtained by imaging a plurality of sites in a first endoscopic examination that is a previous endoscopic examination have been accumulated; At a timeout stage during the implementation period of the second endoscopy, which is the current endoscopy, at least one first endoscopy image among the plurality of first endoscopy images that satisfies a specific condition is displayed on a display device as a reference image. Information processing device.

2. The processor causes the display device to display first information obtained during the implementation period of the first endoscopy at a timeout stage performed during the implementation period of the second endoscopy. The information processing device according to claim 1 .

3. The processor, acquiring second information required for a timeout performed during the implementation period of the second endoscopy from a device in which the second information is stored; The acquired second information is displayed on the display device.

3. The information processing device according to claim 1 or 2.

4. A processor, The processor, acquiring a plurality of first endoscopic examination images from a storage device in which a plurality of first endoscopic examination images obtained by imaging a plurality of sites in a first endoscopic examination that is a previous endoscopic examination have been accumulated; During a period during which a second endoscopic examination is being performed, at least one first endoscopic examination image that satisfies a specific condition among the plurality of first endoscopic examination images is displayed on a display device as a reference image; acquiring second information required for a timeout performed during the implementation period of the second endoscopy from a device in which the second information is stored; The acquired second information is displayed on the display device. Information processing device.

5. The processor stores the second information in the storage when a timeout that is performed within a period during which the second endoscopy is performed is completed. The information processing device according to claim 4.

6. The first endoscopy image is associated with information obtained upon timeout of the first endoscopy. The information processing device according to claim 1 .

7. The second endoscopy image obtained by imaging in the second endoscopy is associated with information obtained at the timeout of the second endoscopy. The information processing device according to claim 1 .

8. The specific condition is a condition that is selected in accordance with an instruction accepted by a reception device. The information processing device according to claim 1 .

9. The specific condition is a condition that is selected by performing image recognition processing on the plurality of first endoscopic examination images and / or information processing on metadata of the plurality of first endoscopic examination images. The information processing device according to claim 1 .

10. The specific condition is selected by performing image recognition processing on the plurality of first endoscopic examination images and / or information processing on metadata of the plurality of first endoscopic examination images. The condition is that the item is selected and selected in accordance with the instructions received by the receiving device. The information processing device according to claim 1 .

11. The metadata includes endoscopic examination information obtained during the implementation period of the first endoscopic examination. The information processing device according to claim 9.

12. The endoscopy information includes information obtained upon timeout of the first endoscopy. The information processing device according to claim 11.

13. The processor performs a first notification process to notify when a location corresponding to a characteristic portion in the reference image is captured in a second endoscopic examination image obtained by imaging in the second endoscopic examination. The information processing device according to claim 1 .

14. The first notification process includes a process of changing a display mode of the reference image. The information processing device according to claim 13.

15. The processor changes the display mode of the reference image according to a positional relationship between a characteristic part in the reference image and a part corresponding to the characteristic part that is captured in a second endoscopic examination image obtained by imaging in the second endoscopic examination. The information processing device according to claim 1 .

16. The processor performs a second notification process for notifying when the positional relationship becomes a default positional relationship. The information processing device according to claim 15.

17. The processor performs a support process for supporting adjustment of an imaging condition of the second endoscopy to an imaging condition of the first endoscopy, based on the reference image and a second endoscopy image obtained by imaging in the second endoscopy. The information processing device according to claim 1 .

18. The support process includes an output process for outputting support information required to match the imaging conditions of the second endoscopy to the imaging conditions of the first endoscopy. The information processing device according to claim 17.

19. The support information is derived based on a result of comparing a first feature portion shown in the reference image with a second feature portion shown in the second endoscopic inspection image. The information processing device according to claim 18.

20. The support process includes a third notification process of notifying when an image capturing condition of the second endoscopy coincides with an image capturing condition of the first endoscopy. The information processing device according to claim 17.

21. The processor changes a composition of the reference image in accordance with a second endoscopy image obtained by capturing an image in the second endoscopy. The information processing device according to claim 1 .

22. The processor determines whether or not a common lesion is captured in the reference image and the second endoscopic examination image based on the reference image and the second endoscopic examination image obtained by capturing the image in the second endoscopic examination. The information processing device according to claim 1 .

23. The processor performs a fourth notification process for notifying that the determination result has been confirmed when the processor determines that a common lesion is captured in the reference image and the second endoscopic examination image and an instruction to confirm the determination result has been received by the reception device. The information processing device according to claim 20.

24. The processor generates lesion-related information regarding a lesion when a common lesion is captured in the reference image and a second endoscopy image obtained by imaging in the second endoscopy. The information processing device according to claim 1 .

25. The lesion-related information includes size-related information regarding a size of the lesion. The information processing device according to claim 24.

26. The size-related information includes transition specifying information capable of specifying a transition of the size over time. The information processing device according to claim 25.

27. The transition identification information is derived based on a size of the lesion shown in the reference image and / or a size of the lesion shown in the second endoscopic image.

27. The information processing device according to claim 26.

28. The transition identification information is derived based on a size of the lesion shown in the reference image and / or a size and a type of the lesion shown in the second endoscopic image.

28. The information processing device according to claim 27.

29. The lesion-related information includes information capable of identifying a type of the lesion, information capable of identifying the number of the lesions, and / or information capable of identifying a state of the lesion. The information processing device according to claim 24.

30. The lesion-related information is associated with the reference image in which the lesion appears and / or the second endoscopic image in which the lesion appears. The information processing device according to claim 24.

31. The processor generates a report including the lesion-related information, the reference image including the lesion, and / or the second endoscopy image including the lesion. The information processing device according to claim 24.

32. The processor causes the display device to display the reference image and a second endoscopic examination image obtained by capturing the image in the second endoscopic examination so as to be contrasted with each other. The information processing device according to claim 1 .

33. An information processing device according to any one of claims 1 to 3; an endoscope for imaging the plurality of sites during the endoscopic examination; Endoscopic device.

34. Acquiring a plurality of first endoscopic examination images from a storage device in which a plurality of first endoscopic examination images obtained by imaging a plurality of sites in a first endoscopic examination that is a previous endoscopic examination have been accumulated; and and displaying, on a display device, at least one first endoscopic examination image that satisfies a specific condition among the plurality of first endoscopic examination images as a reference image at a timeout stage performed during the implementation period of the second endoscopic examination, which is the current endoscopic examination. Information processing methods.

35. On the computer, Acquiring a plurality of first endoscopic examination images from a storage device in which a plurality of first endoscopic examination images obtained by imaging a plurality of sites in a first endoscopic examination that is a previous endoscopic examination have been accumulated; and A program for executing a process that includes displaying at least one first endoscopic examination image among the multiple first endoscopic examination images that satisfies specific conditions as a reference image on a display device at a timeout stage that occurs during the implementation period of the second endoscopic examination, which is the current endoscopic examination.

Images