Medical support system and medical support method

Abstract

An image acquiring unit 22 acquires a simultaneously-captured image group including a plurality of endoscopic images in which the same subject has been captured in a simultaneous-capture mode and which are different from each other in terms of at least one among the image-capture method and the image processing method. A display image generating unit 26 generates a display image and displays the display image on a display device. The display image generating unit 26 can change the number of endoscopic images to be displayed from among the simultaneously-captured image group.

Description

Technical Field

[0001] The present disclosure relates to a medical support system and a medical support method.

Background Art

[0002] An endoscope observation apparatus is connected to an endoscope inserted into a patient's digestive tract, and displays, in real time, an image inside the digestive tract captured by the endoscope on a display device. When a doctor operates the release switch of the endoscope, the endoscope observation apparatus captures an endoscope image at the timing when the release switch is operated, and transmits the captured endoscope image to an image storage server. The endoscope images stored in the image storage server are used for creating an inspection report by a doctor after the inspection is completed.

[0003] Patent Document 1 discloses an endoscope apparatus that simultaneously displays, in real time, a normal light observation image and a narrow-band light observation image of the same biological tissue on a display device during an inspection.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] After the inspection is completed, in order to create an inspection report, a doctor operates an information processing device such as a personal computer to read out an endoscope image from an image storage server and display it on a display device. At this time, by displaying the endoscope image so that the doctor can easily check the state of the lesion, efficient diagnosis by the doctor becomes possible. The present disclosure has been made in view of such a situation, and an object thereof is to provide a display technique for endoscope images for realizing efficient diagnosis by a doctor.

Means for Solving the Problems

[0006] To solve the above problems, a medical support system according to an aspect of the present disclosure includes an image acquisition unit that acquires a simultaneous imaging image group including a plurality of endoscopic images obtained by imaging the same subject in a simultaneous imaging mode, where at least one of the imaging method or the image processing method is different for each of the plurality of endoscopic images, and a display image generation unit that generates a display image and causes the display image to be displayed on a display device. The display image generation unit can change the number of endoscopic images to be displayed from the simultaneous imaging image group.

[0007] A medical support method according to another aspect of the present disclosure includes acquiring a simultaneous imaging image group including a plurality of endoscopic images obtained by imaging the same subject in a simultaneous imaging mode, where at least one of the imaging method or the image processing method is different for each of the plurality of endoscopic images, setting the number of endoscopic images to be displayed from the simultaneous imaging image group, selecting the set number of endoscopic images from the simultaneous imaging image group, and causing the selected endoscopic images to be displayed on a display device.

[0008] Note that any combination of the above components, and those obtained by converting the expression of the present disclosure among a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as aspects of the present disclosure.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Mode for Carrying Out the Invention

[0010] FIG. 1 shows the configuration of a medical support system 1 according to an embodiment. The medical support system 1 is provided in a medical facility such as a hospital that performs endoscopic examinations. In the medical support system 1, an endoscopic observation device 4, an image storage server 8, and an information processing device 10 are communicably connected via a network 2 such as a LAN (Local Area Network).

[0011] The endoscopic system 3 is provided in an examination room and includes an endoscopic observation device 4, a display device 5, and an endoscope 6. The endoscope 6 has a light guide for transmitting illumination light supplied from the endoscopic observation device 4 to illuminate the inside of the subject. At the distal end of the endoscope, there are provided an illumination window for emitting the illumination light transmitted by the light guide to the subject, and an imaging unit for imaging the subject at a predetermined cycle and outputting an imaging signal to the endoscopic observation device 4. The endoscopic observation device 4 supplies illumination light corresponding to the observation mode to the endoscope 6. The imaging unit includes a solid-state imaging device (for example, a CCD image sensor or a CMOS image sensor) that converts incident light into an electrical signal.

[0012] The endoscope observation device 4 has an image processing unit that performs image processing on the imaging signal photoelectrically converted by the solid-state imaging device. The image processing unit generates an endoscope image and outputs it from the display device 5. In addition to normal image processing such as A / D conversion and noise removal, the image processing unit has a function of performing special image processing for the purpose of highlighting display. By equipping the image processing unit with a special image processing function, the endoscope observation device 4 can generate an endoscope image without special image processing and an endoscope image with special image processing from the imaging signals captured using the same illumination light. The endoscope observation device 4 is connected to an endoscope 6 inserted into the subject, and displays the image inside the digestive tract captured by the endoscope 6 on the display device 5 in real time.

[0013] The doctor observes the endoscope image displayed on the display device 5 according to the examination procedure. When the doctor operates the release switch of the endoscope 6, the endoscope observation device 4 captures (saves) the endoscope image at the timing when the release switch is operated, and transmits the captured endoscope image to the image storage server 8 for recording.

[0014] <Observation mode> The endoscope system 3 has a function of generating endoscope images in a plurality of observation modes, and the display device 5 displays an endoscope image generated in one observation mode. The "observation mode" is defined by the combination of the imaging method of the subject and the image processing method of the imaging signal. The endoscope observation device 4 generates an endoscope image in the set observation mode and displays it on the display device 5. The endoscope system 3 of the embodiment is equipped with the following observation modes.

[0015] (1) WLI (White Light Imaging) observation mode In the WLI observation mode, the endoscope 6 irradiates the subject with normal light (white light) to capture an image, and the endoscope observation device 4 performs normal image processing such as noise removal on the imaging signal to generate an endoscope image and displays it on the display device 5. In the WLI observation mode, an endoscope image is generated using the imaging method using normal light and the normal image processing method.

[0016] (2) TXI (Texture and Color Enhancement Imaging) Observation Mode In the TXI observation mode, the endoscope 6 irradiates the subject with normal light (white light) to capture an image, and after the endoscopic observation device 4 performs normal image processing such as noise removal on the captured image signal, special image processing is performed to optimize the three elements of "structure", "color tone", and "brightness" on the mucosal surface to generate an endoscopic image, which is then displayed on the display device 5. In the TXI observation mode, an endoscopic image is generated using an imaging method using normal light, a normal image processing method, and a special image processing method. The TXI observation mode provides an endoscopic image that emphasizes slight changes in color tone and structure.

[0017] (3) RDI (Red Dichromatic Imaging) Observation Mode In the RDI observation mode, the endoscope 6 irradiates the subject with narrow-band light of green, amber, and red to capture an image, and the endoscopic observation device 4 performs normal image processing such as noise removal on the captured image signal to generate an endoscopic image, which is then displayed on the display device 5. In the RDI observation mode, an endoscopic image is generated using an imaging method using narrow-band light of green, amber, and red and a normal image processing method. The RDI observation mode provides an endoscopic image that forms contrast in deep tissues.

[0018] (4) NBI (Narrow Band Imaging) Observation Mode In the NBI observation mode, the endoscope 6 irradiates the subject with narrow-band light of blue and green to capture an image, and the endoscopic observation device 4 performs normal image processing such as noise removal on the captured image signal to generate an endoscopic image, which is then displayed on the display device 5. In the NBI observation mode, an endoscopic image is generated using an imaging method using narrow-band light of blue and green and a normal image processing method. The NBI observation mode provides an endoscopic image that emphasizes capillaries and fine structures on the mucosal surface layer.

[0019] (5) AFI (Autofluorescence imaging) Observation Mode The AFI observation mode is an observation mode in which the endoscope 6 irradiates excitation light (390 - 470 nm) to image the subject, and after the endoscopic observation device 4 performs normal image processing such as noise removal on the imaging signal, an endoscopic image converted to green according to the signal intensity is generated and displayed on the display device 5. In the AFI observation mode, an endoscopic image is generated using an imaging method using blue excitation light, a normal image processing method, and a special image processing method. The AFI observation mode provides an endoscopic image that emphasizes tumorous lesions and normal mucosa in different color tones.

[0020] As described above, in the plurality of observation modes, at least one of the imaging method or the image processing method is different from each other. Specifically, when comparing the WLI observation mode and the TXI observation mode, both imaging methods use white light and are the same, but the image processing methods are different. Also, when comparing the WLI observation mode (TXI observation mode), the RDI observation mode, the NBI observation mode, and the AFI observation mode, different imaging methods (irradiation light) are used. The doctor selects an observation mode suitable for the observation situation and displays the endoscopic image on the display device 5.

[0021] <Imaging (Capture) Mode> The endoscope system 3 is equipped with two imaging (capture) modes. The imaging mode is a mode in which an endoscopic image is captured according to the operation of the release switch of the endoscope 6, and includes a "normal imaging mode" and a "simultaneous imaging mode". During the examination, the endoscope system 3 sets either the "normal imaging mode" or the "simultaneous imaging mode" as the imaging mode.

[0022] In the "normal imaging mode", when the release switch is operated, the endoscope system 3 captures the endoscopic image generated in the set observation mode, that is, the endoscopic image displayed on the display device 5, and transmits it to the image storage server 8. The capture function in the normal imaging mode is a standard function also installed in conventional endoscope systems.

[0023] In the "simultaneous imaging mode", when the release switch is operated, the endoscope system 3 captures the endoscope image generated in the set observation mode, immediately generates and captures the endoscope image by an imaging method and an image processing method of another observation mode, and transmits a plurality of types of endoscope images to the image storage server 8. Thus, the capture function in the simultaneous imaging mode is a function of simultaneously imaging (capturing) a plurality of endoscope images in which at least one of the imaging method or the image processing method is different from each other, triggered by an operation of the release switch by a doctor. The plurality of endoscope images captured simultaneously are images of substantially the same range of the same subject at the same angle of view. Here, "simultaneous" includes substantially simultaneous and is used as an expression including a time difference that can be regarded as substantially simultaneous. Also, the angle of view being "the same" includes that the angle of view is substantially the same and is used as an expression including an angular error that can be regarded as substantially the same.

[0024] In the simultaneous imaging mode, after capturing the endoscope image generated in the set observation mode, the endoscope system 3 changes at least one of the imaging method or the image processing method to generate an endoscope image in another observation mode. The time for generating the endoscope image in another observation mode is, for example, several milliseconds to several tens of milliseconds, and the endoscope observation device 4 may continue to display the still image immediately before switching the observation mode on the display device 5 while generating the endoscope image in the switched observation mode.

[0025] Hereinafter, in the simultaneous imaging mode, the endoscopic image generated in the set observation mode is referred to as the "reference image", and the endoscopic image generated in an observation mode other than the set observation mode is referred to as the "additional image". Which observation mode to generate the additional image in may be determined in advance in the endoscopic observation apparatus 4 in relation to the observation mode of the reference image. For example, when the reference image is generated in the WLI observation mode, it may be determined that the additional image is generated in the TXI observation mode. The observation mode of the additional image may be determined according to the site being imaged, or may be specified by the doctor at the timing of operating the release switch. Note that the number of additional images is not limited to one, and a plurality of additional images may be captured by one operation of the release switch.

[0026] The endoscopic observation apparatus 4 adds metadata to the captured endoscopic image. The metadata includes at least an image number indicating the order of imaging in the examination, irradiation light information indicating the type of imaging method, image processing information indicating whether or not special image processing has been performed, and simultaneous imaging information indicating whether or not it is an additional image captured in the simultaneous imaging mode. The simultaneous imaging information may include the image number of the reference image.

[0027] The image storage server 8 records the endoscopic images transmitted from the endoscopic observation apparatus 4 in association with an examination ID for identifying the endoscopic examination. When the image storage server 8 receives a request from the information processing apparatus 10 to read an endoscopic image specified by the examination ID, the image storage server 8 transmits a plurality of endoscopic images associated with the examination ID to the information processing apparatus 10.

[0028] The information processing apparatus 10 is provided in the doctor's office and is used when the doctor creates an examination report. The input unit 14 is a tool for the user to input operations, such as a mouse, a touch pen, and a keyboard. When creating an examination report, the doctor causes the display device 12 to display the endoscopic images captured in the examination, checks the state of the lesion, and inputs the examination results.

[0029] FIG. 2 shows the functional blocks of the information processing apparatus 10. The information processing apparatus 10 includes an operation reception unit 20, an image acquisition unit 22, a mode setting unit 24, a display image generation unit 26, a priority setting unit 28, and an image storage unit 30. The display image generation unit 26 generates a display image and causes it to be displayed on the display device 12.

[0030] The configuration of the information processing apparatus 10 shown in FIG. 2 can be realized by any processor, memory, auxiliary storage device, and other LSIs in terms of hardware, and can be realized by a program loaded in the memory and the like in terms of software. Here, however, the functional blocks realized by their cooperation are depicted. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

[0031] After the end of the endoscopic examination, the user, who is a doctor, inputs a user ID and a password to the information processing apparatus 10 and logs in. When the user logs in, an application for creating an examination report is launched, and a list of the completed examinations is displayed on the display device 12. Examination information such as the patient name, patient ID, examination date and time, and examination type is listed in this list of completed examinations, and the user operates the input unit 14 to select the examination to be the target of report creation. When the operation reception unit 20 receives the selection operation of the examination, the image acquisition unit 22 acquires a plurality of endoscopic images associated with the examination ID of the selected examination from the image storage server 8, stores them in the image storage unit 30, and the display image generation unit 26 generates a report input image and causes it to be displayed on the display device 12.

[0032] FIG. 3 shows an example of a report input screen. The display image generation unit 26 generates a report input image for the user to input inspection results and causes the display device 12 to display it. While the report input screen is being displayed, the report tab 100b is in a selected state. Information such as patient name, patient ID, date of birth, inspection type, inspection date, and performing doctor is displayed at the upper part of the report input screen. The report input screen is composed of two areas. On the left side, an image display area 102 for displaying thumbnails of endoscopic images attached to the report is provided, and on the right side, an inspection result input area 104 for the user to input inspection results such as diagnosis content is provided.

[0033] The inspection result input area 104 is an area for the user to input inspection results. In the illustrated example, areas are provided for inputting the diagnosis content of "esophagus", "stomach", and "duodenum", which are the observation ranges in upper gastrointestinal endoscopy. The inspection result input area 104 may have a format in which a plurality of selection options for inspection results are displayed and the user inputs the diagnosis content by selecting check boxes, or may have a free format in which text can be freely input.

[0034] The image display area 102 is an area for arranging and displaying endoscopic images attached to the report. The user selects the endoscopic images to be attached to the report from the list screen of endoscopic images. The user selects the recorded image tab 100a to cause the display device 12 to display a list of the endoscopic images captured in the inspection.

[0035] FIG. 4 shows an example of a list screen of endoscopic images. The list display icon 110 is an icon indicating a display format for displaying a list of endoscopic images. In this list screen, all the endoscopic images 130a to 130h captured in the inspection are arranged and displayed in the image display area 120 in the order of capture. Here, the case where eight endoscopic images are captured in the inspection is shown, but in an actual inspection, dozens of endoscopic images are captured.

[0036] In the endoscope system 3 of the embodiment, in the simultaneous imaging mode, by operating the release switch once, a plurality of endoscope images of the same subject are captured, and at least one of the imaging method or the image processing method is different for each of the plurality of endoscope images. When a plurality of endoscope images simultaneously captured in the simultaneous imaging mode are referred to as a "simultaneous imaging image group", when creating an inspection report, the image acquisition unit 22 acquires a plurality of simultaneous imaging image groups captured at different times from the image storage server 8. Therefore, when all the endoscope images 130a to 130h are arranged and displayed, the endoscope images included in the simultaneous imaging image group are arranged continuously, which may interfere with efficient diagnosis, especially when there are many simultaneous imaging image groups. In the example shown in FIG. 4, the endoscope images 130a to 130c are included in the first simultaneous imaging image group, the endoscope images 130d to 130e are included in the second simultaneous imaging image group, and the endoscope images 130g to 130h are included in the third simultaneous imaging image group. Note that the endoscope image 130f is an image captured in the normal imaging mode and does not belong to the simultaneous imaging image group.

[0037] Therefore, the information processing apparatus 10 of the embodiment has a function that allows the user to select the number of endoscope images to be displayed from the simultaneous imaging image group. The display image generation unit 26 can change the number of endoscope images to be displayed from the simultaneous imaging image group according to the selection by the user.

[0038] FIG. 5 shows an image management table held in the image storage unit 30. When storing an endoscope image in the image storage unit 30, the image acquisition unit 22 generates an image management table from the metadata of each endoscope image. As described above, the metadata of the endoscope image includes an image number indicating the order of imaging, irradiation light information indicating the type of imaging method, image processing information indicating whether or not special image processing has been performed, and simultaneous imaging information indicating whether or not it is an additional image captured in the simultaneous imaging mode. Here, the simultaneous imaging information is the image number of the reference image.

[0039] In the image management table, "irradiation light information" indicates the irradiation light used during imaging, and "image processing information" indicates whether special image processing has been performed. In the image processing information, the item value "OFF" indicates that no special image processing has been performed, the item value "ON" indicates that special image processing has been performed, and the item value "-" indicates that there is no option to perform special image processing on the image captured using that irradiation light. "Simultaneous imaging information" indicates whether the image is an additional image captured in the simultaneous imaging mode. The item value "-" indicates that it is not an additional image, and the item value "p" indicates that the image is an additional image captured simultaneously with the reference image of image number p. The endoscope images with image numbers 1 to 8 correspond to endoscope images 130a to 130h.

[0040] The endoscope image 130a with image number 1 was captured using normal light and no special image processing was performed, so it is an image captured in the WLI observation mode. The endoscope image 130b with image number 2 was captured using normal light and special image processing was performed, so it is an image captured in the TXI observation mode. The endoscope image 130c with image number 3 was captured using narrow-band light (green, amber, red), so it is an image captured in the RDI observation mode. The simultaneous imaging information "1" is added to the endoscope images 130b and 130c. Therefore, the endoscope images 130b and 130c are images captured simultaneously as additional images for the endoscope image 130a in the simultaneous imaging mode, and the endoscope images 130a to 130c constitute the first simultaneous imaging image group.

[0041] The endoscope image 130d with image number 4 was captured using normal light and no special image processing was performed, so it is an image captured in the WLI observation mode. The endoscope image 130e with image number 5 was captured using normal light and special image processing was performed, so it is an image captured in the TXI observation mode. The endoscopic image 130e has the simultaneous imaging information "4" added thereto. Therefore, the endoscopic images 130d to 130e constitute a second group of simultaneous imaging images.

[0042] Since the endoscopic image 130f with image number 6 is captured using narrow-band light (blue, green), it is an image captured in the NBI observation mode. In the image management table, since there is no endoscopic image with the simultaneous imaging information "6" added, the endoscopic image 130f is captured in the normal imaging mode.

[0043] Since the endoscopic image 130g with image number 7 is captured using narrow-band light (blue, green), it is an image captured in the NBI observation mode. Since the endoscopic image 130h with image number 8 is captured using narrow-band light (green, amber, red), it is an image captured in the RDI observation mode. The endoscopic image 130h has the simultaneous imaging information "7" added thereto. Therefore, the endoscopic images 130g to 130h constitute a third group of simultaneous imaging images.

[0044] The "display priority order" determines the order of additional images to be preferentially displayed in the display mode for comparison and display. For example, when two endoscopic images are arranged side by side in the comparison display mode, if the group of simultaneous imaging images contains three or more endoscopic images, the additional image with a higher display priority order is selected and displayed side by side with the reference image.

[0045] FIG. 6 shows an example of a display priority table held by the priority setting unit 28. The display priority is set for the imaging method and / or the image processing method. In this priority table, the order of priority display in the comparison display mode is set in the order of the observation modes of WLI, TXI, RDI, NBI, and AFI. The priority setting unit 28 sets priorities for the additional images in the simultaneous imaging image group according to this priority table. In the image management table shown in FIG. 5, the endoscope images with image numbers 1 to 3 constitute the simultaneous imaging image group, a priority of "1" is set for the endoscope image with image number 2, and a priority of "2" is set for the endoscope image with image number 3. Therefore, when displaying this simultaneous imaging image group in the comparison display mode, the endoscope image with image number 1, which is the reference image, and the endoscope image with image number 2, which is the additional image, will be displayed side by side.

[0046] FIG. 7 shows an example of a list screen of endoscope images in the embodiment. In the list display mode, the mode setting unit 24 sets a display mode that determines the number of endoscope images to be displayed from the simultaneous imaging image group. The display image generation unit 26 generates a display image including the set number of endoscope images according to the set display mode and causes the display device 12 to display it.

[0047] On the list screen of the embodiment, a list switching button 122 is provided for the user to select the number of endoscope images to be displayed from the simultaneous imaging image group. When the user operates the list switching button 122, the mode setting unit 24 switches the display mode on the list screen. In the example shown in FIG. 7, it is marked as "additional image non-display mode" beside the list switching button 122. The additional image non-display mode is a display mode in which the number of endoscope images to be displayed from the simultaneous imaging image group is set to 1. When the mode setting unit 24 sets the additional image non-display mode, the display image generation unit 26 generates a display image including one endoscope image selected from the simultaneous imaging image group and causes the display device 12 to display it.

[0048] In the additional image non-display mode, the display image generation unit 26 selects a reference image from the group of simultaneously captured images and includes it in the display image. In this example, the display image generation unit 26 has selected a reference image from the group of simultaneously captured images, and the endoscopic images 130a, 130d, and 130g are respectively the reference images in the group of simultaneously captured images. Note that the endoscopic image 130f is an image captured in the normal imaging mode and does not constitute the group of simultaneously captured images.

[0049] Thus, according to the additional image non-display mode, endoscopic images captured at different timings are arranged and displayed in a list, so that the visibility of each endoscopic image can be enhanced, and the diagnostic efficiency by the user can be improved.

[0050] In the additional image non-display mode, the display image generation unit 26 arranges the simultaneous imaging icon 132 indicating the existence of non-displayed images in association with the selected one endoscopic image. By displaying the simultaneous imaging icon 132, the user can recognize the existence of endoscopic images captured in another observation mode.

[0051] FIG. 8 shows another example of the list screen of the endoscopic images in the embodiment. When the user operates the list switching button 122, the mode setting unit 24 switches the display mode on the list screen. In the example shown in FIG. 8, it is indicated as "full image display mode" beside the list switching button 122. The full image display mode is a display mode for displaying all the endoscopic images in the group of simultaneously captured images. When the mode setting unit 24 sets the full image display mode, the display image generation unit 26 generates a display image including all the endoscopic images in the group of simultaneously captured images and causes the display device 12 to display it. In the full image display mode, similar to the list screen shown in FIG. 4, the endoscopic images included in the group of simultaneously captured images are arranged continuously. Therefore, for example, when the ratio of the endoscopic images included in the group of simultaneously captured images to the total number of endoscopic images is very small, the full image display mode may be set.

[0052] In the full-image display mode, the display image generation unit 26 arranges a reference image icon 134 indicating that it is a reference image in association with the reference image of the simultaneous imaging image group, and arranges an additional image icon 136 indicating that it is an additional image in association with the additional image of the simultaneous imaging image group. By displaying the reference image icon 134 and the additional image icon 136, the user can recognize the simultaneous imaging image group.

[0053] Note that the display priority order in the comparison display mode may be displayed on the additional image icon 136. As described above, the priority setting unit 28 sets the display priority order of the additional images based on the imaging method and / or the image processing method, and records it in the image management table (see FIG. 5). The display image generation unit 26 refers to the image management table and includes the display priority order "1" in the additional image icon 136 of the endoscope image 130b, and includes the display priority order "2" in the additional image icon 136 of the endoscope image 130c.

[0054] The user can change the display priority order using the input unit 14. For example, when the user double-clicks on the additional image icon 136 of the endoscope image 130c using a mouse, "1" may be displayed on the additional image icon 136 of the endoscope image 130c, and "2" may be displayed on the additional image icon 136 of the endoscope image 130b. When the operation reception unit 20 receives a double-click operation on the additional image icon 136 of the endoscope image 130c as a user operation to change the display priority order, the priority setting unit 28 may change the display priority order in the image management table. Specifically, the display priority order of image number 2 is changed to 2, and the display priority order of image number 3 is changed to 1.

[0055] FIG. 9 shows another example of the list screen of the endoscopic images in the embodiment. When the user operates the list switching button 122, the mode setting unit 24 switches the display mode on the list screen. In the example shown in FIG. 9, "comparative display mode" is indicated beside the list switching button 122. The comparative display mode is a display mode in which the number of endoscopic images to be displayed from the simultaneous imaging image group is set to a predetermined number of two or more. When the mode setting unit 24 sets the comparative display mode, the display image generation unit 26 generates a display image including a predetermined number of endoscopic images selected from the simultaneous imaging image group and causes the display device 12 to display it. The display image generation unit 26 selects a predetermined number of endoscopic images from the simultaneous imaging image group based on the display priority order set in the image management table. In the comparative display mode of the embodiment, two endoscopic images are selected from the simultaneous imaging image group and displayed.

[0056] In the comparative display mode, the display image generation unit 26 selects a reference image and an additional image with the display priority "1" from the simultaneous imaging image group and includes them in the display image. In this example, the display image generation unit 26 has selected endoscopic images 130a, 130d, 130g as the reference images and endoscopic images 130b, 130e, 130h with the display priority "1", and arranges the reference images and the additional images alternately. In the comparative display mode, the endoscopic image 130c with the display priority "2" is not displayed, nor is the endoscopic image 130f captured in the normal imaging mode.

[0057] In the comparative display mode, since the number of additional images to be compared with the reference image is limited for display, the user can efficiently compare the images with each other.

[0058] The display image generation unit 26 has a function of enlarging and displaying an endoscopic image 130 in an editable manner when a predetermined operation is performed on one endoscopic image 130 in the list display mode. When the operation reception unit 20 receives a user operation for instructing an enlarged display on one endoscopic image 130, the display image generation unit 26 generates a display image including the endoscopic image 130 enlarged in an editable manner and causes the display device 12 to display it. The user operation for instructing an enlarged display may be a double-click operation on the endoscopic image 130.

[0059] FIG. 10 shows an example of an editing screen of an endoscopic image in the embodiment. In the additional image non-display mode and the full image display mode, when the user double-clicks on the endoscopic image 130a, the display image generation unit 26 generates a display image including the endoscopic image 130a enlarged in an editable manner and causes the display device 12 to display it. The user can perform various drawings on the endoscopic image 130a on the editing screen. In this example, the user has placed drawing marks 140a and 140b on the lesion part. By attaching the edited endoscopic image to a report, the position of the lesion part can be easily specified from the drawing marks 140a and 140b added to the endoscopic image later.

[0060] FIG. 11 shows another example of an editing screen of an endoscopic image in the embodiment. In the comparison display mode, when the user double-clicks on the endoscopic image 130a, the display image generation unit 26 generates a display image including the endoscopic image 130a enlarged in an editable manner and the endoscopic image 130b and causes the display device 12 to display it.

[0061] In the comparison display mode, the reference image of the simultaneous imaging image group and one or more additional images (one additional image in the embodiment) are always displayed as a set. Since the comparison display mode is used for the purpose of the user comparing and contrasting a plurality of endoscopic images in the simultaneous imaging image group, it is preferable to continue the same display mode even during image editing. Therefore, in the comparison display mode, when the operation reception unit 20 receives a double-click operation on one of the two endoscopic images selected and displayed from the simultaneous imaging image group, the display image generation unit 26 generates a display image including the two endoscopic images and causes the display device 12 to display it.

[0062] On the editing screen, when the additional image to be displayed can be changed, the display image generation unit 26 arranges a change button 142 for changing the additional image on the editing screen. The case where the additional image can be changed is a case where there is an endoscopic image that is not being displayed in the simultaneous imaging image group. When the operation reception unit 20 receives a user operation instructing the change of the endoscopic image, the display image generation unit 26 changes the enlarged endoscopic image 130b to another endoscopic image 130c in the simultaneous imaging image group. In the embodiment, although the target of image change is the additional image, it is also possible to change the reference image to the additional image.

[0063] On the editing screen, the user can change the simultaneous imaging image group to be displayed. When the operation reception unit 20 receives a user operation instructing the change of the simultaneous imaging image group, the display image generation unit 26 generates a display image including a predetermined number of endoscopic images of a different simultaneous imaging image group and causes the display device 12 to display it. The user operation instructing the change of the simultaneous imaging image group may be, for example, a mouse wheel operation.

[0064] FIG. 12 shows another example of the editing screen of the endoscopic image in the embodiment. When transitioning from the comparison display mode to the editing mode, by continuing the display mode of the comparison display even when changing the simultaneous imaging image group to be displayed, efficient editing work of the user can be supported.

[0065] When the operation reception unit 20 receives a user operation to be drawn on the magnified endoscopic image, the display image generation unit 26 performs drawing on the endoscopic image according to the user operation and also performs drawing on other endoscopic images included in the simultaneous imaging image group according to the user operation. The display image generation unit 26 may make the display modes of the drawing different between the endoscopic image on which the user operation has been received and the endoscopic image on which the user operation has not been received.

[0066] FIG. 13 shows a state where drawing marks are arranged on an endoscopic image. When drawing is performed on the endoscopic image 130a, the display image generation unit 26 reflects the drawing content on the endoscopic image 130b. On the other hand, when drawing is performed on the endoscopic image 130b, the display image generation unit 26 reflects the drawing content on the endoscopic image 130a. Note that, for the non-displayed endoscopic image 130c in the same simultaneous imaging image group, processing for reflecting the drawing content is also performed. In FIG. 13, the solid-line drawing mark 140c indicates the drawing content added by the user to the endoscopic image 130a, and the dotted-line drawing mark 140e indicates the drawing content reflecting the drawing mark 140c. Similarly, the solid-line drawing mark 140f indicates the drawing content added by the user to the endoscopic image 130b, and the dotted-line drawing mark 140d indicates the drawing content reflecting the drawing mark 140f. By performing the reflection processing of the drawing content in this way, the labor of the user for drawing on another endoscopic image can be reduced. Also, by making the display modes of the drawing different between the endoscopic image on which the user's drawing operation has been received and the endoscopic image on which the user's drawing operation has not been received, the user can confirm on which endoscopic image the drawing has been made.

[0067] FIG. 14 shows an example of the comparison display area 124. When the comparison display icon 112 is selected, the display image generation unit 26 generates a comparison display area 124 for arranging and displaying a plurality of endoscopic images separately from the image display area 120 for displaying the endoscopic images in a list. The number of images to be displayed in the comparison display area 124 may be arbitrarily selected by the user, and in this example, two images are set.

[0068] The user arranges and compares any endoscopic image in the comparison display area 124. The user can arrange the endoscopic image in the comparison display area 124 by performing a drag-and-drop operation on the endoscopic image to the comparison display area 124. In the comparison display area 124, since the endoscopic image is enlarged compared to the display size in the image display area 120, the image display in the comparison display area 124 is suitable for the user to distinguish the differences between the arranged endoscopic images. The comparison display area 124 is often used, for example, when comparing an endoscopic image captured this time with an endoscopic image captured in the past.

[0069] The operation reception unit 20 receives a drag-and-drop operation of the user instructing the display in the comparison display area 124 for the endoscopic image displayed in the image display area 120. When the dragged-and-dropped endoscopic image is an image included in the simultaneous imaging image group, the display image generation unit 26 selects endoscopic images in the set number of displays from the simultaneous imaging image group based on the display priority order, generates a display image including the comparison display area 124 with the arranged endoscopic images, and causes the display device 12 to display it. FIG. 14 shows a state in which the endoscopic image 130a is dragged and dropped into the comparison display area 124, and the endoscopic image 130a and the endoscopic image 130b are arranged and displayed in the comparison display area 124. Among the endoscopic image 130b and the endoscopic image 130c included in the same simultaneous imaging image group, if the display priority order of the endoscopic image 130c is higher, the endoscopic image 130b is displayed next to the endoscopic image 130a.

[0070] As described above, the medical support system 1 of the embodiment can change the number of displayed endoscopic images captured in the simultaneous imaging mode according to the display mode. Therefore, the user can efficiently perform image diagnosis by setting an appropriate display mode.

[0071] The mode setting unit 24 may automatically determine the display mode to be set first by referring to various parameters. The mode setting unit 24 may set the display mode based on at least one of the inspection item, doctor information, and simultaneous imaging number ratio. For example, from the perspective of educational purposes, ranks are set for each of the inspection item, doctor information, and simultaneous imaging number ratio, and the mode setting unit 24 may automatically set the display mode to be set first from the sum of the rank values of each item.

[0072] FIG. 15(a) shows an example of a rank table for inspection items, FIG. 15(b) shows an example of a rank table for the number of years of doctor's service, and FIG. 15(c) shows an example of a rank table for the simultaneous imaging number ratio. These rank tables are stored in a storage device (not shown). The simultaneous imaging number ratio is the ratio that the number of endoscopic images captured in the simultaneous imaging mode occupies with respect to the total number of endoscopic images captured in the inspection. The mode setting unit 24 derives the rank values of the inspection item, doctor information, and simultaneous imaging number ratio, and calculates the total value.

[0073] FIG. 16 shows an example of a correspondence table associating the total rank value with the display mode. This correspondence table is stored in a storage device (not shown). When the mode setting unit 24 calculates the total rank value, it may set the list display mode and the enlarged display mode from the correspondence table. For example, for an inspection where the total rank value is calculated to be 8 - 9, it is assumed that the inspection is performed by a doctor with little experience such as a trainee doctor, and the mode setting unit 24 defaults the list display mode to the "additional image non - display mode". This enables doctors with little experience to accumulate training in diagnosing using only the reference images. When this doctor double - clicks on the endoscopic image 130 displayed in the image display area 120, the display image generation unit 26 enlarges and displays two endoscopic images included in the simultaneous imaging image group as shown in FIG. 11. In this way, for doctors with little experience, when transitioning from the additional image non - display mode to the editing mode, arranging and displaying the endoscopic images in the display mode of comparison display can contribute to the improvement of the doctor's diagnostic ability and efficient diagnosis.

[0074] As described above, the present disclosure has been described based on a plurality of embodiments. These embodiments are illustrative, and those skilled in the art will understand that various modifications are possible for each component and each combination of processing processes, and such modifications are also within the scope of the present disclosure. In the embodiment, the endoscope system 3 is equipped with a "normal imaging mode" and a "simultaneous imaging mode", but in a modification, the endoscope system 3 only needs to be equipped with at least the "simultaneous imaging mode" and may not be equipped with the "normal imaging mode".

Industrial Applicability

[0075] The present disclosure can be used in the technical field of displaying endoscope images.

Explanation of Signs

[0076] 1 ··· Medical support system, 2 ··· Network, 3 ··· Endoscope system, 4 ··· Endoscope observation device, 5 ··· Display device, 6 ··· Endoscope, 8 ··· Image storage server, 10 ··· Information processing device, 12 ··· Display device, 14 ··· Input unit, 20 ··· Operation reception unit, 22 ··· Image acquisition unit, 24 ··· Mode setting unit, 26 ··· Display image generation unit, 28 ··· Priority setting unit, 30 ··· Image storage unit.

Claims

1. A medical support system comprising a processor having hardware, The processor, A simultaneously captured image group is obtained, the simultaneously captured image group including a plurality of endoscopic images captured simultaneously in response to a single operation of a release switch, the plurality of endoscopic images being different from one another in at least one of an image capturing method and an image processing method; setting a first display mode in which the number of endoscopic images to be displayed from the simultaneously captured image group is set to a predetermined number of 2 or more; selecting a predetermined number of endoscopic images (two or more) from the simultaneously captured image group, and generating a display image for displaying the selected endoscopic images on a display device; receiving a user operation to instruct an enlarged display of one of the two or more predetermined number of endoscopic images selected and displayed from the simultaneously captured image group; generating a display image including a predetermined number of enlarged endoscopic images, which is equal to or greater than two; Medical support system.

2. The processor, Accepting a user operation to instruct a change in the endoscopic image; changing the enlarged and displayed endoscopic image to another endoscopic image in the simultaneously captured image group; The medical support system according to claim 1 .

3. The processor, acquiring a group of a plurality of simultaneously captured images captured at different times; accepting a user operation to instruct a change to the group of simultaneously captured images being enlarged and displayed; generating a display image including two or more predetermined number of endoscopic images enlarged from a group of simultaneously captured images different from the group of simultaneously captured images being enlarged and displayed; The medical support system according to claim 1 .

4. The processor, Accepting a user operation to draw on the enlarged endoscopic image; performing drawing on the endoscopic image in response to a user operation, and also performing drawing on other endoscopic images included in the simultaneously captured image group in response to a user operation; The medical support system according to claim 1 .

5. The processor, The display mode of the drawing is made different between an endoscopic image for which a user operation has been accepted and an endoscopic image for which a user operation has not been accepted. The medical support system according to claim 4.

6. The processor, setting a second display mode in which the number of endoscopic images to be displayed from the simultaneously captured image group is set to one; generating a display image including one endoscopic image selected from the group of simultaneously captured images; The medical support system according to claim 1 .

7. The processor, An icon indicating the presence of a hidden image is placed in association with one selected endoscopic image; The medical support system according to claim 6.

8. The processor, setting a third display mode in which all of the endoscopic images of the simultaneously captured image group are displayed; generating a display image including all of the endoscopic images of the simultaneously captured image group; The medical support system according to claim 1 .

9. The processor, accepting a user operation for changing the display priority order set based on the imaging method and / or the image processing method; The medical support system according to claim 8.

10. The processor, selecting a predetermined number of endoscopic images, two or more, from the simultaneously captured image group based on a display priority order set based on an imaging method and / or an image processing method; The medical support system according to claim 1 .

11. The processor, A user operation is received to instruct the display of the displayed endoscopic images in a comparison display area having a set number of images to be displayed; selecting the number of endoscopic images to be displayed from the group of simultaneously captured images based on a display priority order set for the imaging method and / or the image processing method, and generating a display image including a comparison display area in which the endoscopic images are arranged; The medical support system according to claim 6.

12. The processor, Set the display mode based on the examination item, doctor information, and the ratio of simultaneous imaging. The medical support system according to claim 1 .

13. The processor, setting a second display mode in which the number of endoscopic images to be displayed from the simultaneously captured image group is set to one; generating a display image including an endoscopic image acquired by a predetermined imaging method or image processing method from the simultaneously captured image group; The medical support system according to claim 12.

14. The plurality of endoscopic images included in the simultaneously captured image group are images of substantially the same range. The medical support system according to claim 1 .

15. The plurality of endoscopic images included in the simultaneously captured image group are acquired in response to a single release switch operation by a doctor in an endoscopic observation device. The medical support system according to claim 1 .

16. A medical support method executed by a medical support system having a processor, comprising: The processor, A simultaneously captured image group is obtained, the simultaneously captured image group including a plurality of endoscopic images captured simultaneously in response to a single operation of a release switch, the plurality of endoscopic images being different from one another in at least one of an image capturing method and an image processing method; setting a display mode in which the number of endoscopic images to be displayed from the simultaneously captured image group is set to a predetermined number of 2 or more; selecting a predetermined number of endoscopic images (two or more) from the simultaneously captured image group, and generating a display image for displaying the selected endoscopic images on a display device; receiving a user operation to instruct an enlarged display of one of the two or more predetermined number of endoscopic images selected and displayed from the simultaneously captured image group; generating a display image including a predetermined number of enlarged endoscopic images, which is equal to or greater than two; Medical assistance methods.

Images