Image capturing apparatus, control method, and storage medium

Abstract

To suppress the possibility of performing imaging while forgetting to set necessary medical information for information indicating an image analysis region SOLUTION: An information processing apparatus comprising: an imaging unit; a display unit; an acquisition unit configured to acquire one or more pieces of predetermined medical information; a setting unit configured to set the acquired medical information as medical information to be used for association; and a display control unit configured to display an imaging standby screen for a recording image on the display unit, wherein the imaging standby screen includes a display of a live view image captured by the imaging unit, an analysis unit that performs image analysis on the live view image to acquire an analysis result, and an association unit that associates the analysis result and the set medical information with the image for recording captured by the imaging unit, and the display control unit displays region information indicating the predetermined region on the imaging standby screen in a case where the medical information is set, and refrains from displaying the region information in a case where the medical information is not set, and SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a control method, and a program. [Background technology]

[0002] There is known a technique for recording medical information (e.g., photographer information, patient information, etc.) associated with an image captured in a medical setting. For example, Patent Document 1 discloses an imaging device that associates acquired patient information with a captured image. In order to prevent incorrect patient information from being associated with a captured image, the imaging device of Patent Document 1 displays the acquired patient information superimposed on a live view image, allowing the user to check the patient information in advance.

[0003] Furthermore, a technique is known in which information indicating a specific range (area) is superimposed on a live view image. For example, Patent Document 2 discloses an imaging device that switches a guide display indicating an image processing range according to specific conditions. The imaging device of Patent Document 2 switches the guide display depending on whether the user is adjusting the focus or not, thereby reducing the user's perception of the guide display as a nuisance. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-145984 [Patent Document 2] Patent No. 6882417 Summary of the Invention [Problem to be solved by the invention]

[0005] It is conceivable that image analysis technology utilizing deep learning could be used to estimate the disease in the affected area in a captured image. In this case, it is desirable to acquire an image of the area where you want to identify the disease, large enough to allow for analysis with sufficient accuracy. Therefore, it is conceivable to superimpose information (for example, a frame) indicating the area where image analysis will be performed on the live view image.

[0006] However, when a frame is superimposed on a live view image, the user's attention is drawn to the image analysis, but may be distracted from the medical information that needs to be set (e.g., photographer information, patient information, etc.). As a result, the user may take a photograph without setting the patient information, etc., and may not be able to associate the patient information, etc. with the captured image. Conventional technologies cannot address this issue.

[0007] The present invention has been made in consideration of this situation, and aims to provide a display control technology that reduces the possibility that a user will take an image while forgetting to set the necessary medical information for information (e.g., a frame) that indicates the area of ​​image analysis. [Means for solving the problem]

[0008] In order to solve the above problem, the present invention provides an imaging device comprising an imaging unit, a display unit, an acquisition means for acquiring one or more predetermined medical information, a setting means for setting the acquired one or more medical information as medical information to be used for association, a display control means for controlling a shooting standby screen for an image to be recorded to be displayed on the display unit, wherein the shooting standby screen includes a display of a live view image captured by the imaging unit, an analysis means for acquiring an analysis result by performing image analysis on a portion of the live view image displayed in a predetermined area on the shooting standby screen, and an association means for associating the analysis result and the one or more set medical information with the image for recording captured by the imaging unit, wherein the display control means controls to display area information indicating the predetermined area on the shooting standby screen if the one or more medical information have been set, and to refrain from displaying the area information if the one or more medical information have not been set. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a display control technology that reduces the possibility that a user will perform imaging without setting the necessary medical information for information (e.g., a frame) that indicates the area of ​​image analysis.

[0010] Other features and advantages of the present invention will become more apparent from the accompanying drawings and the following detailed description of the preferred embodiment of the present invention. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing the configuration of an imaging device 100. [Figure 2] 1 is a diagram showing the configuration of a network system within a hospital in which an imaging device 100 participates. [Figure 3] 4 is a flowchart of processing executed by the imaging device 100. [Figure 4] 4 is a flowchart showing details of the medical information setting process in S306 of FIG. 3. [Figure 5]FIG. 10 is a diagram showing a display example of a shooting standby screen. [Figure 6] 4 is a flowchart showing details of the photographing process in S312 of FIG. 3. [Figure 7] 4 is a flowchart showing details of the captured image transmission process in S310 of FIG. 3. [Figure 8] 3A is a flowchart showing details of the image analysis process in S304 of Fig. 3. FIG. 3B is a flowchart showing a process for waiting for the completion of the analysis process started in S804. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] [First embodiment] FIG. 1 is a block diagram showing the configuration of an imaging device 100. FIG. 2 is a diagram showing the configuration of a network system within a hospital in which the imaging device 100 participates. As shown in FIG. 2, the network system within the hospital has an in-hospital network 201 and an in-hospital system 202. The in-hospital system 202 includes one or more department systems. In the example of FIG. 2, five department systems are included in the in-hospital system 202. The imaging device 100 connects to the in-hospital network 201 via a communication unit 109, which will be described later, and communicates with the in-hospital system 202. A plurality of imaging devices 100 (for example, imaging devices 100a and 100b shown in FIG. 2) may be connected to the in-hospital network 201.

[0014] 1, the imaging device 100 includes an internal bus 150. A CPU 101, a memory 102, a nonvolatile memory 103, an image processing unit 104, a display 105 (display unit), an operation unit 106, a recording medium I / F 107, a communication unit 109, and a camera unit 110 (imaging unit) are connected to the internal bus 150. The components connected to the internal bus 150 are configured to be able to exchange data with each other via the internal bus 150.

[0015] The memory 102 is formed of, for example, RAM (a volatile memory using a semiconductor element, etc.). The CPU 101 controls the entire imaging device 100 using the memory 102 as a work memory in accordance with a program stored in, for example, nonvolatile memory 103. The nonvolatile memory 103 stores image data, audio data, other data, and various programs for operating the CPU 101. The nonvolatile memory 103 is formed of, for example, a hard disk (HD) or a ROM.

[0016] The image processing unit 104 operates under the control of the CPU 101. The image processing unit 104 performs various image processing on image data stored in the nonvolatile memory 103 or the recording medium 108, image data acquired via the communication unit 109, and images captured by the camera unit 110. The image processing performed by the image processing unit 104 includes A / D conversion processing, D / A conversion processing, encoding processing, compression processing, decoding processing, enlargement / reduction processing (resizing processing), noise reduction processing, color conversion processing, and the like. The image processing unit 104 may be configured with a circuit block dedicated to performing specific image processing. Depending on the type of image processing, the CPU 101 may also perform image processing according to a program without using the image processing unit 104.

[0017] The display 105 displays images under the control of the CPU 101. The display 105 can also display GUI screens constituting a Graphical User Interface (GUI) under the control of the CPU 101. The CPU 101 generates a display control signal according to a program, and controls each component of the imaging device 100 to generate a video signal to be displayed on the display 105 and output it to the display 105. The display 105 displays an image based on the output video signal. Note that the imaging device 100 itself is only provided with an interface for outputting a video signal to be displayed on the display 105, and the display 105 may be configured as an external monitor.

[0018] The operation unit 106 is a device that inputs instructions from the photographer to the imaging device 100. The operation unit 106 includes, for example, a power button, a shutter button, a mode dial, and the like. The mode dial is a dial that switches between operation modes such as a shooting mode. The operation unit 106 may include a touch panel. The touch panel is an input device that is configured to be planar and overlaid on the display 105, and that is configured to output coordinate information according to the position that is touched.

[0019] The recording medium I / F 107 is configured to allow attachment of a recording medium 108 such as a hard disk, a memory card, a CF card, an SD card, or a USB memory. The recording medium I / F 107 reads data from the attached recording medium 108 and writes data to the recording medium 108 under the control of the CPU 101.

[0020] The communication unit 109 is connected to the hospital network 201 via a wired cable or wirelessly, and is configured to input and output video signals, audio signals, various data, etc. The communication unit 109 transmits and receives data to and from the hospital system 202 via the hospital network 201. The data that the communication unit 109 receives from the hospital system 202 includes medical information (e.g., information about the medical department, information about the photographer, information about the patient, and information about the affected area) and information about image data linked to the medical information. The data that the communication unit 109 transmits to the hospital system 202 includes image data such as a barcode or QR code required to refer to the medical information in the hospital system 202, and a photograph of the affected area taken with the medical information attached thereto.

[0021] The camera unit 110 is composed of an imaging element (imaging sensor) that is made up of a CCD or CMOS element that converts an optical image into an electrical signal, etc. The camera unit 110 includes a group of lenses (taking lenses) including a zoom lens and a focus lens, a shutter with an aperture function, the imaging element, an A / D converter that converts an analog signal output from the imaging element into a digital signal, and a barrier that covers the imaging system to prevent it from getting dirty or damaged.

[0022] The image processing unit 104 performs predetermined pixel interpolation, resizing, color conversion, and other processes on the data captured by the camera unit 110. The image processing unit 104 also has the function of detecting faces in images and estimating the name of a disease corresponding to an affected area in an image through image analysis processing utilizing deep learning. Based on the calculation results obtained by the image processing unit 104, the CPU 101 performs exposure control, distance measurement control, and AWB (auto white balance) processing. The image data captured by the camera unit 110 and processed by the image processing unit 104 is displayed on the display 105. The digital signal captured by the camera unit 110, A / D converted by an A / D converter, and stored in the memory 102 is converted to analog by a D / A converter and sequentially transferred to and displayed on the display 105, thereby enabling the display of a live view image (LV image). The live view display can be performed during standby for still image capture, standby for video capture, video recording, and other times, and displays the captured subject image in near real time.

[0023] The CPU 101 controls the camera unit 110 and the image processing unit 104 to start operations such as AF (autofocus) processing, AE (auto exposure) processing, and AWB processing in response to a shooting preparation instruction based on a user operation on the operation unit 106. The CPU 101 controls the camera unit 110 and the image processing unit 104 to start a series of shooting processing (main shooting) operations in response to a shooting instruction. The series of shooting processing (main shooting) operations include performing a main exposure, reading out a signal from the image sensor, and performing image processing on the captured image in the image processing unit 104 to generate an image file and record it on the recording medium 108. The shooting instruction can be issued by a user operation on the operation unit 106. The camera unit 110 is capable of shooting still images and videos.

[0024] Next, processing executed by the image capture device 100 will be described with reference to the flowchart of Fig. 3. Unless otherwise specified, the processing of each step in this flowchart is realized by the CPU 101 controlling each component of the image capture device 100 in accordance with a control program stored in the non-volatile memory 103.

[0025] In S301, the CPU 101 determines whether the operation mode of the imaging device 100 is a medical mode. The imaging device 100 has a medical mode (first mode) and a normal mode (second mode) as its operation modes. A user (e.g., a photographer) can select the operation mode in advance on a setting screen displayed on the display 105. If the operation mode is the medical mode, the process proceeds to S302, and if the operation mode is the normal mode, the process proceeds to S303.

[0026] In S302, the CPU 101 controls the communication unit 109 to perform processing to connect the imaging device 100 to the in-hospital system 202 via the in-hospital network 201. As described above, in the example of FIG. 2, five medical department systems are included in the in-hospital system 202. The imaging device 100 connects to one of the five medical department systems included in the in-hospital system 202. There are no particular limitations on the method for selecting the medical department system to connect to, but for example, a configuration may be adopted in which the user selects the medical department system by operating the operation unit 106. Furthermore, a configuration may be adopted in which the medical department system to which the previous connection was made is automatically selected when connecting for the second time or later.

[0027] In S303, the CPU 101 displays a shooting standby screen for an image to be recorded on the display 105. The shooting standby screen (LV screen) includes a display of an LV image.

[0028] 5A is a diagram showing an example of an image capture standby screen displayed in S303. When the operation mode of the imaging device 100 is the medical mode, the image capture standby screen includes a photographer area 501, a patient area 502, and an affected area 503 as areas for displaying medical information (medical information areas) set in S305 (described later). When the operation mode of the imaging device 100 is the normal mode, the image capture standby screen does not include a medical information area. Therefore, medical information is not acquired or set, and the set medical information is not added (associated) to the captured image.

[0029] The photographer area 501 is an area for displaying photographer information (e.g., photographer name, photographer ID, and the photographer's department, etc.) from the medical information. The patient area 502 is an area for displaying patient information (e.g., patient name, patient ID, date of birth, and gender, etc.) from the medical information. The affected area area 503 is an area for displaying affected area information (e.g., affected area name) from the medical information. Each medical information area may display at least one item included in the corresponding medical information. In the following description, the photographer area 501 displays the photographer name, the patient area 502 displays the patient name, and the affected area area 503 displays the affected area name. Therefore, on the shooting standby screen, the photographer name, patient name, and affected area name are displayed superimposed on the LV image. However, since medical information has not yet been set at the time of S303, no medical information is displayed in each medical information area.

[0030] The imaging standby screen also includes a send button 504 and an end button 505. When the end button 505 is operated, the CPU 101 ends the processing of this flowchart for the current medical department. The send button 504 will be described later. The send button 504 may not be displayed if an image of the affected area has not yet been captured.

[0031] In S304, the CPU 101 controls the image processing unit 104 to perform image analysis processing. Details of the image analysis processing will be described later with reference to FIG. 8(a). In the image analysis processing, an analysis result (for example, the name of a disease corresponding to an affected area included in a captured image) is estimated by using any known technology such as deep learning. Note that, as will be described later with reference to FIG. 8(a), the image analysis processing takes some time to complete. Therefore, the CPU 101 starts a process that waits for the completion of the analysis processing, and proceeds to S305 without waiting for the completion of the analysis processing.

[0032] In S305, the CPU 101 determines whether or not a medical information setting instruction has been issued by the user. If a medical information setting instruction has been issued, the process proceeds to S306, and if a medical information setting instruction has not been issued, the process proceeds to S307. A medical information setting instruction is issued by the user by selecting the photographer area 501, the patient area 502, or the affected area 503 using the operation unit 106 (for example, by touching the touch panel).

[0033] In S306, the CPU 101 performs medical information setting processing for the medical information area selected in S305 (the photographer area 501, the patient area 502, or the affected area area 503). Details of the medical information setting processing will be described later with reference to FIG.

[0034] FIG. 5(b) is a diagram showing an example of an imaging standby screen after medical information setting processing has been performed for each of the photographer area 501, the patient area 502, and the affected area 503. As will be understood from S311 described below, the determination in S305 is repeated during imaging standby. Therefore, the user can perform medical information setting processing corresponding to each medical information area by selecting each of the photographer area 501, the patient area 502, and the affected area 503. After medical information setting is complete, the imaging standby screen also displays a frame (image analysis frame 506) indicating the range (predetermined area) where image analysis by the image processing unit 104 will be performed. Details of the processing for displaying the image analysis frame 506 will be described later with reference to FIG. 4.

[0035] It should be noted that the image analysis frame 506 does not need to completely match the analysis range (predetermined region) of the image. Even if the image analysis frame 506 does not completely match the analysis range, as long as the image analysis frame 506 serves as an indicator of the analysis range (as long as the image analysis frame 506 helps the user recognize the analysis range), it can be said that the image analysis frame 506 indicates the analysis range. Furthermore, the information (region information) indicating the analysis range (predetermined region) is not limited to the image analysis frame 506. Any information can be used as information indicating the analysis range (predetermined region) as long as it serves as an indicator of the analysis range.

[0036] In S307, the CPU 101 determines whether or not the following two conditions are met: "the image analysis frame 506 is displayed on the display 105" and "the image analysis result is set as medical information." If these two conditions are met, it means that one or more pieces of predetermined medical information (e.g., photographer information, patient information, and affected area information) have been set (details will be described later with reference to S405), and the analysis process that starts in S804, which will be described later, has been completed and the image analysis result (e.g., disease name) has been acquired. If the two conditions are met, the process proceeds to S308; if not, the process proceeds to S309.

[0037] In S308, CPU 101 displays information indicating the image analysis result (analysis result information) superimposed on the LV image on display 105. CPU 101 also displays a button (reanalysis button) for the user to instruct reanalysis of the image superimposed on the LV image. Furthermore, CPU 101 changes the display mode of image analysis frame 506.

[0038] FIG. 5(c) is a diagram showing an example of a standby screen displayed as a result of the processing of S308. Compared to FIG. 5(b) described above, FIG. 5(c) additionally displays an image analysis result 507 and a reanalysis button 508. Therefore, the user can know the image analysis result (in the example of FIG. 5(c), the disease name "urticaria"). Furthermore, the display mode (second display mode) of the image analysis frame 506 in FIG. 5(c) differs from the display mode (first display mode) of the image analysis frame 506 in FIG. 5(b). This makes it easier for the user to recognize that the image analysis has been completed. The image analysis result 507 may be displayed near the image analysis frame 506 or at a position away from the image analysis frame 506.

[0039] Note that the specific change in the display mode of the image analysis frame 506 is not particularly limited. For example, the CPU 101 may change the appearance of the image analysis frame 506 by changing the color, transparency, or thickness of the image analysis frame 506. That is, the CPU 101 may change the display mode of the image analysis frame 506 from a display mode (first display mode) in which the image analysis frame 506 is displayed with a first appearance to a display mode (second display mode) in which the image analysis frame 506 is displayed with a second appearance different from the first appearance. Alternatively, the CPU 101 may hide the image analysis frame 506. In this way, hiding the image analysis frame 506 is also included in changing the display mode of the image analysis frame 506. In other words, the display mode after the change (second display mode) may be a mode in which the image analysis frame 506 is not displayed.

[0040] In S309, CPU 101 determines whether a transmission instruction has been issued by the user. If a transmission instruction has been issued, the process proceeds to S310. If a transmission instruction has not been issued, the process proceeds to S311. The operation for issuing a transmission instruction by the user is, for example, a touch operation on send button 504 in FIG. 5(b).

[0041] In S310, the CPU 101 controls the communication unit 109 to perform processing (captured image transmission processing) of transmitting the captured image obtained by the imaging processing in S312, which will be described later, to the in-hospital system 202. Details of the captured image transmission processing will be described later with reference to FIG.

[0042] In S311, the CPU 101 determines whether or not a shooting instruction has been issued by the user. If a shooting instruction has been issued, the process proceeds to S312, and if a shooting instruction has not been issued, the process returns to S304. The shooting instruction issued by the user is assumed to be an instruction to shoot by operating the operation unit 106 by the user, such as pressing the shutter button or operating the touch panel.

[0043] In S312, CPU 101 controls camera unit 110 to perform shooting processing and acquire a captured image for recording (image for recording). Thereafter, the processing returns to S304. Details of the shooting processing will be described later with reference to FIG. 6.

[0044] Fig. 8(a) is a flowchart showing details of the image analysis processing in S304 of Fig. 3. In S801, the CPU 101 determines whether or not the image analysis result has been set as the medical information. If the image analysis result has been set as the medical information, the processing proceeds to S802. In this case, the image analysis result set as the medical information is the analysis result previously obtained by the analysis processing in S804 (described below). If the image analysis result has not been set as the medical information, the processing proceeds to S804.

[0045] In S802, the CPU 101 determines whether a reanalysis instruction (a predetermined user instruction) has been issued by the user. If a reanalysis instruction has been issued (a predetermined trigger has occurred), the process proceeds to S803. If a reanalysis instruction has not been issued, the process of this flowchart ends. The user's operation for issuing a reanalysis instruction is, for example, a touch operation on the reanalysis button 508 in FIG. 5(c).

[0046] In S803, the CPU 101 stops displaying the image analysis result 507 and the reanalysis button 508 and changes the display mode of the image analysis frame 506. The specific change in the display mode of the image analysis frame 506 in S803 is not particularly limited. For example, the display mode after the change in S803 (third display mode) may be the same as the display mode before the change in S308 (first display mode). In this case, the shooting standby screen on the display 105 returns to the state of FIG. 5(b) from the state of FIG. 5(c), for example. As another example, the display mode after the change in S803 (third display mode) may be different from both the display mode before the change in S308 (first display mode) and the display mode after the change in S308 (second display mode).

[0047] In S803, CPU 101 may erase the set image analysis result, or may retain it without erasing it. If the set image analysis result is retained, CPU 101 may perform control to continue displaying image analysis result 507. If the image analysis result is retained and the next shooting process (S312) is performed before the reanalysis is completed, CPU 101 can associate the most recent image analysis result with the captured image (details of the association will be described later with reference to FIG. 6). Furthermore, even if the set image analysis result is retained, the image analysis result is treated as not being set in the determination of S307 in FIG. 3 while the reanalysis is being performed.

[0048] In S804, the CPU 101 starts analysis processing for a preset analysis range of the LV image (image analysis for a portion of the LV image displayed in a predetermined area on the shooting standby screen) by controlling the image processing unit 104. Here, the image processing unit 104 uses any known technology such as deep learning to analyze an image of the skin to determine a skin disease, or analyze an image of the oral cavity to determine the progress of tooth decay, thereby obtaining an analysis result.

[0049] The analysis process in S804 takes some time to complete, so the CPU 101 starts a process to wait for the analysis process to be completed, and proceeds to S305 in FIG.

[0050] 8B is a flowchart of a process for waiting for the completion of the analysis process started in S804. The process of this flowchart is executed in parallel with the process of the flowchart of FIG.

[0051] In S851, the CPU 101 determines whether the analysis process is complete. The CPU 101 repeats the determination in S851 until the analysis process is complete. When the analysis process is complete, the process proceeds to S852.

[0052] In S852, the CPU 101 acquires the image analysis result from the image processing unit 104 and sets the image analysis result as medical information to be added to the captured image (in the case of the second or subsequent analysis process, the image analysis result is updated from the previous one to the current one). The medical information including the image analysis result is held in the memory 102.

[0053] As can be seen from S307 and S308 in FIG. 3 , if one or more pieces of predetermined medical information have not been set, the reanalysis button 508 is not displayed. In this case, the image analysis frame 506 is also not displayed. Therefore, there is a possibility that an image of an appropriate part of the patient has not been acquired, and therefore an appropriate image analysis result has not been obtained. Therefore, in S802 in FIG. 8( a), the CPU 101 may determine whether a predetermined time has elapsed since the previous analysis process. If the predetermined time has elapsed, the process may proceed to S804 (S803 is skipped). In other words, if one or more pieces of predetermined medical information have not been set, the CPU 101 may periodically update the image analysis result by periodically performing a new image analysis (analysis process) on a new portion of the live view image displayed in the analysis range. Conversely, if the image analysis frame 506 is displayed, it is considered likely that an appropriate image analysis result has been acquired. Therefore, the CPU 101 does not update the image analysis result unless there is a user instruction, such as a touch operation on the reanalysis button 508. This makes it possible to associate more accurate image analysis results with captured images.

[0054] Fig. 4 is a flowchart showing details of the medical information setting process in S306 in Fig. 3. In S401, the CPU 101 controls the communication unit 109 to perform a process of acquiring medical information corresponding to the medical information area selected in S305 from the in-hospital system 202. For example, when the photographer area 501 is selected, the CPU 101 performs a process of acquiring photographer information as medical information.

[0055] The specific method for acquiring medical information is not particularly limited. For example, the CPU 101 transmits an acquisition request to the in-hospital system 202. In response to the acquisition request, the in-hospital system 202 transmits a list (medical information list) including one or more candidates of medical information to the imaging device 100. In this case, the in-hospital system 202 stores in advance medical information lists (e.g., a photographer information list, a patient information list, and an affected area information list) for each medical department. For example, when the photographer area 501 is selected in S305, the CPU 101 transmits a photographer information acquisition request to the medical department system connected in S302, thereby acquiring the photographer information list corresponding to the medical department from the medical department system.

[0056] Another example of a method for acquiring medical information is a method using a barcode or a QR code (hereinafter referred to as "barcode, etc."). It is assumed that photographers and patients are individually assigned barcodes, etc., for information management purposes. The assigned barcodes, etc., are printed, for example, on the photographer's ID card or the patient's medical card. The CPU 101 controls the camera unit 110 to capture an image of the photographer's or patient's barcode, etc., and transmits the obtained image data to the in-hospital system 202. The in-hospital system 202 transmits medical information identified by the barcode, etc., in the image data to the imaging device 100. For example, when the photographer area 501 is selected in S305, the user places the photographer's barcode, etc., within the imaging range of the camera unit 110 and issues an instruction to capture an image by operating the operation unit 106. In response to the instruction, the CPU 101 captures an image and transmits the obtained image data to the in-hospital system 202. This allows the CPU 101 to acquire photographer information identified by the photographer's barcode, etc., from the in-hospital system 202.

[0057] Another example of a method for acquiring medical information is a method using a photographer ID or patient ID input by a user. The user inputs the photographer ID or patient ID by operating the operation unit 106. The CPU 101 transmits the input photographer ID or patient ID to the in-hospital system 202. The in-hospital system 202 transmits photographer information or patient information identified by the received photographer ID or patient ID to the imaging device 100. If the user inputs only a portion of the photographer ID or patient ID, the in-hospital system 202 may transmit a medical information list (photographer information list or patient list) corresponding to multiple photographers or multiple patients that match the input photographer ID or patient ID. For example, if the photographer area 501 is selected in S305, the user inputs the photographer ID (or a portion thereof) by operating the operation unit 106. The CPU 101 transmits the input photographer ID (or a portion thereof) to the in-hospital system 202. This allows the CPU 101 to acquire, from the in-hospital system 202, photographer information (or a photographer information list) identified by the input photographer ID (or a part thereof).

[0058] The source of the medical information in S401 is not limited to an external system such as the in-hospital system 202. For example, a configuration may be adopted in which the medical information list is stored in advance in the recording medium 108, and the CPU 101 acquires the medical information list from the recording medium 108 in S401.

[0059] In S402, the CPU 101 determines whether or not the medical information has been successfully acquired. If the medical information has been successfully acquired, the process proceeds to S403, and if the medical information has not been successfully acquired, the process proceeds to S404. Acquisition of the medical information may fail, for example, if a connection to the in-hospital system 202 has failed due to a communication error in S302, or if a barcode or the like is not included in the image captured in S401.

[0060] In S403, the CPU 101 sets the medical information acquired in S401 as medical information to be added to the captured image (medical information to be used for association). The set medical information is held in the memory 102. The CPU 101 also updates the medical information display area on the shooting standby screen so as to display the set medical information. For example, if the photographer area 501 is selected in S305, the acquired photographer information is set as photographer information to be added to the captured image. Furthermore, the photographer's name included in the set photographer information is displayed in the photographer area 501.

[0061] Prior to setting the medical information, CPU 101 may control display 105 to display a screen for the user to confirm whether the acquired medical information is correct. If a user operation indicating that the medical information is correct is performed, CPU 101 sets the medical information. If a user operation indicating that the medical information is incorrect is performed, CPU 101 discards the acquired medical information without setting the medical information.

[0062] In S404, CPU 101 determines whether or not a command to reacquire medical information has been issued by the user. For example, CPU 101 displays on display 105 a screen that prompts the user to select whether or not to reacquire medical information. The user can issue a command to reacquire medical information by operating operation unit 106. If a command to reacquire medical information has been issued, the process returns to S401; if a command to reacquire medical information has not been issued, the process proceeds to S405.

[0063] In S405, CPU 101 determines whether one or more predetermined pieces of medical information have been set. If one or more predetermined pieces of medical information have been set, processing proceeds to S406; if not, processing of this flowchart ends. The one or more predetermined pieces of medical information are, for example, all types of medical information that can be set by the user through the medical information setting processing of S306 (in the example of FIG. 5(a) , photographer information, patient information, and affected area information), but are not limited to this. For example, the one or more predetermined pieces of medical information may be patient information and affected area information. In this case, even if photographer information has not been set, if patient information and affected area information have been set, processing proceeds to S406.

[0064] In S406, the CPU 101 performs control to display the image analysis frame 506 on the shooting standby screen. After this, as described above with reference to Fig. 5(b), the image analysis frame 506 is displayed on the shooting standby screen superimposed on the LV image. This makes it possible to notify the user of the image analysis range.

[0065] Note that when the CPU 101 acquires medical information such as patient information by communicating with the in-hospital system 202 (external device) via the communication unit 109, if the communication fails, the CPU 101 cannot set the medical information (see the explanation of S402). Therefore, if communication with the in-hospital system 202 fails, the CPU 101 may perform control to display the image analysis frame 506 on the shooting standby screen even if one or more predetermined pieces of medical information have not been set. In this case, even if medical information such as patient information cannot be acquired, the image analysis frame 506 is displayed, making it possible to associate highly accurate image analysis results with the captured image.

[0066] Fig. 6 is a flowchart showing details of the photographing process in S311 in Fig. 3. In S601, CPU 101 controls camera unit 110 to photograph the affected area of ​​the patient and obtain an image of the affected area (captured image).

[0067] In S602, the CPU 101 determines whether or not set medical information exists. If at least one of the photographer information, patient information, and affected area information has been set in S403 of Fig. 4 and stored in the memory 102, or if the image analysis results have been set in S852 of Fig. 8(b) and stored in the memory 102, it is determined that medical information exists, and the process proceeds to S603. If set medical information does not exist, the process of this flowchart ends.

[0068] In S603, the CPU 101 adds (associates) the medical information set in S403 (e.g., photographer information, patient information, and affected area information) and the image analysis results, which are medical information set in S852, to the captured image. The method of addition (association) is not particularly limited, but an example is a method of storing the medical information in the header area of ​​an image file containing the captured image. The CPU 101 temporarily stores the captured image with the medical information added in the memory 102, or records it on the recording medium 108 via the recording medium I / F 107.

[0069] Fig. 7 is a flowchart showing details of the captured image transmission process in S310 in Fig. 3. In S701, the CPU 101 determines whether or not the imaging device 100 is connected to the in-hospital system 202 via the communication unit 109. If the imaging device 100 is connected to the in-hospital system 202, the process proceeds to S702, and if the imaging device 100 is not connected to the in-hospital system 202, the process proceeds to S705.

[0070] In S702, the CPU 101 selects captured images to be transmitted to the in-hospital system 202. The method for selecting captured images is not particularly limited, and for example, the CPU 101 may automatically select all captured images stored in the memory 102 or the recording medium 108. As another example, a configuration may be adopted in which the user is allowed to select captured images. In this case, for example, the CPU 101 displays thumbnails of the captured images stored in the memory 102 or the recording medium 108 on the display 105. The CPU 101 selects one or more captured images selected by the user in the thumbnail display as captured images to be transmitted to the in-hospital system 202.

[0071] In S703, the CPU 101 transmits the captured image selected in S702 to the in-hospital system 202. At this time, the CPU 101 may perform control so that the transmitted captured image is deleted from the memory 102 or the recording medium 108, or may keep it stored.

[0072] In S704, the CPU 101 displays on the display 105 a message indicating that the transmission of the captured image has been completed.

[0073] In S705, the CPU 101 displays an error message on the display 105 indicating that the transmission of the captured image has failed.

[0074] As described above, according to the first embodiment, the imaging device 100 acquires one or more pieces of predetermined medical information (for example, photographer information, patient information, and affected area information) and sets the acquired one or more pieces of medical information as medical information to be used for association (S401 to S403). The imaging device 100 also controls the display 105 to display a shooting standby screen for an image to be recorded (S303). The shooting standby screen includes a display of a live view image captured by the camera unit 110. The imaging device 100 acquires an analysis result by performing image analysis on a portion of the live view image displayed in a predetermined area on the shooting standby screen (S804). The imaging device 100 associates the analysis result and the set one or more pieces of medical information with the image to be recorded captured by the camera unit 110 (S603). Here, if one or more pieces of medical information have been set, the imaging device 100 displays area information (e.g., image analysis frame 506) indicating a specified area on the shooting standby screen, and if one or more pieces of medical information have not been set, the imaging device 100 controls to refrain from displaying the area information (S405 to S406).

[0075] Thus, according to this embodiment, if one or more predetermined pieces of medical information (e.g., photographer information, patient information, and affected area information) have not been set, area information (e.g., image analysis frame 506) indicating a predetermined area where image analysis is performed is not displayed. Therefore, according to this embodiment, the possibility that the user's attention will be distracted from the necessary medical information before setting the necessary medical information is reduced, and the possibility that the user will perform imaging without setting the necessary medical information is reduced.

[0076] As explained in S302 of FIG. 3, the CPU 101 can selectively connect to a specific medical department system. At this time, there is a possibility that a medical department that does not require image analysis (a predetermined medical department) is selected. If a medical department that does not require image analysis is selected, the CPU 101 may omit the processes of S304, S307, and S308 of FIG. 3. The CPU 101 may also omit the processes of S405 and S406 of FIG. 4 (i.e., the CPU 101 may perform control to refrain from displaying the image analysis frame 506 regardless of whether one or more predetermined pieces of medical information have been set).

[0077] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0078] [summary] The above-described embodiment discloses at least the inventions shown in the following items, but is not limited to these inventions. (Item 1) An imaging unit; A display unit; An acquisition means for acquiring one or more predetermined pieces of medical information; a setting means for setting the acquired one or more pieces of medical information as medical information to be used for association; a display control means for controlling the display unit to display a shooting standby screen for an image to be recorded, the shooting standby screen including a display of a live view image captured by the imaging unit; and an analysis means for acquiring an analysis result by performing image analysis on a portion of the live view image displayed in a predetermined area on the shooting standby screen; an associating means for associating the analysis result and the one or more pieces of medical information set with the image for recording captured by the imaging unit; Equipped with The display control means When the one or more pieces of medical information have been set, area information indicating the predetermined area is displayed on the imaging standby screen; If the one or more pieces of medical information have not been set, refrain from displaying the area information. Control it so that An imaging device characterized by: (Item 2) When the one or more pieces of medical information have been set and the analysis result has been acquired, the display control means controls so that analysis result information indicating the analysis result is displayed on the imaging standby screen. 2. The imaging device according to item 1, (Item 3) When the one or more pieces of medical information have not been set and the analysis result has been acquired, the display control means controls to refrain from displaying the analysis result information on the imaging standby screen. 3. The imaging device according to item 2, (Item 4) When one or more pieces of medical information have been set, When the analysis result has not been acquired, the display control means controls the display mode of the region information to be a first display mode in which the region information is displayed in a first appearance; When the analysis result has been acquired, the display control means controls the display mode of the area information to be a second display mode different from the first display mode. 4. The imaging device according to any one of items 1 to 3, characterized in that: (Item 5) The second display mode is a mode in which the area information is displayed in a second appearance different from the first appearance, or a mode in which the area information is not displayed. 5. The imaging device according to item 4, (Item 6) When one or more pieces of medical information have been set, In response to a predetermined trigger occurring after the analysis result has been acquired, the analysis means updates the analysis result by performing new image analysis on a portion of a new live view image displayed in the predetermined area; While the new image analysis is being performed, the display control means controls the display mode of the region information to be a third display mode. 6. The imaging device according to item 4 or 5, (Item 7) The third display mode is the same as the first display mode. 7. The imaging device according to item 6, (Item 8) the predetermined trigger is a predetermined user instruction; 8. The imaging device according to item 6 or 7, characterized in that: (Item 9) If the one or more pieces of medical information have not been set, the analysis means periodically performs new image analysis on a portion of a new live view image displayed in the predetermined area, thereby periodically updating the analysis result. 9. The imaging device according to any one of items 1 to 8, characterized in that: (Item 10) the acquiring means acquires the one or more pieces of medical information by communicating with an external device; When the communication with the external device by the acquisition means fails, the display control means controls to display the area information on the imaging standby screen even if the one or more pieces of medical information have not been set. 10. The imaging device according to any one of items 1 to 9, characterized in that: (Item 11) Further provided is a selection means for selecting a medical department, When a predetermined medical department is selected by the selection means, the display control means controls to refrain from displaying the area information regardless of whether the one or more pieces of medical information have been set. 11. The imaging device according to any one of items 1 to 10, characterized in that: (Item 12) A control method executed by an imaging device including an imaging unit and a display unit, an acquisition step of acquiring one or more predetermined pieces of medical information; a setting step of setting the acquired one or more pieces of medical information as medical information to be used for association; a display control step of controlling the display unit to display a shooting standby screen for an image to be recorded, the shooting standby screen including a display of a live view image captured by the imaging unit; an analysis step of acquiring an analysis result by performing image analysis on a portion of the live view image displayed in a predetermined area on the shooting standby screen; an associating step of associating the analysis result and the set one or more pieces of medical information with the recording image captured by the imaging unit; Equipped with The display control step includes: When the one or more pieces of medical information have been set, area information indicating the predetermined area is displayed on the imaging standby screen; If the one or more pieces of medical information have not been set, refrain from displaying the area information. Control it so that A control method comprising: (Item 13) 12. A program for causing a computer to function as each of the means of the imaging device according to any one of items 1 to 11.

[0079] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0080] 100...imaging device, 101...CPU, 102...memory, 103...nonvolatile memory, 104...image processing unit, 105...display, 106...operation unit, 107...recording medium I / F, 109...communication unit, 108...recording medium, 110...camera unit, 150...internal bus

Claims

1. An imaging unit; A display unit; An acquisition means for acquiring one or more predetermined pieces of medical information; a setting means for setting the acquired one or more pieces of medical information as medical information to be used for association; a display control means for controlling the display unit to display a shooting standby screen for an image to be recorded, the shooting standby screen including a display of a live view image captured by the imaging unit; and an analysis means for acquiring an analysis result by performing image analysis on a portion of the live view image displayed in a predetermined area on the shooting standby screen; an associating means for associating the analysis result and the one or more pieces of medical information set with the image for recording captured by the imaging unit; Equipped with The display control means When the one or more pieces of medical information have been set, area information indicating the predetermined area is displayed on the imaging standby screen; If the one or more pieces of medical information have not been set, refrain from displaying the area information. Control it so that An imaging device characterized by:

2. When the one or more pieces of medical information have been set and the analysis result has been acquired, the display control means controls so that analysis result information indicating the analysis result is displayed on the imaging standby screen.

2. The imaging device according to claim 1.

3. When the one or more pieces of medical information have not been set and the analysis result has been acquired, the display control means controls to refrain from displaying the analysis result information on the imaging standby screen.

3. The imaging device according to claim 2.

4. When the one or more pieces of medical information have been set, When the analysis result has not been acquired, the display control means controls the display mode of the region information to be a first display mode in which the region information is displayed in a first appearance; When the analysis result has been acquired, the display control means controls the display mode of the area information to be a second display mode different from the first display mode.

2. The imaging device according to claim 1.

5. The second display mode is a mode in which the area information is displayed in a second appearance different from the first appearance, or a mode in which the area information is not displayed.

5. The imaging device according to claim 4.

6. When the one or more pieces of medical information have been set, In response to a predetermined trigger occurring after the analysis result has been acquired, the analysis means updates the analysis result by performing new image analysis on a portion of a new live view image displayed in the predetermined area; While the new image analysis is being performed, the display control means controls the display mode of the region information to be a third display mode.

5. The imaging device according to claim 4.

7. The third display mode is the same as the first display mode.

7. The imaging device according to claim 6.

8. the predetermined trigger is a predetermined user instruction; 7. The imaging device according to claim 6.

9. If the one or more pieces of medical information have not been set, the analysis means periodically performs new image analysis on a portion of a new live view image displayed in the predetermined area, thereby periodically updating the analysis result.

2. The imaging device according to claim 1.

10. the acquiring means acquires the one or more pieces of medical information by communicating with an external device; When the communication with the external device by the acquisition means fails, the display control means controls to display the area information on the imaging standby screen even if the one or more pieces of medical information have not been set.

2. The imaging device according to claim 1.

11. Further provided is a selection means for selecting a medical department, When a predetermined medical department is selected by the selection means, the display control means controls to refrain from displaying the area information regardless of whether the one or more pieces of medical information have been set.

2. The imaging device according to claim 1.

12. A control method executed by an imaging device including an imaging unit and a display unit, an acquisition step of acquiring one or more predetermined pieces of medical information; a setting step of setting the acquired one or more pieces of medical information as medical information to be used for association; a display control step of controlling the display unit to display a shooting standby screen for an image to be recorded, the shooting standby screen including a display of a live view image captured by the imaging unit; an analysis step of acquiring an analysis result by performing image analysis on a portion of the live view image displayed in a predetermined area on the shooting standby screen; an associating step of associating the analysis result and the set one or more pieces of medical information with the recording image captured by the imaging unit; Equipped with The display control step includes: When the one or more pieces of medical information have been set, area information indicating the predetermined area is displayed on the imaging standby screen; If the one or more pieces of medical information have not been set, refrain from displaying the area information. Control it so that A control method comprising:

13. A program for causing a computer to function as each of the means of the imaging device according to any one of claims 1 to 11.

Images