Imaging device

Abstract

To provide an imaging device for taking an image for examination, the imaging device capable of acquiring information necessary for the examination.SOLUTION: An imaging device having an imaging part for taking an image for examination and a communication part for communicating with a predetermined system includes: selection means for selecting an examination purpose; acquisition means for acquiring imaging control information corresponding to the selected examination purpose from the system through the communication part; and control means for setting at least one of a setting value of an imaging condition, a display content displayed in display means together with the image taken by the imaging part, and items of information associated with the image taken by the imaging part on the basis of the imaging control information acquired according to the examination purpose.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an imaging device that images a patient's affected area for examination, for example, in a medical setting or the like.

Background Art

[0002] There is a medical imaging device that images an image for assisting in the diagnosis of an affected area. Patent Document 1 describes a medical imaging device having two light sources, a first light source and a second light source, that irradiate in different directions. In a first imaging state in which the affected area is imaged in a normal imaging state, the first light source is used, and in a second imaging state in which the affected area is imaged in an imaging state different from the normal imaging state, the second light source is used.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, the light sources used are different between a first imaging state in which the affected area is imaged in a normal imaging state and a second imaging state in which the affected area is imaged in an imaging state different from the normal imaging state. However, there are cases where the imager cannot image an image in an imaging state suitable for the patient or the type of examination and cannot obtain the information necessary for the examination.

[0005] Therefore, an object of the present invention is to provide an imaging device capable of acquiring information necessary for an examination in an imaging device that images an image for examination.

Means for Solving the Problems

[0006] To achieve the above object, the imaging device of the present invention An imaging device having an imaging unit for capturing images for inspection and a communication unit for communicating with a predetermined system, comprising: a selection means for selecting an inspection purpose; an acquisition means for acquiring imaging control information corresponding to the selected inspection purpose from the system via the communication unit; and a control means for setting at least one of the following based on the imaging control information acquired according to the inspection purpose: a set value for the imaging conditions, a display content to be displayed on the display means together with the image captured by the imaging unit, and an item of information to be associated with the image captured by the imaging unit. Part selection means for selecting a part of the target to be photographed based on part information included in the aforementioned imaging control information. It is characterized by having the following: [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an imaging device capable of acquiring information according to the purpose of inspection. [Brief explanation of the drawing]

[0008] [Figure 1] Medical system device configuration diagram showing the device configuration of a medical system. [Figure 2] Block diagram of a digital still camera operating as an imaging device. [Figure 3] Block diagram showing a computer operating as an image processing device. [Figure 4] Sequence diagram showing the process of recording a patient's injury or illness status using a medical system. [Figure 5] Conceptual diagram showing the correspondence between imaging control information and inspection objectives. [Figure 6] Flowchart showing the image capture control process [Figure 7] Flowchart showing the process for verifying captured images [Figure 8] A diagram showing a digital still camera viewed from the back. [Figure 9] A diagram showing a digital still camera viewed from the back. [Figure 10] A diagram showing a digital still camera viewed from the back. [Figure 11] A diagram showing a digital still camera viewed from the back. [Figure 12]Figure showing the digital still camera as seen from the back [Figure 13] Figure showing the digital still camera as seen from the back [Figure 14] Figure showing the digital still camera as seen from the back [Figure 15] Flowchart showing the confirmation process of the captured image [Figure 16] Flowchart showing the inference execution process in the image processing apparatus [Figure 17] Conceptual diagram showing the correspondence between imaging control information and inspection purposes [Figure 18] Flowchart showing the imaging control process [Figure 19] Figure showing the digital still camera as seen from the back [Figure 20] Flowchart showing the determination process of the bird's-eye view image and the enlarged image [Figure 21] Sequence diagram showing the process of recording the patient's injury status using the medical system [Figure 22] Figure showing the digital still camera as seen from the back [Figure 23] Figure showing the digital still camera as seen from the back [Figure 24] Figure showing the digital still camera as seen from the back [Figure 25] Figure showing the digital still camera as seen from the back [Figure 26] Conceptual diagram showing the correspondence between imaging control information and inspection purposes [Figure 27] Sequence diagram showing the process of recording the patient's injury status using the medical system

Mode for Carrying Out the Invention

[0009] 〔Example 1〕 In the medical system of this embodiment, an imaging device that photographs the patient's affected part at the medical site and supports diagnosis will be described. The bedsore that photographs a single affected part and diagnoses the affected part will be taken as an example for explanation.

[0010] In this embodiment, the patients managed by the medical system have their name, age, gender, medical history, etc., pre-registered in the electronic medical record system described later, and are assigned a unique patient management number. A barcode indicating this patient management number is attached to the patient's arm as a wristband.

[0011] Furthermore, in the medical system of this embodiment, information for changing and controlling the setting values ​​of imaging conditions in the imaging device, screen display content (such as imaging assist figures), and image-related information (such as examination evaluation items) is referred to as imaging control information. Details of the imaging control information and the control method of the imaging device will be described later.

[0012] Furthermore, the aforementioned imaging conditions refer to the parameters (settings) used during imaging in the imaging device. Specifically, these include parameters (settings) related to focus position, zoom position, ISO sensitivity, exposure, Tv (shutter speed) value, Av (aperture) value, white balance, flash firing status, color adjustment processing, and edge enhancement processing.

[0013] [System Configuration] Figure 1 is a diagram showing the configuration of the medical system in this embodiment. The medical system in this embodiment consists of devices that perform the following processes.

[0014] The imaging device 101 of the present invention captures images of the affected area 107 of a patient, as described later, and generates visible light image data. In the following description, the images captured by the imaging device 101 will be described as visible light image data. Patient information, such as a patient management number associated with the patient, is added to the captured image data and transmitted to the image processing device 106, which will be described later. The imaging device 101 is a portable, compact imaging device, similar to a commercially available digital camera, and is equipped with a display device for confirming the captured images and input / output devices for various operations such as selecting patient information. The imaging device 101 connects to the network 102, which will be described later, as soon as the power is turned ON by the photographer, and becomes able to communicate with other devices in the medical system via the network 102. It maintains a state where it can communicate continuously until the power is turned OFF.

[0015] The image processing device 106 communicates with the imaging device 101 to transmit the examination purpose and imaging control information (described later), perform inference according to the examination purpose and transmit the results, and acquire patient information from the electronic medical record system 104.

[0016] The electronic medical record system 104, upon receiving a request from the electronic medical record display terminal 103 (described later), communicates with the image processing device 106 and the image management system 105 (described later) to send and receive data related to the creation and editing of electronic medical records. In this embodiment, the electronic medical record system 104 stores the patient's management number within the medical system, name, age, gender, medical history, location of past injuries and illnesses on the body, and diagnosis results in its internal storage device. In response to a patient information acquisition request from the image processing device 106, it can acquire a series of patient information from the patient's management number. Furthermore, in response to a patient information acquisition request from the image processing device 106, it can communicate with the image management system 105 (described later) to acquire image data related to the diagnosis of past injuries and illnesses, as well as images of the inference results described later.

[0017] The electronic medical record display terminal 103 communicates with the electronic medical record system 104 to display and create electronic medical records, accept inputs during creation, and perform information addition processing to images.

[0018] The image management system 105 receives image data captured by the imaging device 101 via the image processing device 106 and records it in its internal storage device, associated with the patient management number and the date and time of capture. When an image data acquisition request is received from the electronic medical record system 104 or the image processing device 106, the system transmits image data that meets the specified conditions to the requesting party.

[0019] The affected area 107 of the patient indicates the area of ​​examination within the medical system. The state of the injury or illness may be diagnosed based on the manifestation of symptoms throughout the body, or based on the manifestation of symptoms in only a part of the body. When the symptoms are throughout the body, the examination covers a wide range of areas, such as the head, trunk, lower limbs, and upper limbs.

[0020] The image processing device 106, imaging device 101, electronic medical record system 104, and image management system 105 communicate via network 102. Furthermore, communication between the electronic medical record display terminal 103 and the electronic medical record system 104 is performed using an HDMI® cable or similar. While network, HDMI®, and USB are given as examples of communication methods, they are not limited to these.

[0021] In the medical system of this embodiment, the imaging device 101, the electronic medical record display terminal 103, the electronic medical record system 104, the image management system 105, and the image processing device 106 are described as separate devices. However, the functions of two or more devices may be realized in a single device, for example, by providing the configuration of the imaging device 101 in the electronic medical record display terminal 103.

[0022] [Imaging device] A digital still camera that operates as the imaging device 101 in this embodiment will be described.

[0023] Figure 2 is a block diagram showing a digital still camera operating as an imaging device according to one embodiment of the present invention. By executing a predetermined control program in the digital still camera, the imaging process described below is realized, and it functions as an imaging device.

[0024] In Figure 2, 200 is the imaging unit, which reads an optical image using a solid-state image sensor and generates electrical image data through analog-to-digital conversion. 201 is the CPU, which controls the entire digital still camera. 202 is a ROM that stores the operating procedures of the CPU 201 (for example, programs for processing when the digital still camera is powered on and basic input / output processing). 203 is RAM, which functions as the main memory of the CPU 201. Various programs, including control programs for implementing the processes described later, are loaded into RAM 203 from ROM 202, etc., and executed by the CPU 201. RAM 203 also provides a work area for the CPU 201 when it executes various processes. 204 is the display device (display unit), which performs various displays under the control of the CPU 201. For example, it displays data stored on a storage medium. It also displays live view images captured by the imaging unit 200, captured images taken by the imaging unit 200 in response to the release button (shutter button) being operated and a shooting command being input, and various setting screens. 205 is an input device such as buttons for performing various operations. For example, it includes the release button located on the top of the digital still camera, and the directional keys and setting keys located on the back. The touch panel provided on the display device 204 is also included as an input device. The user (photographer) can input various instructions to the digital still camera (imaging device 101) by performing user operations on the input device. 206 is a media drive, which allows for the insertion of removable storage media, storage of data, and retrieval of stored data. 207 is a network interface (communication unit) that is connected to the computer network 210 via a wireless communication line 209. Data is sent and received from server computers and personal computers through this network interface. 208 is a system bus (consisting of an address bus, data bus, and control bus) that connects the above-mentioned units. 211 is an image processing unit. The CPU 201 temporarily stores the image data generated by the imaging unit 200 and its attribute information in the RAM 203.Then, if necessary, the image processing unit 211 performs a series of image processing operations to produce image data that matches the visual characteristics of humans. 212 is the file generation unit. The CPU 201 converts the image to a general-purpose still image format in the file generation unit. In this embodiment, it is a JPEG image.

[0025] The hardware configuration of the imaging device 101 in this embodiment is the same as that of a normal digital camera, and it is assumed to be a small, portable digital camera (portable imaging device). Furthermore, a special control program for the medical system of this embodiment is installed as the control program for the imaging device 101 and stored in the ROM 202. Therefore, by preparing a new control program for the medical system, it is possible to realize the imaging device 101 of this embodiment using an existing normal imaging device without preparing a new imaging device for the medical system of this embodiment.

[0026] [Image processing device] A computer operating as the image processing device 106 in this embodiment will be described.

[0027] Figure 3 is a block diagram showing a computer operating as an image processing device according to one embodiment of the present invention. The computer performs the image processing described below by executing a predetermined control program and functions as an image processing device.

[0028] In Figure 3, 301 is the CPU that controls the image processing device. 302 is ROM that stores the operating procedures of the CPU 301 (for example, programs such as computer startup and basic input / output processing). 303 is RAM, which functions as the main memory of the CPU 301. Various programs, including control programs for implementing the processes described later, are loaded into RAM 303 from the hard disk drive 305, etc., and executed by the CPU 301. RAM 303 also provides a work area when the CPU 301 executes various processes. 304 is a display that performs various displays under the control of the CPU 301. 305 is a hard disk drive (hereinafter referred to as HDD), which is used for saving and loading application programs, data, libraries, etc. 306 is an input device such as a pointing device or keyboard. 307 is a storage medium mounting unit (media drive) that mounts a removable storage medium and enables the reading of data captured and stored by a digital still camera. 308 is a network interface (communication unit) and is connected to a computer network 311 via a wireless or wired communication line 310. Data is sent and received from devices that can communicate via this network interface 308. In this embodiment, the imaging device 101 is connected via this network interface, and the CPU 301 sends and receives various data to and from the imaging device 101, acquires images taken by the imaging device 101, and records them on the HDD 305. 309 is a system bus (consisting of an address bus, a data bus, and a control bus) that connects each of the above-mentioned units.

[0029] [Sequence of processes for recording the state of injury or illness] Figure 4 is a sequence diagram showing the process of recording a patient's injury or illness using the medical system in this embodiment. Figure 4 mainly explains the flow of data transmission and reception between the devices that make up the medical system. These processes are realized by each device and system executing the process based on the control program of each device and system.

[0030] In step S401, the imaging device 101 prompts the photographer to photograph the barcode on the patient's wristband, and performs imaging control according to the photographer's shutter button operation, recording the barcode as image data.

[0031] In step S402, the imaging device 101 transmits the barcode image data recorded in step S401, that is, the patient image data corresponding to the patient, to the image processing device 106.

[0032] In step S403, the image processing device 106 receives barcode image data from the imaging device 101, analyzes the image data, and performs barcode reading. The barcode reading process is assumed to be a general reading process, and the symbol and number indicated by the barcode are obtained. In this embodiment, the patient management number of the patient wearing the wristband is obtained.

[0033] In step S404, the image processing device 106 requests patient information from the electronic medical record system 104 using the patient management number obtained in step S403 as a search condition. If information on past medical examination results for illnesses or injuries is stored at that time, the details of the examinations for illnesses or injuries from a predetermined nearby date and time are included in the patient information when the request for acquisition is made. In this embodiment, the predetermined nearby date and time is within one month.

[0034] In step S405, the electronic medical record system 104 searches for patient information from the patient management number based on the patient information acquisition request in step S404.

[0035] In step S406, the electronic medical record system 104 transmits the patient information obtained from the search in step S405 to the image processing device 106.

[0036] In step S407, the image processing device 106 transmits the patient information received from the electronic medical record system 104 in step S406 to the imaging device 101.

[0037] In step S408, the imaging device 101 displays the patient's name, age, and gender, along with the OK button and NG button, on the display device 204 within the imaging device 101, which are patient information received from the image processing device 106 in step S407. The imaging device then prompts the photographer to confirm whether the patient matches the one being examined. If the photographer selects the OK button, the process proceeds to step S409, described below; if the photographer selects the NG button, the process returns to step S401.

[0038] In step S409, if the imaging device 101 contains multiple examination items for illnesses or injuries within the past month, it displays them on the display device 204 as candidates for examination purposes. If there are no examination items, all examination purposes are displayed on the display device 204 as candidates. Hereafter, the screen that displays the candidates for examination purposes will be referred to as the examination purpose selection screen. The display method will be described later.

[0039] In step S410, the imaging device 101 determines whether the operator has selected an inspection purpose. If an inspection purpose has been selected, the process proceeds to step S411. If no selection has been made, the display of candidate inspection purposes in step S409 continues.

[0040] In step S411, the imaging device 101 transmits the examination purpose selected by the photographer in step S410 to the image processing device 106.

[0041] In step S412, the image processing device 106 reads imaging control information corresponding to the inspection purpose from the HDD 305 to the RAM 303. The correspondence between the inspection purpose and the imaging control information will be described later.

[0042] In step S413, the image processing device 106 transmits imaging control information to the imaging device 101.

[0043] In step S414, the imaging device 101 stores imaging control information in the RAM 203 and displays candidate patient body parts on the display device 204 based on the imaging control information. It then prompts the operator to select a body part. Hereafter, the screen displaying the candidate body parts will be referred to as the body part selection screen. The display method will be described later.

[0044] In step S415, the imaging device 101 determines whether or not the photographer has performed a body part selection operation. If a body part selection operation has been performed, the device transitions to a state that accepts imaging operations and proceeds to step S416, which will be described later. If no selection operation has been performed, the display of candidate body parts as in step S414 continues.

[0045] In step S416, the imaging device 101 performs a series of imaging processes in accordance with the photographer's imaging operation (shooting instruction) to the shutter button and stores the generated image data as a diseased area image in the media drive 206. At that time, an image ID that can be uniquely identified in the medical system of this embodiment is generated and recorded in the header of the image data.

[0046] In step S417, the imaging device 101 transmits the image data (image of the affected area) generated in step S416 to the image processing device 106, along with the imaging conditions at the time of shooting, the purpose of the examination selected in S410, the patient information and patient management number received in S407, and the imaging area selected in S416. This information may be included in the image file of the image data and transmitted, or it may be in a separate file. It is sufficient that the image data and each piece of information are associated with each other.

[0047] In step S418, the image processing device 106 associates the image data, imaging conditions, examination purpose, imaging site, and patient management number received from the imaging device 101 with an image ID and stores them in the HDD 305. Then, according to the examination purpose, it loads the image data into the RAM 303, analyzes it, and performs inference to assist in diagnosis. The inference to assist in diagnosis is assumed to be the inference of the area and condition of the affected area using machine learning. The learning model used in machine learning is stored in the HDD 305 and is loaded into the RAM 303 when inference is executed. Then, it also calculates quantifiable information such as the area and condition of the affected area. In the case of a pressure ulcer, it analyzes the area of ​​the affected area, the pressure ulcer portion, the necrotic portion, the inflamed / infected portion, the granulation tissue portion, and depth information. For the pressure ulcer portion, necrotic tissue portion, inflamed / infected portion, granulation tissue portion, and depth information, training data is created from images of pressure ulcer cases taken in advance, based on the color and distribution of pixels that make up the image, indicating which pixels correspond to which conditions. Then, the training data is used to create a learning model, and inference is performed. The pressure ulcer depth information refers not to physical depth information, but to information on how far the tissue beneath the skin is exposed. In the case of other conditions such as xerosis, atopic dermatitis, psoriasis, burns, urticaria, and dental examinations, training data is created from images of each symptom. Then, a learning model is created in the same way as for pressure ulcers, and inference is performed to output the inference result (analysis result). In the following description of this embodiment, the inference process is described as being created and executed using the learning model used in the machine learning described above. However, it is also possible to analyze the image using, for example, brightness information, color information, hue information, gradient information, etc., and perform inference processing using gradient information or the distribution of color or intensity information in a region. Inference processing may also be performed using feature matching using shape feature information of the grayscale distribution or brightness gradient, or pixels with a brightness gradient above a certain level may be extracted as edge information, and rule-based image processing such as position, centroid, and tilt detection may be used or used in combination for inference processing.

[0048] In step S419, the image processing device 106 transmits to the imaging device 101 the image data that constitutes the inference result (analysis result) and numerical information of the state, along with the image ID of the image data that was the target of the inference. The imaging device 101 receives the inference result (analysis result) and image ID transmitted from the image processing device 106 (image analysis result reception).

[0049] In step S420, the imaging device 101 displays numerical information representing the state, which is the inference result (analysis result), superimposed on the image data captured in step S416 (image data corresponding to the received image ID) on the display device 204. In addition, it displays a confirmation button and a retake button to prompt the photographer to confirm that there is no mistake between the target image and the inference result. Hereafter, the screen displaying the inference result will be referred to as the inference result display screen. The display method will be described later.

[0050] In step S421, if the imaging device 101 determines that the confirmation button has been selected by the photographer, it proceeds to step S422. If it determines that the retake button has been selected, it returns to step S416.

[0051] In step S422, the imaging device 101 displays a screen on the display device 204 for selecting evaluation items and candidate evaluation values ​​associated with those evaluation items, based on the imaging control information. It then prompts the photographer to edit the evaluation values. Hereafter, the screen for editing the evaluation values ​​will be referred to as the evaluation value editing screen. If the state can be obtained as a numerical value from the inference results on the evaluation value editing screen, the evaluation value corresponding to that numerical value will be displayed selected. If the state cannot be obtained as a numerical value from the inference results, the evaluation value will be displayed unselected. The evaluation value editing screen also has a "Complete" button. The display method will be described later.

[0052] In step S423, if the imaging device 101 determines that the photographer has selected the "Complete" button, it proceeds to step S424. If the "Complete" button is not selected, the display of the evaluation value editing screen from step S422 continues.

[0053] In step S424, the imaging device 101 transmits the evaluation value selected on the evaluation value editing screen, along with the image ID and patient management number, to the image processing device 106.

[0054] In step S425, the image processing device 106 uses the image ID to associate the image data to be inferred with the evaluation value received from the imaging device 101 and stores it in the RAM 303.

[0055] In step S426, the image processing device 106 converts patient information, examination purpose, imaging site and evaluation value, and image data into a medical image format called Digital Imaging and Communications in Medicine (registered trademark) (hereinafter abbreviated as DICOM).

[0056] In step S427, the image processing device 106 transmits the DICOM® image along with patient information (patient management number, etc.), examination purpose, imaging site, and inference result image to the image management system 105.

[0057] In step S428, the image management system 105 returns a response to the image processing device 106 indicating that transmission is complete.

[0058] In step S429, the image processing device 106 returns a response to the imaging device 101 indicating that transmission is complete.

[0059] [Imaging control information] Figure 5 is a conceptual diagram showing the correspondence between imaging control information and inspection objectives in this embodiment. In this embodiment, the information shown in Figure 5 is assumed to be stored in the HDD 305 of the image processing device 106, and is used when reading the imaging control information corresponding to the inspection objective received from the imaging device 101 in step S411 in step S412.

[0060] Column 501 indicates the purpose of the examination. It corresponds to the injury or illness in the patient's affected area.

[0061] Column 502 is a column indicating imaging conditions. It stores the imaging conditions specified for the imaging device 101 to acquire images appropriate for examination assistance for each examination purpose, and these conditions cannot be changed by the operator. Parameters not specified in the imaging conditions can be changed by the user's input device 205 on the imaging device 101.

[0062] Column 503 represents a list of candidate body parts. These will be the candidate body parts to be displayed on the imaging device 101. Appropriate candidate body parts are maintained for each examination purpose. Depending on the examination purpose, the state of the injury or illness may be diagnosed based on the overall presentation of symptoms throughout the body, or based on the presentation of only some symptoms. The system allows the user to switch between candidate body parts according to their choice of which to record on the imaging device 101.

[0063] 504 is a column showing assist figures during imaging. These figures are displayed superimposed on the subject image on the display device 204 when the affected area is imaged by the imaging device 101. Depending on the purpose of the examination, the state of the injury or illness may be diagnosed based on the appearance of symptoms throughout the body, or based on the appearance of some symptoms. Furthermore, when diagnosing the state of the injury or illness based on the appearance of symptoms throughout the body, the assist figures will switch depending on the selected area.

[0064] Column 505 is a column indicating the confirmation conditions. It holds the conditions for determining whether or not an image suitable for diagnosis has been captured for the purpose of the examination. The confirmation conditions for whether or not an image suitable for diagnosis has been captured will change depending on whether the state of the injury or illness is diagnosed based on the manifestation of symptoms throughout the body or based on the manifestation of some symptoms. The confirmation conditions will be switched according to the photographer's selection of which type of recording they want to record with the imaging device 101.

[0065] Column 506 is a column that shows information related to the display of inference results. It stores whether or not the imaging device 101 displays the inference results from the image processing device 106, and the information that constitutes the inference result.

[0066] Column 507 is a column that shows the evaluation items. It holds evaluation items according to the purpose of the test.

[0067] Column 508 shows the evaluation values ​​for each evaluation item. It holds the values ​​that make up the evaluation items according to the purpose of the test.

[0068] Column 509 indicates whether a diagnosis can be made using multiple images. Depending on the purpose of the examination, the state of the injury or illness may require a diagnosis using multiple images or a diagnosis using only one image; this column records which of the two is the case.

[0069] Column 510 indicates whether imaging of multiple body parts is required. Depending on the purpose of the examination, there may be multiple body parts that must be imaged in order to assess the condition, and this column stores whether there are multiple body parts that need to be imaged. It may also store information on the body parts that are required to be imaged. Furthermore, it may store information so that different information is applied depending on whether the symptoms are to be assessed using the whole body (whole body flag ON) or not.

[0070] Row 511 indicates imaging control information when the injury or illness being examined is a pressure ulcer.

[0071] Row 512 indicates imaging control information when the injury or illness being examined is a burn.

[0072] Row 513 indicates imaging control information when the disease being examined is xerosis (dry skin deficiency).

[0073] Row 514 shows imaging control information when the disease being examined is atopic dermatitis.

[0074] Row 515 indicates imaging control information when the disease being examined is psoriasis.

[0075] Row 516 indicates imaging control information when the disease being examined is urticaria (hives).

[0076] Row 517 indicates imaging control information when the target of the examination is a tooth.

[0077] In step S411, if the purpose of the examination is pressure ulcers, in step S412, the image processing device 106, under the control of the CPU 301, reads the information of row 511 corresponding to pressure ulcers into the RAM 303. Then, in step S413, it transmits imaging control information to the imaging device 101.

[0078] In step S411, if the purpose of the examination is burns, the information from row 512 corresponding to burns is similarly read into RAM 303, and in step S413, imaging control information is transmitted to the imaging device 101.

[0079] Similarly, if the purpose of the examination is for xerosis, atopic dermatitis, psoriasis, urticaria, or dental issues, the imaging control information for the corresponding row is transmitted to the imaging device 101.

[0080] [Image control processing] Figure 6 is a flowchart showing the imaging control process in this embodiment. Using Figures 5 and 6, we will explain how the imaging device 101 changes the imaging conditions, screen display content, and image-related information based on the imaging control information. For processes already explained in Figure 4, the corresponding steps will be described. While Figure 4 mainly explained the flow of data transmission and reception between devices constituting the medical system, Figure 6 will explain the processing in the imaging device 101 in detail. The processing in this flowchart is realized by the imaging device 101 reading the control program and executing the processing based on the read control program.

[0081] In step S601, the CPU 201 acquires patient information from the image processing device 106. This corresponds to step S407. The patient verification in step S408 has already been explained in Figure 4, so the explanation is omitted here.

[0082] Steps S602 to S603 correspond to the processing in step S409.

[0083] In step S602, the CPU201 determines whether there is only one examination purpose included in the patient information. If there is only one, the examination purpose is considered to have been uniquely identified, and the examination purpose is automatically selected without displaying the examination purpose selection screen described later, and the process proceeds to step S605. If there is more than one examination purpose, the process proceeds to step S603.

[0084] In step S603, if the patient information contains multiple test items for illnesses or injuries within the last month, the CPU 201 displays the test purpose selection screen, described later, on the display device 204 as a list of candidate test purposes. If there are no test items, all test purposes are displayed as candidates on the test purpose selection screen, described later, on the display device 204.

[0085] In step S604, the CPU201 determines whether the photographer has selected an inspection purpose. This corresponds to step S410. If it is determined that an inspection purpose has been selected, the process proceeds to step S605. If no inspection purpose has been selected, the process returns to step S603 and continues to display the inspection purpose selection screen.

[0086] In step S605, the CPU 201 transmits the inspection purpose selected by the photographer in step S604 or the inspection purpose uniquely identified in step S602 to the image processing device 106. This corresponds to step S411.

[0087] In step S606, the CPU 201 receives imaging control information from the image processing device 106 and stores it in the RAM 203. This corresponds to step S413.

[0088] Steps S607 to S617 correspond to steps S414 to S416 and provide a more detailed explanation of the processing in the imaging device 101.

[0089] In step S607, the CPU 201 refers to the information in column 510 of the imaging control information and determines whether or not multiple site imaging is required. If it is determined that multiple site imaging is required, the process proceeds to step S609. If it is determined that multiple site imaging is not required, the process proceeds to step S608.

[0090] In step S608, CPU201 turns OFF the multiple site flag, which indicates that imaging of multiple sites is required.

[0091] In step S609, the CPU201 turns on the multiple site flag, indicating that imaging of multiple sites is required.

[0092] In step S610, the CPU 201 refers to the information in column 509 of the imaging control information and determines whether multiple image detection is possible. If multiple image detection is possible, the process proceeds to step S611. If multiple image detection is not possible, the process proceeds to step S613.

[0093] In step S611, the CPU 201 displays a screen on the display device 204 allowing the photographer to choose whether or not to evaluate based on systemic symptoms, and determines whether or not the photographer has selected to evaluate based on systemic symptoms. If evaluation based on systemic symptoms is selected, the evaluation of systemic symptoms is performed based on evaluations of multiple body parts, not just one. If the photographer selects to evaluate based on systemic symptoms, the process proceeds to step S612. If the photographer selects not to evaluate based on systemic symptoms, the process proceeds to step S613.

[0094] In step S612, the CPU201 turns on the whole-body flag, which indicates that multiple body parts will be photographed and an evaluation of systemic symptoms will be performed based on the images taken from multiple body parts.

[0095] In step S613, CPU201 turns OFF the systemic symptoms flag, which indicates that an assessment of systemic symptoms will be performed.

[0096] In step S614, the CPU 201 refers to the information in column 503 of the imaging control information and displays a body part selection screen on the display device 204 for selecting a body part to be imaged from among the body part candidates. The CPU 201 refers to the imaging control information and displays only the body parts that the photographer can select as body part candidates on the body part selection screen, and does not display body parts that cannot be selected. In addition, if the body part candidates differ depending on whether the whole-body flag is ON or OFF, the CPU 201 displays the body part that matches the whole-body flag as a body part candidate on the body part selection screen and makes it available for selection by the photographer. Regarding the display of the body part selection screen, it is preferable to refer to the information in column 510 of the imaging control information and display body parts that are required to be imaged in a way that makes it identifiable that they are required to be imaged. For example, it is preferable to display text, symbols, or marks indicating that they are required to be imaged along with the body part candidates, or to display required body part candidates and non-required body part candidates in different colors.

[0097] In step S615, the CPU 201 determines whether the photographer has selected a body part on the body part selection screen. This corresponds to step S415. If a body part has been selected, the CPU 201 determines the selected body part to be photographed and proceeds to step S616 to photograph the determined body part. If no selection operation is performed, the display of the body part selection screen in step S614 continues.

[0098] In step S616, the CPU 201 refers to the information in the imaging condition column 502 within the imaging control information and controls the imaging unit 200 so that imaging can be performed with parameters that match the imaging conditions. If the imaging conditions differ depending on whether the whole-body flag is ON or OFF, the imaging conditions that match the whole-body flag are adopted as parameters.

[0099] In step S617, the CPU 201 refers to the information in column 504 of the assist figures in the imaging control information and displays the imaging assist figure superimposed on the captured live view image. The imaging assist figure is a figure that assists the photographer in taking appropriate examination images, and the type of assist image superimposed on the live view image is changed according to the imaging control information. Also, if the imaging assist figure differs depending on whether the whole-body flag is ON or OFF, the imaging assist figure that matches the whole-body flag is adopted. If the imaging assist figure is the contour line of the affected area from the previous imaging (previous examination) of the selected area, the image of the affected area from the previous imaging acquired as patient information in step S601 is read and the contour line of the affected area is drawn. However, if there is no corresponding image within the predetermined period, the imaging assist figure will not be drawn. In this embodiment, the predetermined period is set to one month.

[0100] In step S618, the CPU 201 performs a series of imaging processes using the imaging unit 200, image processing unit 211, and file generation unit 212 according to the photographer's imaging operation, and stores the generated image data in the media drive 206. This corresponds to step S416.

[0101] In step S619, the CPU 201 displays the image data generated in step S618 on the display device 204.

[0102] In step S620, the CPU 201 refers to the information in the confirmation condition column 505 within the imaging control information and determines whether image data suitable for the examination purpose has been generated. If the confirmation conditions differ depending on whether the whole-body flag is ON or OFF, the confirmation conditions that match the whole-body flag are adopted. If the confirmation conditions are met, it is determined that an image suitable for the examination purpose has been generated. If the confirmation conditions are not met, it is determined that an image suitable for the examination purpose has not been generated. If it is determined that an image suitable for the examination purpose has been generated, the process proceeds to step S621. If it is determined that an image suitable for the examination purpose has not been generated, information to support imaging is displayed according to the unmet confirmation conditions, and the process returns to step S618 to prompt the photographer to retake the image. The confirmation process in step S620 will be described later using Figure 7.

[0103] In step S621, the CPU 201 refers to the information in the inference result display status column 506 within the imaging control information and determines whether or not to display the inference result on the display device 204 of the imaging device 101. If it is determined that the inference result should be displayed, the process proceeds to step S622. If it is determined that the inference result should not be displayed, the process proceeds to step S626.

[0104] In step S622, the CPU 201 transmits the image data, along with the imaging conditions at the time of acquisition, the purpose of the examination, and the patient management number, to the image processing device 106. This corresponds to step S417.

[0105] In step S623, the CPU 201 receives image data representing the inference result and numerical information representing the state from the image processing device 106, along with the image ID of the image data targeted for inference. This corresponds to step S419.

[0106] In step S624, the CPU 201 displays the inference results and numerical information representing the state superimposed on the captured image data. This corresponds to step S420.

[0107] In step S625, the CPU 201 determines whether the verification of the inference result is complete. If it determines that the photographer has selected the verification button, the process proceeds to step S626. If it determines that the retake button has been selected, the process returns to step S618. This corresponds to step S421.

[0108] In step S626, the CPU 201 refers to the evaluation item column 507 in the imaging control information and determines whether or not there is an evaluation value that should be edited in the imaging device 101. If it is determined that there is an evaluation value that should be edited in the imaging device 101, the process proceeds to step S627. If it is determined that there is no evaluation value that should be edited in the imaging device 101, the process proceeds to step S629.

[0109] In step S627, the CPU 201 configures the screen based on the information in the evaluation item column 507 and evaluation value column 508 within the imaging control information, and displays the evaluation value editing screen, described later, on the display device 204. This corresponds to step S422.

[0110] In step S628, the CPU201 determines whether the photographer has selected the "edit complete" button. If it determines that the "edit complete" button has been selected, the process proceeds to step S629. If the "edit complete" button has not been selected, the process returns to step S627 and continues to display the evaluation value editing screen. This corresponds to step S423.

[0111] In step S629, the CPU 201 stores the scanned body parts in the RAM 203.

[0112] In step S630, the CPU201 determines whether the whole-body flag or the multiple-part flag is ON. If the whole-body flag or the multiple-part flag is ON, the process proceeds to step S631. If both the whole-body flag and the multiple-part flag are OFF, the CPU determines that imaging of multiple parts is unnecessary and proceeds to step S634.

[0113] In step S631, the CPU 201 refers to column 510 in the imaging control information indicating whether multiple body parts must be photographed, and compares it with the body parts already photographed stored in RAM 203 in step S629 to determine whether there are any body parts that have not been photographed. Depending on whether the whole-body flag is ON or OFF, if the body parts that must be photographed differ, the information of the body parts that match the whole-body flag is used. If it is determined that there are body parts that have not been photographed, the process returns to step S614. If it is determined that there are no body parts that have not been photographed, the process proceeds to step S632.

[0114] In step S632, CPU201 determines whether the whole body flag is ON or OFF. If the whole body flag is ON, the process proceeds to step S633. If the whole body flag is OFF, the process proceeds to step S634.

[0115] In step S633, CPU201 calculates evaluation values ​​to assess the disease based on systemic symptoms. The calculation method for the systemic evaluation value of atopic dermatitis, known as easiScore, is shown in Equation 1. In Equation 1, the site scores for erythema, weeping / papules, scratch marks, lichenification, and weeping area of ​​the head / neck are designated as head1, head2, head3, head4, and head5, respectively. The site scores for erythema, weeping / papules, scratch marks, lichenification, and weeping area of ​​the trunk are designated as body1, body2, body3, body4, and body5, respectively. The site scores for erythema, weeping / papules, scratch marks, lichenification, and weeping area of ​​the upper limbs are designated as arm1, arm2, arm3, arm4, and arm5, respectively. The site scores for erythema, weeping / papules, scratch marks, lichenification, and weeping area of ​​the lower limbs are designated as foot1, foot2, foot3, foot4, and foot5, respectively. easiScore= (head1+head2+head3+head4)×head5×0.1+(body1+body2+body3+body4)×body5×0.3+(arm1+arm2+arm3+arm4)×arm5×0.2+(foot1+foot2+foot3+foot4)×foot5×0.4…Formula 1

[0116] Similarly, burns, xerosis, psoriasis, and urticaria will also be calculated based on a general disease evaluation formula derived from systemic symptoms.

[0117] In step S634, the CPU 201 transmits the evaluation value selected in step S627 and the evaluation value calculated in step S632 to the image processing device 106 along with the image ID and patient management number. If there is no evaluation value, the CPU 201 transmits the unsent image data along with the patient management number to the image processing device 106.

[0118] [Image verification process] Figure 7 is a flowchart showing the image verification process in this embodiment. This process verifies whether an image suitable for the examination purpose has been generated. This is a detailed explanation of the process in step S620 described above. The explanation will be given using the case where the examination purpose is pressure ulcers as an example. Note that the process in this flowchart is realized when the imaging device 101 reads the control program and executes the process based on the read control program.

[0119] In step S701, the CPU 201 reads the information from the confirmation condition column 505 in the imaging control information into the RAM 203. If the purpose of the examination is pressure ulcers, the following will be confirmed in steps S702 and later, as described below: focus position: center, subject distance: within a predetermined range, facing the subject directly, no breaks in the area where a predetermined color is distributed, and brightness / edge evaluation value: within a predetermined range. For evaluation, the image data and the imaging conditions and measurement information from the distance sensor for focusing, which are stored in the RAM 203 in association with the image data, will be used.

[0120] In step S702, the CPU 201 determines whether the focus position at the time of imaging is in the center of the image. The focus position in the image is assumed to be stored as coordinate information in the header information attached to the image data. If the focus position is within a predetermined range from the center coordinates of the image, the CPU determines that the focus position is in the center and proceeds to step S703. If the CPU determines that the focus position is not in the center, it proceeds to step S709.

[0121] In step S703, the CPU 201 determines whether the distance between the sensor surface of the imaging device 101 and the subject is within a predetermined range. The distance between the sensor surface and the subject can be calculated based on the information from the distance measuring sensor at the time of imaging. The subject distance information is stored as a numerical value indicating the distance in the header information attached to the image data. If the subject distance is determined to be within the predetermined range, the process proceeds to step S704. If it is determined to be outside the predetermined range, the process proceeds to step S710.

[0122] In step S704, the CPU 201 determines whether the sensor surface of the imaging device 101 is directly facing the subject. Whether or not they are directly facing is determined using the distance measurement evaluation value on the image. If the subject distance differs significantly at the top, bottom, right, and left of the image, it is determined that the sensor is not directly facing the subject. If it is determined that the sensor is directly facing the subject, the process proceeds to step S705. If it is determined that the sensor is not directly facing the subject, the process proceeds to step S713.

[0123] In step S705, the CPU201 determines whether the area where the predetermined color is distributed is cut off in the image. The predetermined color is, in the case of a pressure ulcer, an area that is redder than the normal skin color. The determination is made based on whether the area where the predetermined color is distributed extends beyond the edge of the image. If it is determined that it extends beyond the edge, the process proceeds to step S714. If it is determined that it does not extend beyond the edge, the process proceeds to step S706.

[0124] In step S706, the CPU 201 determines whether the brightness of the image is within a predetermined range. This determination is made by checking whether the average value of the pixel values ​​that make up the image is within a predetermined range. If it is determined that the brightness of the image is within the predetermined range, the process proceeds to step S707. If it is determined that the brightness of the image is outside the predetermined range, the process proceeds to step S715.

[0125] In step S707, the CPU 201 determines whether the edge evaluation value is within a predetermined range. The edge evaluation value is the average of the difference between the pixel values ​​of brightness of the original image and the image after applying a smoothing filter to the pixel values ​​of brightness of the image. If the edge evaluation value is smaller than the predetermined value, it is determined that the image data lacks detail. If it is determined that the edge evaluation value is within the predetermined range, the process proceeds to step S708. If it is determined that the edge evaluation value is outside the predetermined range, the process proceeds to step S718.

[0126] In step S708, the CPU 201 determines that an image suitable for the inspection purpose has been generated and stores it in the RAM 203 as an OK confirmation result, associated with the image data.

[0127] In step S709, the CPU 201 notifies the photographer that the focus position is off-center by displaying a message on the display device 204 instructing them to take the picture so that the affected area is in the center.

[0128] In step S710, the CPU 201 determines whether the subject distance is too far. If it determines that the subject distance is too far, the process proceeds to step S711. If it determines that the subject distance is not too far, i.e., too close, the process proceeds to step S712.

[0129] In step S711, the CPU 201 notifies the photographer that the subject is too far away by displaying "The distance to the subject is too far" on the display device 204.

[0130] In step S712, the CPU 201 notifies the photographer that the subject is too close by displaying "The distance to the subject is too close" on the display device 204.

[0131] In step S713, the CPU 201 notifies the photographer that the subject is not facing the imaging device directly, by displaying on the display device 204 a message instructing the photographer to ensure that the subject and the camera are facing each other directly.

[0132] In step S714, the CPU 201 notifies the photographer that the affected area extends beyond the image by displaying on the display device 204 that the affected area may not be included in the image.

[0133] In step S715, the CPU 201 determines whether the image brightness is too bright or not. If it determines that it is too bright, the process proceeds to step S716. If it determines that it is not too bright, i.e., too dark, the process proceeds to step 717.

[0134] In step S716, the CPU 201 notifies the photographer that the shooting environment is too bright by displaying on the display device 204 "The shooting environment is too bright. Please darken it."

[0135] In step S717, the CPU 201 notifies the photographer that the shooting environment is too dark by displaying "The shooting environment is too dark. Please make it brighter" on the display device 204.

[0136] In step S718, the CPU 201 notifies the photographer that the image data is significantly lacking in edges and detail. This notification displays on the display device 204, for example, "There may be camera shake or subject blur. Please keep the camera as still as possible and ensure the patient does not move while taking the picture."

[0137] In step S719, the CPU 201 determines that an image suitable for the inspection purpose has not been generated, and stores the confirmation result as "NG" in the RAM 203, associating it with the image data. It then prompts the user to retake the image.

[0138] [User interface for imaging device] Figure 8 shows a rear view of a digital still camera operating as an imaging device according to one embodiment of the present invention. Details of the various buttons and user interface of the digital still camera operating as an imaging device according to this embodiment will be explained using Figure 8. Figure 8 is an example showing the inspection purpose selection screen mentioned above. All buttons except the power button 801 will be explained when the digital still camera is powered on.

[0139] 801 is the power button that switches the power ON / OFF. When the digital still camera is turned off, if the photographer presses it, CPU201 determines that the photographer has given a power input command and turns the power on. When the power is on, if the photographer presses it, CPU201 determines that the photographer has given a power off command and turns the power off.

[0140] 802 is the shutter release button. When the photographer presses the shutter release button 802, the CPU 201 determines that a still image capture command has been issued.

[0141] 803 is the up button, 804 is the right button, 805 is the down button, 806 is the left button, and 807 is the select button, all of which function as input devices 205. When the photographer presses buttons 803 through 806, the CPU 201 determines that the photographer has performed a selection target switching operation and switches the selection target on the display 808, which will be described later. When the photographer presses button 807, the CPU 201 determines that the photographer has performed a select operation and stores the selected information in RAM 203, while also switching the state of the imaging device 101. This corresponds to the selection operation in step S604 and step S614 described above.

[0142] 808 is a display and functions as a display device 204. 809 has both display and touch panel functions and functions as an input device 205. When the photographer presses any point on the screen with their finger, the CPU 201 determines that there is an input instruction from the photographer, determines the content of the operation from the pressed position, and performs various processes such as display updates.

[0143] 809 is the screen title, displaying a string of text that informs the photographer what they need to select.

[0144] 810 is the focus frame, a frame that informs the photographer of the item currently selected.

[0145] Area 811 is the selection item display area, which lists the candidate options.

[0146] 812 is a scroll bar, used to switch the display area when the selection options cannot all be displayed in the selection area.

[0147] When the photographer presses buttons 803 through 806, the CPU 201 determines that the photographer has made a selection change operation and moves the focus frame 810 within the selection item display area 811.

[0148] Figure 9 shows a rear view of a digital still camera operating as an imaging device according to one embodiment of the present invention. An example of a site selection screen for the purpose of examination of pressure ulcers is shown. Components that perform the same function as those in Figure 8 are numbered according to the corresponding numbers in Figure 8.

[0149] 901, 902, 903, 904, 905, 906, and 907 are candidate sites. As explained in S614, row 511 of the imaging control information shows candidate site column 503 where the examination purpose is pressure ulcer.

[0150] Figure 10 shows a view from the rear of a digital still camera operating as an imaging device according to one embodiment of the present invention. An example is shown where an image assistance diagram is drawn for the purpose of examination when the purpose is pressure ulcers.

[0151] 1001 is an example of patient information display, showing the patient's name, gender, and age. To prevent the photographer from mistaking the patient, this information is superimposed on the subject image during shooting.

[0152] 1002 is the date information for the previous photo shoot.

[0153] 1003 represents the outline of the affected area from the previous image. This is drawn to obtain image data that is easy to compare when examining the same area on the same patient.

[0154] Information 1001 to 1003 is included in the patient information received from the image processing device 106 in step S601.

[0155] Figure 11 shows a view from the rear of a digital still camera operating as an imaging device according to one embodiment of the present invention. An example of the inference result display screen when the purpose of the examination is pressure ulcers is shown.

[0156] 1101 is an example of displaying quantified inference results. It displays the calculated area of ​​the pressure ulcer portion, necrotic portion, inflamed / infected portion, and granulation tissue portion in a pressure ulcer. The area is calculated by the image processing device 106, which uses trigonometric functions to determine the correspondence between the distance on the image and the distance in the real world from the subject distance and field of view at the time of imaging, and then calculates the area from the number of pixels that make up the image.

[0157] Image 1102 is an example where the pressure ulcer area was superimposed and colored onto the image.

[0158] Image 1103 is an example where the necrotic area has been superimposed and colored onto the image.

[0159] Image 1104 is an example where the inflamed and infected area has been superimposed and colored onto the image.

[0160] Button 1105 is a confirmation button. If the photographer presses the OK button 807 after selecting it using the right direction button 806, the CPU 201 determines that the photographer has selected the confirmation button.

[0161] Button 1106 is a retake button. If the photographer presses the OK button 807 after selecting it using the left direction button 804, the CPU 201 determines that the photographer has selected the retake button. This corresponds to the processing in steps S421 and S625.

[0162] Figure 12 shows a view from the rear of a digital still camera operating as an imaging device according to one embodiment of the present invention. An example of the evaluation value editing screen when the purpose of the examination is pressure ulcers is shown.

[0163] Examples 1201 and 1202 show evaluation values ​​that could not be calculated from inference. Even if an evaluation value cannot be obtained from inference from image data, the system can observe the affected area and input the evaluation value that should be recorded at the same time. Furthermore, if the photographer observes the affected area and decides that the inferred evaluation value should be changed, the photographer will perform an operation to change the evaluation value. When the photographer aligns the focus frame 810 with the evaluation value to be changed using the up, down, left, and right direction buttons 803, 804, 805, and 806 and presses the confirm button 807, the CPU 201 determines that the photographer has performed an evaluation value change operation and displays candidate evaluation values. Then, when the photographer moves the focus frame 810 using the up and down direction buttons 803 and 805 and presses the confirm button 807, the CPU 201 determines that an evaluation value selection operation has been performed. The CPU 201 then updates the display with the selected evaluation value and stores the selected evaluation value in the RAM 203.

[0164] 1203 is the complete button. If the photographer presses the OK button 807 after selecting it using the down arrow button 807, the CPU 201 determines that the photographer has selected the complete button. This corresponds to the processing in steps S423 and S628.

[0165] In this embodiment, in step S634, the evaluation value is transmitted to the image processing device 106 along with the image ID and patient management number. However, if the image of the affected area that forms the basis of the evaluation value is stored in the RAM 203 or media drive 206 of the imaging device 101, the evaluation value may be written as header information for the image and transmitted to the image processing device 106 along with the image data. In this case, the image processing device 106 can retain the evaluation value as supplementary information for the image without having to associate the image ID with the target image after receiving the evaluation value and image ID.

[0166] In this embodiment, when there is only one examination objective associated with the patient, the imaging device's examination objective candidate screen is not displayed. However, all examination objectives may always be displayed as candidates, or the candidate examination objectives may be set in the image processing device according to the clinical department that manages the imaging device. By configuring the imaging device to select from all examination objectives as candidates, various examinations can be performed simultaneously on a single patient. Furthermore, if the image processing device is configured to select the examination objective and the imaging device receives it, and the examination objective is set in the image processing device, it is possible to prevent the photographer from mistakenly selecting the wrong examination objective.

[0167] In this embodiment, the evaluation items were as follows: DESIGN-R® (registered trademark) if the injury or illness being examined was a pressure ulcer; the specified symptom sum score system with grading of scaling, roughness, redness, and cracks (hereinafter abbreviated as SRRC) if it was xerosis; Eczema Area and Severity Index (hereinafter abbreviated as EASI) if it was atopic dermatitis; Psoriasis Area Severity Index (hereinafter abbreviated as PASI) if it was psoriasis; and Total Body Surface Area (hereinafter abbreviated as TBSA) and burn depth if it was a burn. However, if the evaluation items and evaluation values ​​differ depending on the country or region, the evaluation item column 507 and evaluation value 508 of the imaging control information shown in Figure 5 can be changed. In this way, it is possible to accommodate new evaluation items and evaluation values ​​without modifying the program in the imaging device.

[0168] As explained above, the imaging device of this embodiment receives imaging control information from the image processing device according to the inspection purpose and changes the imaging conditions, screen display content, and image-related information. Therefore, even if a new inspection purpose is added, it only needs to be added to the imaging control information shown in Figure 5. In this way, it is possible to respond to new inspection purposes without modifying the program in the imaging device.

[0169] As explained above, the imaging device of this embodiment determines whether the inspection objective can be confirmed from the captured image and notifies the user of the determination result. Therefore, the photographer can confirm whether an appropriate image has been captured. Furthermore, because the photographer is notified of any problems during shooting, it becomes possible to record stable image data regardless of the photographer's skill level.

[0170] In this embodiment, as explained in Figure 11, the inference results for pressure ulcers include the area of ​​the affected area, the pressure ulcer portion, the necrotic portion, the inflamed / infected portion, the granulation tissue portion, and depth information. For other examination purposes, the results inferred by the image processing device should be displayed as shown in the inference result display column 506 in Figure 5. For xerosis, the area of ​​the affected area, the xerotic portion, the area of ​​the affected area, skin roughness, skin redness, and skin cracks should be displayed. For atopic dermatitis, the area of ​​the affected area, the erythematous portion, the moist / papulous portion, the scratched portion, and the lichenified portion should be displayed. For psoriasis, the area of ​​the affected area, the erythematous portion, the moist portion, and the desquamated portion should be displayed. For burns, the area of ​​the affected area, the burn portion, and the burn depth should be displayed. For urticaria, the area of ​​the affected area, the wheal portion, and the number of urticaria should be displayed. Furthermore, if the purpose of the examination is for teeth, i.e., injuries or diseases in the oral cavity, the imaging device may display the type of tooth, the presence or absence of teeth, the presence or absence of prosthetics, the presence or absence of caries, the carious area, and the periodontal disease area. In addition, if the performance of inference in the image processing device improves and information useful for evaluating injuries or diseases becomes available, it may be added. If it is difficult to synthesize all inference results on the inference result display screen of the imaging device, the image processing device may generate and transmit the image that the imaging device expects to display.

[0171] As described above, according to the system comprising the imaging device and image processing device of this embodiment, based on imaging control information, the captured image is transmitted to the image processing device, inference is performed by the image processing device, and the result is received by the imaging device. Therefore, even if the performance of the inference or image processing algorithm improves due to improvements in the performance of machine learning learning models, inference algorithms, or image processing algorithms, the system can benefit from the performance improvement without modifying the program in the imaging device. In addition, if it is difficult for the imaging device to synthesize the inference results, the image processing device can generate and transmit a pre-synthesized image, and the imaging device can display the received image.

[0172] As described above, the imaging device of this embodiment changes the inspection evaluation items according to the inspection purpose based on the imaging control information. Then, the evaluation value is stored attached to the image. As a result, the evaluation value corresponding to the inspection purpose can be recorded on the spot, thus preventing recording omissions. Furthermore, if the inference result in the image processing device is inappropriate, it can be easily corrected on the spot.

[0173] As explained above, the imaging device of this embodiment captures images under imaging conditions appropriate to the inspection purpose based on imaging control information. Therefore, it is possible to record stable image data regardless of the skill level of the photographer.

[0174] As explained above, the imaging device of this embodiment changes the candidate body parts depending on whether the whole-body flag is turned ON or OFF. When the whole-body flag is ON, the candidate body parts are enlarged, and when the whole-body flag is OFF, the candidate body parts are reduced in size. In other words, the granularity of the body part information is changed depending on whether or not the disease or injury being examined is to be diagnosed as a proportion of the patient's whole body. When the purpose of the examination is to evaluate the state of the disease or injury from symptoms throughout the body, it will be necessary to take images of multiple body parts, but by increasing the granularity, the number of images taken to photograph the whole body can be reduced. In addition, it is possible to prevent forgetting to photograph candidate body parts and to calculate an evaluation value of the disease or injury for the whole body.

[0175] In this embodiment, the example of drawing the contour line of the affected area from the previous imaging session was used as an imaging assist figure. However, depending on the purpose of the examination, the whole-body flag, and the body part being imaged, contour lines corresponding to body parts may be displayed. For example, in cases of illness or injury diagnosed based on the proportion of symptoms throughout the body, if the imaging site is the trunk, the contour lines of the shoulders and torso should be displayed. If the imaging site is the head / neck, the contour line from the head to the neck should be displayed; if it is the upper limbs, the contour line of the upper limbs should be displayed; and if it is the lower limbs, the contour line of the lower limbs should be displayed.

[0176] As described above, the imaging device of this embodiment displays a shooting assist figure corresponding to the inspection purpose superimposed on the live view image based on the imaging control information. Therefore, it is possible to record stable image data that is easy to compare even if the photographer or the shooting date is different.

[0177] As described above, the imaging device of this embodiment transmits the examination purpose, patient information, image data, inference results, and evaluation values ​​to the image management system in association with each other. Therefore, it becomes easy to observe the progress of each patient examination purpose, and by displaying the examination purpose, body part selection, and assist figures during the next imaging, selection and imaging operations on the imaging device become simpler.

[0178] As explained above, the imaging device of this embodiment includes information in the imaging control information regarding the purpose of the examination and whether or not imaging is necessary depending on the area. Furthermore, if there are areas that have not been imaged that require imaging, the device will notify the user, thus preventing the user from forgetting to image areas that require imaging.

[0179] As described above, the imaging device of this embodiment allows for appropriate shooting, image processing, and recording of inspection items according to the inspection purpose, even when conditions such as the relative position between the imaging device and the subject, and the light source in the shooting environment cannot be fully controlled.

[0180] [Example 2] In this embodiment, we will explain using a dental examination as an example, in which multiple affected areas are photographed and a diagnosis is made using the image processing device based on the multiple images.

[0181] First, we will explain the imaging control process when the purpose of the examination is teeth, using Figure 6. We will focus on aspects unique to dental examinations, and will omit explanations when they are the same as for other examination purposes.

[0182] In step S605, the CPU 201 transmits the inspection objective as "teeth" to the image processing device 106.

[0183] In step S606, the CPU 201 receives imaging control information from the image processing device 106 and stores it in the RAM 203. If the purpose of the examination is teeth, the CPU 201 receives the information for row 507 for dental examination as imaging control information.

[0184] In step S607, the CPU 201 refers to the information in column 510 of the imaging control information and determines whether multiple imaging sites are required. As indicated in the imaging control information in row 507 for dental examination, multiple imaging sites are required, so the process proceeds to step S609.

[0185] In step S610, the CPU 201 refers to the information in column 509 of the imaging control information and determines whether multiple images can be judged. As indicated in the imaging control information in row 507 for dental examination, multiple images can be judged, so the process proceeds to step S611.

[0186] In step S611, the CPU201 displays a screen allowing the photographer to choose whether or not to evaluate based on systemic symptoms, and determines whether or not the photographer selected to evaluate based on systemic symptoms. In the case of a dental examination, there is no evaluation based on systemic symptoms, so the process proceeds to step S613.

[0187] In step S614, the CPU 201 refers to the information in column 503 of the imaging control information and displays candidate sites on the site selection screen. As per the imaging control information in row 507 for dental examination, the maxilla, lower jaw, frontal view, left view, and right view are listed. An example of the site selection screen for dental examination is shown in Figure 13. Figure 13 will be discussed later.

[0188] In step S616, the CPU 201 refers to the information in the imaging condition column 502 within the imaging control information and controls the imaging unit 200 so that imaging can be performed with parameters that match the imaging conditions. The CPU controls the focus position, zoom position, exposure, ISO, Tv value, Av value, WB, and strobe flash according to the imaging control information in row 507 for dental examination. The focus position is set to the position on the image plane of the tooth selected in the area selection. If the maxilla is selected, it is the position of the convex dental arch; if the mandible is selected, it is the position of the concave dental arch; and for the frontal, left, and right views, it is the central position.

[0189] In step S617, the CPU 201 refers to the information in column 504 of the assist figures in the imaging control information and draws the imaging assist figures superimposed on the image of the subject. As per the imaging control information in row 507 for dental examination, for the maxilla and lower jaw, the contour line of the dental arch and a vertical line passing through the center are drawn, and for the frontal, left, and right views, a vertical line passing through the center and a horizontal line are drawn. An example of the imaging assist screen for dental examination is shown in Figure 14. Figure 14 will be described later.

[0190] In step S620, the CPU 201 refers to the information in the confirmation condition column 505 within the imaging control information and determines whether image data suitable for the examination purpose has been generated. The determination is made according to the imaging control information in row 507 for dental examination. The confirmation process in step S620 will be described later with reference to Figure 15.

[0191] In step S621, the CPU 201 refers to the information in column 506 of the imaging control information regarding the display of inference results and determines whether or not to display the inference results on the display device 204 of the imaging device 101. As indicated in the imaging control information for row 507 for dental examination, there is no display of inference results, so the process proceeds to step S626.

[0192] In step S626, the CPU 201 refers to the evaluation item column 507 in the imaging control information and determines whether or not there is an evaluation value that should be edited in the imaging device 101. As indicated in the imaging control information for row 507 for dental examination, there is none, so the process proceeds to step S629.

[0193] In step S629, the CPU 201 stores the scanned body parts in the RAM 203.

[0194] In step S630, the CPU201 determines whether the whole-body flag or the multiple-part flag is ON. In the case of a dental examination, as mentioned above, the multiple-part flag is ON, so the process proceeds to step S631.

[0195] In step S631, the CPU 201 refers to column 510 in the imaging control information to determine whether multiple areas must be imaged, and compares it with the imaged areas stored in RAM 203 in step S629 to determine whether there are any areas that have not been imaged. As indicated in the imaging control information in row 507 for dental examination, both the upper and lower jaws are required. If it is determined that there are areas that have not been imaged by comparing it with the imaged areas stored in RAM 203, the process returns to step S614. If it is determined that there are no areas that have not been imaged, the process proceeds to step S632.

[0196] In step S632, CPU201 determines whether the whole-body flag is ON or OFF. In the case of a dental examination, as mentioned above, the whole-body flag is OFF, so the process proceeds to step S634.

[0197] In step S634, the CPU 201 transmits the evaluation value selected in step S627 and the evaluation value calculated in step S632 to the image processing device 106 along with the image ID. If there is no evaluation value, the CPU 201 transmits the untransmitted image data, the imaging conditions at the time of shooting, the purpose of the examination, and the patient management number to the image processing device 106. In the case of a dental examination, since there is no evaluation value, the CPU 201 transmits the untransmitted image data, the imaging conditions at the time of shooting, the purpose of the examination, and the patient management number to the image processing device 106.

[0198] Figure 13 shows a rear view of a digital still camera operating as an imaging device according to one embodiment of the present invention. An example of a site selection screen when the purpose of examination is teeth is shown. Components that perform the same function as those in Figures 8 and 9 are numbered according to the corresponding numbers in Figures 8 and 9.

[0199] 1301 and 1302 both indicate the shooting status.

[0200] 1301 indicates that area 901 of the target patient has not been imaged.

[0201] 1302 indicates that area 902 of the target patient has already been imaged.

[0202] Whether or not a part has been photographed is determined by whether the CPU 201 has stored it in the RAM 203 as a photographed part. For parts that have been photographed, it is preferable to display that they have been photographed for all parts. However, for parts that have not yet been photographed, information indicating that they have not been photographed may be displayed only for parts that are required to be photographed.

[0203] Figure 14 shows a rear view of a digital still camera operating as an imaging device according to one embodiment of the present invention. It shows an example of drawing an imaging assist figure when the purpose of examination is teeth and the area to be photographed is the upper jaw. Components that perform the same function as those in Figures 8 and 10 are numbered with the corresponding numbers in Figures 8 and 10.

[0204] 1001 is an example of patient information display, showing the patient's name, gender, and age. To prevent the photographer from mistaking the patient, this information is superimposed on the subject image during shooting.

[0205] 1401 represents the outline of the dental arch. For the maxilla, a convex arch shape is drawn; for the mandible, a concave arch shape is drawn.

[0206] 1402 is a vertical line passing through the center point of the field of view that represents the subject image.

[0207] By using the 1401 and 1402 imaging assist diagrams as a reference when imaging the affected area, it becomes possible to acquire images at the same position regardless of the imaging operator's skill level.

[0208] [Image verification process] Figure 15 is a flowchart showing the image verification process in this embodiment. This process verifies whether an image suitable for the examination purpose has been generated. This is a detailed explanation of the process in step S620 described above. The explanation will be given using the case where the examination purpose is teeth and the area to be photographed is the upper jaw as an example. Processes similar to those in Figure 7 are numbered as in Figure 7 and the explanation is omitted.

[0209] In step S1501, the CPU 201 determines whether the focus position at the time of imaging is near the coordinates of the outline of the shooting assist figure in the image. The focus position in the image is assumed to be stored as coordinate information in the header information attached to the image data. If the focus position is within a predetermined range from the coordinates of the outline of the image, the CPU 201 determines that the focus position is near the coordinates of the outline and proceeds to step S703. If it is determined that the focus position is not near the coordinates of the outline, the CPU 201 proceeds to step S1502.

[0210] In step S1502, the CPU 201 notifies the photographer that the focus position is off from the position of the tooth, and displays a message on the display device 204 instructing them to take the picture so that the affected area is positioned along the shooting assist figure.

[0211] In step S1503, the CPU201 determines whether the area where a predetermined color is distributed is located near the coordinates of the contour line of the image assist figure. The predetermined color is white for teeth, and silver or gold for prosthetics and orthodontic appliances. If it is determined that the area is located near the coordinates of the contour line, the process proceeds to step S714. If it is determined that the area is not located near the contour line, the process proceeds to step S1504.

[0212] In step S1504, the CPU 201 notifies the photographer that the tooth has not been photographed along the contour line, and displays a message on the display device 204 instructing them to take the photograph so that the affected area is positioned along the photographic assist figure.

[0213] Figure 16 is a flowchart showing the inference execution process in the image processing device in this embodiment. In step S634 of Figure 6, the imaging device 101 transmits the results, such as evaluation values, or image data to the image processing device 106. Subsequently, the image processing device 106 executes the inference execution process shown in Figure 16.

[0214] In step S634, the CPU 201 transmits the evaluation value selected in step S627 and the evaluation value calculated in step S632 to the image processing device 106 along with the image ID and patient management number. If there is no evaluation value, the CPU 201 transmits the unsent image data along with the patient management number to the image processing device 106.

[0215] In step S1601, the CPU 301 receives the evaluation value, image ID, patient management number, or image data and patient management number.

[0216] In step S1602, the CPU 301 determines whether the image has been evaluated for the purpose of inspection. Whether or not it has been evaluated is determined by whether or not it has an evaluation value received in S1601. If it is determined that it has not been evaluated, the process proceeds to step S1603. If it is determined that it has been evaluated, the process proceeds to step S1607.

[0217] In step S1603, the CPU 301 performs inference in accordance with the examination purpose associated with the patient management number. The inference is the same as in step S418. If inference is performed on multiple images and different inference results are output for the same area, the inference result with a higher probability value may be adopted, or the result representing a worse symptom may be adopted.

[0218] In step S1604, the CPU 301 displays the inference results according to the examination objective. If the examination objective is teeth, the inference results such as healthy, caries, metal prosthesis, white prosthesis, WSD, implant, and presence of orthodontic appliances are displayed in the form of a callout at the position of each tooth in the diagram showing the dental formula. For periodontal disease, the word "periodontal disease" is displayed in the form of a callout at the position of the gums in the diagram showing the dental formula. Although the types of teeth, such as the upper right canine and first molar, are not displayed individually, the symptoms are displayed as callouts for each tooth on the diagram showing the dental formula by internally associating the tooth type with the symptoms in the inference results.

[0219] In step S1605, the CPU 301 transitions to a screen for editing evaluation values. It may transition to a screen that functions similarly to the evaluation value editing screen of the imaging device 101, or it may allow the user to select a callout representing the inference result from step S1604 and modify it using the input device 306. Then, an editing completion button is placed on the screen.

[0220] In step S1606, the CPU 301 determines whether the "Edit Complete" button has been selected. If it determines that the "Edit Complete" button has been selected, the process proceeds to step S1607. If the "Edit Complete" button has not been selected, the process returns to step S1605 and continues to display the screen on which the evaluation value can be edited.

[0221] In step S1607, the CPU 301 selects whether a warning is necessary. Whether a warning is necessary is determined by whether the conditions specified in advance by the user for each examination purpose are met. In addition, if there are other systems or medical departments that should be notified along with the warning, the user shall also set that information. Although this information is used by the image processing device, since it is information specific to each examination purpose, it may be stored attached to the imaging control information shown in Figure 5. The conditions and notification destinations for each examination purpose are stored in the HDD 305, and for this process, they are read into the RAM 303 and the CPU 301 makes a determination. If the conditions specified by the user are met, the CPU 301 determines that a warning is necessary and proceeds to step S1608. If the conditions specified by the user are met, the CPU 301 determines that a warning is not necessary and proceeds to step S1610. Examples of cases where a warning is necessary include when the examination purpose is a pressure ulcer, when the necrotic tissue area is expanding, or when the depth reaches the bone. The purpose of the examination is when the proportion of the affected area relative to the total body surface area is increasing, or when inflammation has worsened beyond a predetermined level. The purpose of the examination is when burns are detected, when an infection is discovered, when teeth are examined, or when periodontal disease is discovered.

[0222] In step S1608, the CPU 301 displays a warning.

[0223] In step S1609, the CPU 301 sends the evaluation value and warning content, associated with the patient management number, to the notification destination set by the user. If the purpose of the examination is pressure ulcers, the warning notification destinations are the care plan management system of the pressure ulcer care planner and the meal service system. In the case of atopic dermatitis, xerosis, psoriasis, and burns, the system is that of the internal medicine department. In the case of teeth, the system is that of the obstetrics and gynecology or internal medicine department.

[0224] In step S1610, the CPU 301 sends the image management system 105 to the default electronic medical record system 104 if image data is available.

[0225] In this embodiment, we have shown an example of displaying the tooth inference results in the form of callouts within the dental formula diagram. However, the output may also be in the form of dental formula codes, with each tooth and its state paired together.

[0226] In this embodiment, an example of displaying the inference results on an image processing device has been described, but it is also possible to communicate with the electronic medical record system 104 and display the results on the electronic medical record display terminal 103. In that case, after step S1603, the results are sent to the electronic medical record system 104, and the processing from step S1605 to step S1609 is provided by the functions of the electronic medical record system.

[0227] As explained above, the imaging device of this embodiment changes whether or not to display the image analysis results based on the imaging control information. Therefore, depending on the settings of the imaging control information, the inference can be displayed on the imaging device or on the image processing device, depending on the inspection purpose.

[0228] As described above, with the imaging device of this embodiment, the imaging assist figure changes according to the purpose and area of ​​examination based on the imaging control information. In the case of a dental examination in this embodiment, when photographing the front view, a horizontal line for the occlusal position and a vertical line for the center position of the teeth may be displayed. This allows for imaging so that the occlusal position of the teeth is aligned with the horizontal line and the center position of the front teeth is aligned with the vertical line. Therefore, it is possible to record stable image data that is easy to compare even if the photographer or the day of shooting is different.

[0229] As described above, the system comprising the imaging device and image processing device of this embodiment switches whether or not a warning is necessary depending on the purpose of the examination, and if a warning is necessary, it notifies the patient. Furthermore, if necessary, it also notifies the medical system of the relevant clinical department. This makes it possible to prevent missed treatment opportunities.

[0230] [Example 3] In this embodiment, we will describe an example of achieving more appropriate imaging for injuries and illnesses with symptoms including rashes. Since injuries and illnesses with symptoms including rashes can have detailed and widespread affected areas, it is desirable to capture not only magnified images to check the details but also overhead images to check the overall picture as diagnostic images. Therefore, we will explain using the example of an examination for atopic dermatitis in which the affected area is photographed at multiple angles and the image processing device diagnoses the image from each angle.

[0231] First, we will explain the imaging control processing when the purpose of the examination is atopic dermatitis, using Figures 17 and 18. We will focus on the aspects unique to the examination of atopic dermatitis, and will omit explanations when they are the same as for other examination purposes.

[0232] Figure 17 is a conceptual diagram showing the correspondence between imaging control information and inspection objectives in this embodiment. In this embodiment, the information shown in Figure 17 is assumed to be stored in the HDD 305 of the image processing device 106, and is used when reading the imaging control information corresponding to the inspection objective received from the imaging device 101 in step S411 in step S412. Information similar to that in Figure 5 is numbered as in Figure 5, and its explanation is omitted.

[0233] 1701 is a column indicating imaging conditions. It holds the imaging conditions specified for the imaging device 101 to acquire images appropriate for examination assistance for each examination purpose. Parameters not specified in the imaging conditions can be changed by the operator on the imaging device 101. Unlike 502, the zoom position is set manually regardless of the whole-body flag in order to take overhead and magnified images.

[0234] Figure 18 is a flowchart showing the imaging control process in this embodiment. Figure 18 describes the processing in the imaging device 101 in detail. For processes similar to those in Figure 6, the numbers from Figure 6 are used, and explanations are omitted except for those specific to the examination of atopic dermatitis.

[0235] In step S605, the CPU 201 transmits the examination objective as atopic dermatitis to the image processing device 106.

[0236] In step S606, the CPU 201 receives imaging control information from the image processing device 106 and stores it in the RAM 203. If the purpose of the examination is atopic dermatitis, the CPU 201 receives the information for the examination of atopic dermatitis in row 514 as imaging control information.

[0237] In step S1801, the CPU201 determines whether the symptoms of the illness or injury selected by the photographer in step S604 or uniquely identified in step S602 include a rash. If a rash is present, proceed to step S1802. If a rash is not present, proceed to step S629.

[0238] In step S1802, the CPU 201 stores the evaluated field of view in the RAM 203.

[0239] The determination of whether the evaluated field of view is an overhead view or a magnified view will be described later using Figure 20.

[0240] In step S1803, the CPU 201 determines whether both the overhead image and the magnified image have been captured. If both have been captured, the process proceeds to step S629. If neither has been captured, the process proceeds to step S1804.

[0241] In step S1804, the CPU 201 determines whether or not an overhead image has already been taken. If an overhead image has already been taken, the process proceeds to step S1805. If an overhead image has not yet been taken, the process proceeds to step S1806.

[0242] In step S1805, the CPU 201 displays "Please take a magnified image" on the display device 204 to notify the photographer that it will now take a magnified image. The process returns to step S616 for the image to be taken.

[0243] In step S1806, the CPU 201 displays "Please take an overhead image" on the display device 204 to notify the photographer that an overhead image should be taken next. The process returns to step S616 for the image to be taken.

[0244] Figure 19 shows a view from the rear of a digital still camera operating as an imaging device according to one embodiment of the present invention. An example of an image of the affected area when the purpose of examination is atopic dermatitis is shown. Components that perform the same function as those in Figures 8 and 10 are numbered with the corresponding numbers from Figures 8 and 10.

[0245] Figure 19(a) is an example of an overhead view.

[0246] Figure 19(b) is an example of an enlarged image.

[0247] 1901 refers to the affected area, and in the case of atopic dermatitis, it refers to the area of ​​the rash.

[0248] 1902 is in the healthy group.

[0249] 1903 is the background area other than the affected area 1901 and the healthy area 1902, such as a wall.

[0250] 1904 is an example of a field of view display and represents the lens's focal length information. It does not need to be displayed.

[0251] Figure 20 is a flowchart for determining whether the evaluated field of view in this embodiment is an overhead view or a magnified view. This process is performed in step S1802.

[0252] In step S2001, it is determined whether the focal length of the lens at the time of shooting is 50mm or less. If the focal length of the lens is 50mm or less, proceed to step S2002. If the focal length of the lens is not 50mm or less, proceed to step S2003.

[0253] In step S2002, the evaluated field of view is determined to be an overhead view.

[0254] In step S2003, the evaluated field of view is determined to be magnified.

[0255] In Figure 20, the focal length threshold for the lens is set to 50 mm, but any other value may be used, and multiple thresholds may be set depending on the injury or illness being examined.

[0256] Alternatively, the evaluation of the field of view may be determined by other methods to determine whether it is an overhead view or a magnified view. For example, image analysis using color information may be performed, and it may be determined that if a background 1903 is present, it is an overhead view, and if a background 1903 is absent, it is a magnified view. Furthermore, the photographer may be given the option to select whether the captured image is an overhead view or a magnified view.

[0257] As explained above, with the imaging device of this embodiment, when the purpose of the examination is for an injury or illness with symptoms including a rash, it is possible to take both an overhead image and a magnified image, thereby preventing the forgetting to take the image.

[0258] As explained above, the imaging device of this embodiment allows for confirmation of whether an appropriate image has been captured according to the inspection purpose, and furthermore, it notifies the photographer of any problems during shooting, making it possible to record stable image data regardless of the photographer's skill level.

[0259] [Example 4] In this embodiment, we will explain, using Figures 21 to 27, an example of reducing the effort required to select and input a body part by estimating from a pre-image of the imaging device before shooting, and further modifying the shooting conditions to be more favorable using the selection results. This embodiment basically performs the same processing as in Embodiment 1, with only some processing differing from Embodiment 1. In this embodiment, the processing that differs from Embodiment 1 will be explained using Figure 21. Figure 21 is a diagram that explains the processing flow of this embodiment by extracting the processing from inspection purpose transmission S411 to inference execution S418 from Figure 4. Of the processing in Figure 21, processing S411 to S412 and S414 to S418 are the same as in Figure 4, so the explanation will be omitted. Note that the processing in Figure 21 is realized by each device and system executing the processing based on the control program of each device and system, as in Figure 4.

[0260] The imaging device, having read the imaging control information, performs pre-image transmission S2101. Here, the timing of transmitting the pre-image can be the same as when the photographer takes an image of the affected area; the photographer may press the shutter themselves to take a pre-image and send it to the image processing device. Alternatively, if the imaging device is running a live view function that displays a live view image (live image) during the waiting period before taking an image of the affected area in S416, the image taken for live view may be automatically sent to the image processing device as a pre-image.

[0261] Figure 22 shows an example of the display on the imaging device when the photographer presses the shutter to take a picture and transmits a pre-image, and Figure 23 shows an example of the display on the imaging device when the live view function is executed and a pre-image is automatically transmitted while waiting to take a picture of the affected area.

[0262] In S2101, the image processing device receives a pre-image, and in S2102, it performs part estimation. In part estimation, the human body is extracted using brightness information, color information, hue information, gradient information, etc., from the pre-image. Subsequently, part estimation is performed using gradient information and the distribution of color or intensity information in regions. Alternatively, part estimation may be performed by matching using shape feature information of the grayscale distribution or brightness gradient, or pixels with a brightness gradient above a certain level may be extracted as edge information, and part estimation may be performed using their position, centroid, tilt, etc. Furthermore, training data for each part of the human body may be created using machine learning with a neural network such as deep learning, and the parts included in the received pre-image may be estimated. In this embodiment, part estimation is performed in the image processing device, but part estimation may also be performed in the imaging device. Then, in S2103, the image processing device transmits the part estimation result, which is the part information estimated in the part estimation in S2102, and the imaging control information read in S412 to the imaging device. Note that the imaging control information and part estimation result may be transmitted to the imaging device simultaneously as shown in Figure 21, or they may be transmitted separately. For example, the image processing device, upon receiving the inspection objective in S411, transmits imaging control information corresponding to the inspection objective to the imaging device. Then, upon receiving a preliminary image from the imaging device in S2101, it may perform site estimation in S2102 and transmit the site estimation result to the imaging device. Alternatively, the image processing device may transmit a preliminary image from the imaging device to the image processing device before the inspection objective is set.

[0263] In S2102, the image processing unit performs part estimation, and in S2103, the imaging unit acquires the part estimation result (part acquisition). As shown in Figure 22, the part estimated by the image processing unit can be displayed on the imaging unit along with the pre-image. In the composition shown in Figure 22, "head" is estimated as the part, and "head" is displayed as the estimated part along with the pre-image. As shown in Figure 23, when a live view image is used as the pre-image, the imaging unit transmits the images taken as live view images to the image processing unit as pre-images at predetermined intervals in S2101. The image processing unit then performs part estimation for each pre-image received at predetermined intervals in S2102, and transmits the part information of the estimated part as the part estimation result to the imaging unit in S2103. In other words, the process from S2101 to S2103 is repeated until a part is selected by the photographer in S414 and S415, and the imaging unit displays the part estimated by the image processing unit along with the current live view image. The body parts displayed on the imaging device are estimated based on images transmitted as pre-images, not the current live image, so there may be a slight delay in the display of the body parts. Furthermore, when the composition on the imaging device changes, the live image changes, and the body parts estimated in S2102 also change according to the changed live image. For example, in Figure 23, the body part estimation results are displayed as follows: Figure 23(a) whole body → Figure 23(b) waist, upper body → Figure 23(c) upper body → Figure 23(d) head. Note that in Figures 22 and 23, the estimated body parts are displayed along with the image, but it is also possible to display not only the estimated body parts but also the patient information received by the imaging device in S407 and the examination purpose selected in S410.

[0264] When performing part estimation in S2102, it is possible to estimate not only one part but also multiple parts. Depending on the captured image data, the area of ​​the human body captured may be large, and it is possible that multiple parts may be included in the estimation result. In such cases, all of the estimated parts may be included in the part estimation result. If multiple parts are included in the part estimation result, the part estimation result may be generated and transmitted to the imaging device in a way that makes it possible to identify the part with the highest estimation accuracy, for example, by listing them in order of accuracy or by flagging the part with high accuracy.

[0265] Furthermore, in addition to pre-images, site estimation may also be performed using the examination purpose received in S411 and patient information received from the electronic medical record system in S404. For example, if the examination purpose received in S411 indicates that the imaging site within the last month was "head," the head may be selected as the estimation result, or the system may be designed to make it easier to select past imaging sites in cases of past medical history. If the imaging site in a past medical history differs from the estimation result from the pre-images, both the imaging site in the past medical history and the estimation result from the pre-images may be sent to the imaging device as the site estimation result.

[0266] As shown in Figure 22, if the photographer instructs the camera to take a pre-image and pre-images are taken, the image processing device receives the area estimation result in S2103, and then automatically transmits the live view image as the pre-image, as shown in Figure 23. In this case, in response to the photographer's instruction to select an area, the imaging device displays the area candidates in S414.

[0267] In S414, the candidate body parts are displayed as shown in Figure 25, similar to Example 1. In this embodiment, the body part estimation result received from the image processing device in S2103 is displayed as a candidate body part. Similar to Example 1, if there is only one body part in the body part estimation result received in S2103, the display of the candidate body part (body part selection screen) in S414 may be omitted, and the imaging device (CPU201) may automatically select and set the body part to be photographed in S415 from the body part received as the body part estimation result in S2103. Also, if the photographer performs an operation for body part selection without performing the pre-image transmission in S2101, the candidate body parts may be displayed as in Example 1 without using the body part estimation result. Furthermore, when displaying the candidate body parts in S414, or when displaying the estimated body parts as shown in Figures 22 and 23, body parts corresponding to the inspection purpose and body parts that do not correspond to the purpose may be displayed separately. They are displayed separately so that it is clear that they are body parts that are not included in the candidate body parts for the inspection purpose selected in S411. For example. If the area included in the area estimation result received from the image processing device in S2103 is not a candidate area to be photographed for the examination purpose selected in S411, that area may be displayed in gray or with a different color or display format. Alternatively, as shown in Figures 22 and 23, when displaying the area estimation result together with the image, areas that do not correspond to the examination purpose may be displayed in gray, and in the display of area candidates in S414, areas that do not correspond to the examination purpose may be hidden as they cannot be selected. Furthermore, if the area estimation result includes information on the accuracy of the estimation, areas with high estimation accuracy and areas with low estimation accuracy may be displayed separately in the display of area estimation results in Figures 22 and 23 and the display of area candidates (area selection screen) in Figures 24 and 25. For example, areas with high estimation accuracy may be displayed in bold or with a different color.

[0268] In this way, by transmitting a pre-image for site estimation to the image processing device before site selection (S414, S415) and using only the site estimated by the image processing device as a site candidate, site selection can be facilitated. If the pre-image is automatically transmitted using the live view image, the photographer can narrow down the site candidates to the site of the estimation result without performing a special operation. Furthermore, when there is only one site in the site estimation result, the imaging device may automatically select the site of the estimation result in S415 without displaying the site candidates in S414. By doing so, the need for the photographer to select a site is eliminated. Also, by displaying the site of the estimation result together with the image, the photographer can confirm the estimated site. If the site of the estimation result is different from the site to be photographed, the photographer can also change the composition so that the site of the estimation result is changed. When the photographed site is different from the site of the estimation result, it is possible that the composition is not correct, so the photographer can also confirm whether the photograph is taken with the correct composition.

[0269] Note that it is also conceivable to use the site information of the result of site estimation only for associating and recording / transmitting with the affected part image photographed in S416 without using site estimation for the selection of site information or the confirmation of composition. In that case, the pre-image may not be transmitted, and site estimation may be performed using the affected part image photographed in S416 and transmitted to the image processing device in S417.

[0270] When it is desired to take a more suitable affected part image, since the size, depth change amount, appropriate composition, etc. differ depending on the site to be photographed, the photographing conditions and confirmation conditions may be different.

[0271] Therefore, a modified example of Example 4 will be described with reference to FIGS. 26 and 27.

[0272] In Figure 27, the S2104 imaging condition update process is added to the flow shown in Figure 21 described above. In the S2104 imaging condition update process, the image processing device updates the imaging conditions to be sent to the image processing device so that they correspond to the estimated area, according to the area estimation result in S2102. Then, in S2103, the imaging control information updated in S2104 and the area estimation result in S2102 are sent. Note that such imaging condition updates may be performed not only according to the area estimation result in S2102, but also according to the area selection result in S415. In that case, the area selected in S415 is notified (sent) to the image processing device, the image processing device updates the imaging condition information, and the updated imaging condition information is sent to the imaging device, and the imaging device sets its imaging conditions based on the received imaging condition information. Alternatively, imaging condition information corresponding to multiple candidate areas corresponding to the examination purpose may be sent from the image processing device to the imaging device in advance, and the imaging device may set its imaging conditions from the imaging condition information corresponding to the area selected by the imaging device.

[0273] Figure 26 shows examples of imaging and confirmation conditions for each candidate body part when the purpose of examination is pressure ulcers. For example, the head, hands, and arms are not very large, so the subject distance does not need to be very far. When the subject distance is close, it is possible to focus even with a wide aperture, so settings such as wide aperture, low ISO sensitivity, and fast shutter speed are used to suppress noise and motion blur. On the other hand, the back, waist, and legs are large in size and have large depth of field changes, so it is necessary to have a large subject distance and depth of field, so the ISO value is set high and the shutter speed is slow to amplify the brightness of the captured image.

[0274] The above examples illustrate how to update imaging conditions based on the purpose of the examination and the estimated location of the affected area. However, further adjustments may be made based on the characteristics of the patient being examined. For example, if the patient has dark skin and you are photographing an affected area of ​​a skin disease, increase the ISO sensitivity within the recommended range. If the patient is an infant and the area being photographed is the hands or feet, and motion blur is anticipated, increase the shutter speed within the recommended range.

[0275] [Other Embodiments] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Some of the above embodiments may be combined as appropriate.

[0276] Furthermore, the invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. Therefore, the program code itself supplied to and installed in a computer to implement the functional processing of the present invention also implements the present invention. In other words, the computer program itself for implementing the functional processing of the present invention is also included in the present invention.

Claims

1. An imaging device having an imaging unit for capturing images for inspection and a communication unit for communicating with a predetermined system, comprising a selection means for selecting the purpose of inspection, An acquisition means for acquiring imaging control information corresponding to the inspection purpose selected by the selection means from the system via the communication unit, A control means that sets at least one of the following based on the imaging control information acquired according to the inspection purpose: the set value of the imaging conditions, the display content to be displayed on the display means together with the image captured by the imaging unit, and the item of information to be associated with the image captured by the imaging unit. A body part selection means that selects a body part to be photographed based on body part information included in the aforementioned imaging control information, An imaging device characterized by having the following features.

2. The imaging device according to claim 1, characterized in that the imaging device is a portable imaging device.

3. The control means sets the set value of the imaging condition based on the imaging control information acquired according to the inspection purpose, The imaging device according to claim 1, characterized in that the imaging conditions include at least one of focus position, zoom position, ISO sensitivity, exposure, shutter speed, aperture, white balance, color adjustment, and whether or not a flash is fired.

4. The imaging apparatus according to claim 1, characterized in that the control means determines the type of shooting assist figure to be displayed on the display means together with the image captured by the imaging unit, based on the imaging control information received according to the inspection purpose, and controls the display means of the imaging apparatus to display the determined shooting assist figure.

5. The types of imaging assist figures include at least one of the following: a figure showing the affected area during the previous examination, a figure showing the outline of the area to be examined, a straight line for the occlusal position of the teeth, and an outline of the dental arch. The imaging apparatus according to feature 4.

6. The imaging apparatus according to claim 1, characterized in that the control means sets inspection evaluation items associated with the image captured by the imaging unit based on the imaging control information acquired according to the inspection purpose.

7. The imaging apparatus according to claim 6, characterized in that the control means sets the inspection evaluation item from among a plurality of inspection evaluation items, including at least one of DESIGN-R, SRRC, EASI, PASI, and TBSA, based on the imaging control information acquired according to the inspection purpose.

8. The control means is If the purpose of the aforementioned examination is pressure ulcers, then DESIGN-R, If the purpose of the aforementioned examination is sebum deficiency, then SRRC, If the purpose of the aforementioned examination is atopic dermatitis, then EASI, If the purpose of the aforementioned examination is psoriasis, then PASI, If the purpose of the inspection is burns, then TBSA and burn depth, The above is set as an inspection and evaluation item. The imaging apparatus according to feature 7.

9. The imaging apparatus according to claim 1, characterized in that the selection means selects an inspection objective from among a plurality of inspection objectives obtained from the system via the communication unit, in accordance with user operation.

10. The imaging apparatus according to claim 9, characterized in that the selection means automatically selects an inspection purpose when there is only one inspection purpose obtained from the system.

11. The imaging device according to claim 1, wherein the imaging control information includes setting values ​​for imaging conditions, and further includes at least one of the following: area information relating to candidate areas to be imaged, information relating to evaluation items, and information relating to areas that must be imaged.

12. The imaging apparatus according to claim 1, further comprising a patient information acquisition means for transmitting patient images captured by the imaging unit to the system via the communication unit, and receiving and acquiring patient information corresponding to the patient in the patient image from the system via the communication unit.

13. The imaging device according to claim 12, wherein the patient information acquisition means transmits an image of a code relating to a patient to the system as the patient image, and the system receives patient information corresponding to the code obtained by analyzing the patient image from the system.

14. The imaging apparatus according to claim 12, characterized in that the selection means selects an examination purpose from among the examination purposes corresponding to the patient information.

15. The imaging apparatus according to claim 14, characterized in that, if there is no examination purpose corresponding to the patient information, the selection means selects an examination purpose from among a plurality of examination purposes unrelated to the patient information.

16. The imaging apparatus according to any one of claims 1 to 8, characterized in that the control means makes it impossible to change the setting values ​​of imaging conditions that have been set based on the imaging control information acquired according to the inspection purpose, according to user operation, and makes it possible to change the setting values ​​of imaging conditions that have not been set based on the imaging control information according to user operation.

17. The system includes an image analysis result receiving means that transmits an image captured by the imaging unit to the system via the communication unit, and receives the image analysis results of the said image from the system via the communication unit. The imaging apparatus according to claim 1, characterized in that the control means controls the display of information based on the image analysis results received by the image analysis result receiving means together with the image.

18. The control means is If the purpose of the examination is pressure ulcers, at least one of the following will be displayed: area of ​​the affected area, pressure ulcer portion, necrotic portion, inflamed / infected portion, granulation tissue portion, and depth information. If the purpose of the examination is to check for sebum deficiency, at least one of the following shall be displayed: area of ​​the affected area, area of ​​sebum deficiency, area of ​​the affected area, skin roughness, skin redness, and areas of skin cracks. If the purpose of the examination is atopic dermatitis, at least one of the following should be indicated: area of ​​the affected area, erythematous area, moist / papulous area, scratched area, lichenified area. If the purpose of the examination is psoriasis, at least one of the following should be indicated: area of ​​the affected area, erythematous area, moist area, or desquamating area. If the purpose of the examination is burns, at least one of the following shall be displayed: the area of ​​the affected area, the burned area, and the burn depth. If the purpose of the examination is urticaria, at least one of the following shall be displayed: the area of ​​the affected area, the number of wheals, or the number of urticaria. If the purpose of the examination is for injuries or diseases of the oral cavity, at least one of the following will be indicated: type of tooth, presence or absence of tooth, presence or absence of prosthesis, presence or absence of caries, carious area, periodontal disease area. The imaging apparatus according to claim 17, characterized in that it controls in such a manner.

19. The imaging apparatus according to claim 1, characterized in that the part selection means selects the part to be photographed from among candidate parts determined based on the imaging control information.

20. The imaging apparatus according to claim 19, characterized in that the granularity of the candidate sites that can be selected by the site selection means is changed depending on whether or not the injury or disease that is the purpose of the examination is to be evaluated throughout the whole patient.

21. When evaluating the injury or illness for which the examination is intended, the candidate site is a part that covers a wide area of ​​the body. The imaging device according to claim 20, characterized in that, if the injury or illness for which the examination is to be performed is not evaluated throughout the patient's entire body, a narrow area of ​​the body is designated as the candidate site.

22. The imaging apparatus according to claim 21, wherein the control means controls the display means to display a part selection screen for selecting a part to be photographed from among the candidate parts.

23. The imaging apparatus according to claim 22, characterized in that the control means controls the display of the area selection screen so that candidate areas that must be photographed can be identified based on the information regarding areas that must be photographed included in the imaging control information.

24. The aforementioned imaging control information includes information about the area that must be photographed, If the purpose of the examination is any of the following: xerosis, atopic dermatitis, psoriasis, burns, or urticaria, then the imaging control information includes information that the head / neck, trunk, upper limbs, and lower limbs are essential areas for imaging. If the purpose of the examination is for injuries or diseases in the oral cavity, the imaging control information includes information that the upper jaw and lower jaw are essential areas for imaging. The imaging apparatus according to feature 1.

25. The imaging device according to claim 19, characterized in that the granularity of the candidate sites is changed depending on whether or not the injury or illness to be examined includes a rash.

26. The imaging apparatus according to claim 1, characterized in that the control means controls the imaging unit to capture images of multiple different angles of view when the symptoms of the injury or illness to be examined include a rash.

27. The imaging device according to claim 26, characterized in that the control means prompts the imaging device to capture images of uncaptured angles when the symptoms of the injury or illness to be examined include a rash and images of the plurality of different angles of view have not been captured.

28. The imaging device according to claim 26, characterized in that the control means controls the imaging device to capture an overhead image and a magnified image when the symptoms of the injury or illness to be examined include a rash.

29. The imaging apparatus according to claim 28, characterized in that the control means determines whether or not an overhead image and a magnified image have been captured according to the focal length information of the lens of the captured image.

30. The imaging device according to claim 25, characterized in that the injury or illness including the rash is any of xerosis, atopic dermatitis, psoriasis, or urticaria.

31. It has a body part selection means for selecting the body part to be photographed, The imaging apparatus according to claim 1, characterized in that the selection means selects a part estimated based on the image captured by the imaging unit as the part to be photographed.

32. The selection means selects the area to be photographed based on the image taken by the imaging unit during the shooting standby period, The control means transmits to the system an image of the affected area captured by the imaging unit in response to a shooting instruction from the user, and information of the area selected by the area selection means, in association with each other. The imaging apparatus according to feature 31.

33. The system includes a region acquisition means for acquiring a region estimated based on an image captured by the imaging unit, The part acquisition means transmits the image captured by the imaging unit during standby to the system, and acquires the part estimated from the system based on the image. The imaging apparatus according to feature 31.

34. The imaging apparatus according to claim 33, characterized in that the part selection means allows the user to select a part from among a plurality of parts acquired by the part acquisition means via a part selection screen.

35. The selection means selects the area to be photographed based on the image taken by the imaging unit during the shooting standby period, The control means controls the display unit to display the selected area together with the live image captured by the imaging unit. The imaging apparatus according to feature 31.

36. The imaging apparatus according to 32, wherein the control means sets the setting value of the imaging condition based on the imaging control information acquired according to the inspection purpose and the part selected by the selection means.

37. A control method for an imaging device having an imaging unit that captures images for inspection and a communication unit that communicates with a predetermined system, Selection process for selecting the purpose of the inspection and An acquisition step of acquiring imaging control information corresponding to the selected inspection purpose from the system via the communication unit; A control step of setting at least one of the following based on the imaging control information acquired according to the inspection purpose: setting the setting value of the imaging conditions, display content to be displayed on the display means together with the image captured by the imaging unit, and items of information to be associated with the image captured by the imaging unit; A region selection step in which a region to be photographed is selected based on region information included in the aforementioned imaging control information, A control method for an imaging device, characterized by having the following features.

38. A program for causing a computer to function as one of the means described in any one of claims 1 to 36.

Images