Image processing device, image processing method, image processing program, image display device, and imaging device

Abstract

[Program] To provide an image of a similar hue by performing prescribed color correction on an image showing a person, even when the illumination environment at the time of imaging is different. [Solution] An image processing device according to the present invention is characterized by comprising: an image acquisition means for acquiring image data of an image showing a person; a color correction means for correcting a hue such that the value indicating a hue of an image region occupied by a skin portion of the person in the image is the value indicating a skin color hue of the person; and an output means for outputting the corrected image to a display unit.

Description

Image processing device, image processing method, image processing program, image display device, and imaging device

[0001] The present invention relates to an image processing device, an image processing method, an image processing program, an image display device, and an imaging device.

[0002] In recent years, medical treatment using information and communication devices (known as "online medical treatment" or "remote medical treatment") is expected to play an important role in improving the quality of medical treatment, increasing convenience for patients, and correcting regional disparities in medical care in remote islands and rural areas, etc. During medical treatment, a doctor will conduct a visual examination to assess the patient's physical condition.

[0003] For example, when a doctor examines a patient for jaundice symptoms via online medical consultation, the doctor will examine the patient by looking at images of the patient's eyes and skin, which are areas where jaundice symptoms are likely to appear. The doctor visually examines the patient's image displayed on the medical institution's display, but the hue and brightness vary depending on the performance of the camera used to capture the patient, the amount of external light at the time of capture, and how the light hits the patient. In other words, the lighting environment at the time of capture has an impact. Therefore, in online medical consultations, it is necessary to correct the color of the image displayed on the medical institution's display so that the doctor can accurately perform the visual examination.

[0004] Patent Literature 1 discloses a remote medical examination that uses color information such as the color of a patient's skin and tongue. For example, when photographing a patient, a color chart is photographed at the same time, and when the image is displayed, a color correction unit corrects the color of the image data so that the color of the color chart displayed on the display unit approaches the color measured under the light source at the location where the display unit is placed.

[0005] Japanese Patent Application Laid-Open No. 2022-137109

[0006] However, it can be troublesome to have to take a photo using a color chart that has been previously agreed upon.

[0007] Therefore, in order to solve the above-mentioned problems, the present invention aims to provide images with similar hues even when the lighting environment at the time of shooting is different by performing a predetermined color correction on images containing people.

[0008] In order to solve such problems, the first image processing device of the present invention is characterized by comprising: (1) an image acquisition means for acquiring image data of an image showing a human; (2) a color correction means for correcting the hue of the image area occupied by the human skin in the image so that the value indicating the hue becomes a value indicating the human skin hue; and (3) an output means for outputting the corrected image to a display unit.

[0009] The second image processing method of the present invention is characterized in that (1) an image acquisition means acquires image data of an image containing a human, (2) a color correction means corrects the hue of the image area occupied by the human skin so that the value indicating the hue becomes a value indicating the human skin hue, and (3) an output means outputs the corrected image to a display unit.

[0010] The third image processing program of the present invention is characterized in that it causes a computer to function as (1) an image acquisition means for acquiring image data of an image showing a human being, (2) a color correction means for correcting the hue of an image area occupied by the human skin in the image so that the value indicating the hue becomes a value indicating the human skin hue, and (3) an output means for outputting the corrected image to a display unit.

[0011] The fourth image display device of the present invention is an image display device that displays an image containing a human on a display unit, and is characterized by comprising: (1) an image acquisition means for acquiring image data of the image containing a human; (2) a color correction means for correcting the hue of the image area occupied by the human's skin so that the value indicating the hue becomes a value indicating the human's skin hue; and (3) an output means for outputting the corrected image to the display unit.

[0012] The fifth imaging device of the present invention is an imaging device characterized by comprising: (1) an imaging unit; (2) image acquisition means for acquiring image data of an image captured by the imaging unit, the image data including a human; (3) color correction means for correcting the hue of an image area occupied by the human skin in the image so that the value indicating the hue becomes a value indicating the human skin hue; and (4) output means for outputting the corrected image.

[0013] According to the present invention, by performing predetermined color correction on an image in which a person appears, it is possible to provide an image with a similar hue even if the lighting environment at the time of shooting is different.

[0014] 1 is an internal configuration diagram showing the internal configuration of a medical support server 3 according to an embodiment. FIG. 2 is a configuration diagram showing the overall configuration of a remote medical support system 9 according to an embodiment. FIG. 3 is an explanatory diagram explaining a biconical HSL color space. FIG. 4 is an explanatory diagram explaining the hue values ​​of facial images acquired for each program in the skin color hue value analysis process of an embodiment. FIG. 5 is a diagram showing the analysis results of skin color hue values ​​by an experimental method (part 1) according to an embodiment. FIG. 6 is an explanatory diagram explaining a method for measuring the hue value of facial skin color for each image in a method (part 2) of an embodiment. FIG. 7 is a diagram showing the analysis results of skin color hue values ​​by an experimental method (part 1) according to an embodiment. FIG. 8 is a sequence diagram showing the image processing operation for color correction of a patient image in the remote medical support system according to an embodiment. FIG. 9 is a flowchart showing the color correction process according to an embodiment. FIG. 10 is a diagram showing patient images and hue histograms before and after color correction (part 1). FIG. 11 is a diagram showing patient images and hue histograms before and after color correction (part 2). FIG. 12 is a diagram showing patient images and hue histograms before and after color correction (part 3). FIG. 13 is a diagram showing patient images and hue histograms before and after color correction (part 4).

[0015] (A) Embodiments Hereinafter, embodiments of an image processing device, an image processing method, an image processing program, an image display device, and an imaging device according to the present invention will be described in detail with reference to the drawings.

[0016] In this embodiment, the present invention is applied to a remote medical care support system. However, the present invention is not limited to the remote medical care support system. The present invention can be widely applied to cases where color correction is performed on a display device when an image is displayed on the display device. The present invention can also be applied to cases where color correction is performed on image information output by an imaging device.

[0017] For example, in the notation "A to B" indicating a numerical range, value A indicates the lower limit and value B indicates the upper limit, and both value A and value B are included in the numerical range.

[0018] (A-1) Configuration of the Embodiment FIG. 2 is a diagram showing the overall configuration of a remote medical care support system 9 according to the embodiment.

[0019] In FIG. 2, the remote medical assistance system 9 includes a medical institution terminal 1, a patient terminal 2, and a medical assistance server 3, all of which are connectable to a network NT.

[0020] In the remote medical care support system 9, the patient terminal 2 transmits a patient image showing the part of the patient that is the target of medical treatment to the medical institution terminal 1. The patient image viewed by the doctor is displayed on the display unit 12 of the medical institution terminal 1, and the doctor visually examines the patient while looking at the displayed patient image.

[0021] Here, the color of the image displayed on the display unit 12 of the medical institution terminal 1 varies depending on the performance of the camera that captured the patient image, the amount of external light that hits the patient due to the lighting environment at the time of capture, etc., so the color (especially the hue) of the displayed image may not appear properly, and the doctor may not be able to make a proper diagnosis.

[0022] Therefore, when a patient image is displayed on the display unit 12 of the medical institution terminal 1, the medical assistance server 3 corrects the color of the displayed image in response to a request from the medical institution terminal 1, and transmits the corrected patient image to the medical institution terminal 1. As a result, the color-corrected patient image is displayed on the display unit 12 of the medical institution terminal 1, allowing the doctor to perform a correct visual examination.

[0023] In this embodiment, the medical assistance server 3 and the medical institution terminal 1 can exchange information with each other, and the medical assistance server 3 performs color correction on the patient image in response to a request from the medical institution terminal 1.

[0024] For example, the functions of the medical assistance server 3 are stored on a cloud server, and are provided to the medical institution terminal 1. In this case, the medical institution terminal 1 uses a browser or dedicated application to perform input and display only, and data management and various processing are performed by the medical assistance server 3 as a cloud server.

[0025] In another embodiment, the functions of the medical assistance server 3 may be implemented as application software (e.g., an image processing program, etc.). In this case, installing the application software on the medical institution terminal 1 may enable various functions, such as color correction, to be performed on the medical institution terminal 1.

[0026] In this embodiment, an example is given in which the medical institution terminal 1 requests color correction instructions from the medical support server 3, but this is not limited to this. The medical institution terminal 1, which has an image processing program installed, may automatically perform color correction on the patient image and display the patient image after color correction.

[0027] <Patient Terminal 2> The patient terminal 2 is a communication terminal on the patient side and is connectable to the network NT. The patient terminal 2 is operated by the patient himself / herself, the patient's caregiver, etc. The patient terminal 2 can be, for example, a smartphone, a tablet terminal, a laptop personal computer, a dedicated terminal, a mobile terminal, a wearable terminal, etc.

[0028] For example, the patient terminal 2 has a control unit 20 that manages various functions performed on the patient terminal 2, an input unit 21 that accepts information through user operation such as the input function of a keyboard, mouse, or touch panel display, a display unit 22 that displays input information such as an LCD display and the results of a doctor's examination, an imaging unit 23 represented by a camera, a memory unit 24 that stores processing programs that run on the patient terminal 2, data necessary for processing, and captured images, etc., and a communication unit 25 that exchanges information with the network NT.

[0029] In this embodiment, the patient terminal 2 is illustrated as being equipped with an imaging unit 23 that captures images of the patient, but it is also possible to capture patient images with a separate camera and transmit the patient images to the medical institution terminal 1 during medical treatment.

[0030] <Medical Institution Terminal 1> The medical institution terminal 1 is operated by a medical professional such as a doctor, and is a communication terminal connectable to the network NT. The medical institution terminal 1 can be any of a variety of terminals that display patient images for visual examination by a doctor, and can be, for example, a smartphone, a tablet terminal, a laptop personal computer, a dedicated terminal, a mobile terminal, or the like.

[0031] For example, the medical institution terminal 1 has a control unit 10 that controls various functions performed by the medical institution terminal 1, an input unit 11 that accepts information through user operation such as the input function of a keyboard, mouse, or touch panel display, a display unit 12 that displays input information such as an LCD display and patient images, a memory unit 13 that stores processing programs that run on the medical institution terminal 1, data necessary for processing, and patient images (including images after color correction), etc., and a communication unit 14 that exchanges information with the network NT.

[0032] In this embodiment, the medical institution terminal 1 receives a patient image from the patient terminal 2, performs color correction on the patient image, and then the doctor examines the patient. However, this is not limited to this, and color correction may also be performed on a patient image that has been saved in advance.

[0033] <Medical Support Server> The medical support server 3 can be connected to the network NT and supports medical treatment by doctors.

[0034] FIG. 1 is a diagram showing the internal configuration of a medical support server 3 according to an embodiment.

[0035] In FIG. 1 , the medical assistance server 3 includes a control unit 30 , a storage unit 32 , and a communication unit 33 .

[0036] The control unit 30 controls the functions of the medical assistance server 3. The control unit 30 includes a CPU, ROM, RAM, EEPROM, an input / output interface, etc., and the CPU executes processing programs (e.g., an image processing program, a medical assistance program, etc.) to realize various functions of the medical assistance server 3. The control unit 30 has an image processing unit 50 as one of the functions it executes.

[0037] The image processing unit 50 performs predetermined image processing on the image to be examined, and includes an image acquisition unit 51, a color correction unit 52, and an output unit 53.

[0038] The image acquisition unit 51 acquires an image of a patient to be treated from the medical institution terminal 1 or the patient terminal 2 , and provides the acquired patient image to the color correction unit 52 .

[0039] The color correction unit 52 performs color correction on the patient image using a hue value that indicates human skin color.

[0040] Here, the inventors of the present application have conducted extensive research and have found that the hue values ​​of human skin color (hereinafter referred to as "skin color hue values") are approximately the same regardless of race or gender, as will be described later. For example, the skin colors of white and black people appear different, but this is thought to be influenced by factors such as the amount of melanin contained in the skin and the amount of hemoglobin flowing through the blood vessels beneath the skin. For example, a high amount of melanin results in dark skin color, while a low amount results in light skin color. Furthermore, for example, an increase or decrease in hemoglobin can also affect the color tone. Thus, while the brightness (lightness) and color tone (hue) of skin color can vary, the hue values ​​of skin color can be approximately the same regardless of race or gender.

[0041] Therefore, the color correction unit 52 can create an image with a more appropriate hue by correcting the hue of the patient image using the skin color hue value as a reference value, even if there are differences in camera performance, display performance, etc.

[0042] The output unit 53 outputs the patient image corrected by the color correction unit 52 .

[0043] The storage unit 32 stores processing programs (for example, image processing programs, medical assistance programs, etc.), data required for processing, and the like.

[0044] The communication unit 33 communicates with the medical institution terminal 1 and the like via the network NT.

[0045] (A-2) Analysis of Skin Color Hue Values ​​First, the applicant of the present application has analyzed and derived the characteristic that the hue values ​​of human skin colors are approximately the same regardless of race or sex.

[0046] Here, we will explain the results of extracting a human face image (image area containing a face) from the image (or video) being analyzed, measuring the hue value of that face image, and finding that the skin color hue values ​​are almost the same.

[0047] When measuring hue values, differences in the lighting environment during shooting result in differences in the brightness of the human face being analyzed. Therefore, in the following, we will perform two analytical methods: one in which skin-color hue values ​​are derived using a television program as the image to be analyzed (part 1), and another in which skin-color hue values ​​are derived using an image shot under the same lighting environment (part 2). We will then explain the analytical results for each experimental method.

[0048] The analysis of skin color hue values ​​was performed by analyzing the color space of the image to be analyzed using image processing software on a computer.

[0049] In this embodiment, the biconical HSL color space, which is said to be close to human vision, is used as the color system for the color space of the image, but this is not limiting and other color systems may also be used.

[0050] As shown in FIG. 3A, the biconical HSL color space is a biconical color space consisting of three components: hue, saturation, and luminance.

[0051] As shown in FIG. 3B, "hue" can be expressed as an angle ranging from 0° to 360°, where 0° represents red, 60° represents yellow, 120° represents green, 180° represents cyan, 240° represents blue, and 300° represents magenta.

[0052] "Brightness" can be expressed as 256 numbers from "0 to 255" representing the height distance on the vertical axis shown in Figure 3 (A), with the value "0" representing pure black, the value "128" representing intermediate brightness, and the value "255" representing pure white.

[0053] "Saturation" can be expressed as 256 numbers from 0 to 255, representing the radial distance from the vertical axis passing through the center of the plane, with a value of "0" representing no color (black and white) and a value of "255" representing maximum vividness (pure color).

[0054] (A-2-1) Analysis Method (Part 1) <Analysis Method> The images to be analyzed (videos to be analyzed) are television programs. Using multiple television programs, the faces of multiple people appearing in each program are extracted, and the hue values ​​of each of the multiple people's faces are measured and tallied.

[0055] In addition, depending on the television program, the images may switch between studio and location footage, and the lighting environment may change, which can cause the absolute value of the skin color hue value of the face to differ. In other words, under the same lighting environment, the facial color of the same person can be said to be almost the same. Therefore, during measurement, we measured and tabulated the variation in the skin color hue values ​​of multiple people's faces under the same lighting environment, rather than the absolute hue values.

[0056] [Step S1] For each program, facial images (image areas showing faces) of people appearing in each program are extracted. Facial images that appear to be of the same person are collected and stored in a single folder for each program. Note that although the determination of whether or not someone is the same person was made by a human, it can also be done automatically by a computer.

[0057] [Step S2] Analyze the facial images in each folder for all folders to determine the hue value of each face.

[0058] FIG. 4 is an explanatory diagram illustrating the hue values ​​of face images acquired for each program in the analysis process of skin color hue values ​​according to the embodiment.

[0059] Figure 4(A) shows a group of folders that store facial images of people who appeared in "Program A" and the hue values ​​of the facial images, and Figure 4(B) shows a group of folders that store facial images of people who appeared in "Program B" and the hue values ​​of the facial images.

[0060] As shown in Figures 4A and 4B, the folders are divided into groups for each program. Facial images of the same person appearing in the program are stored in one folder. For example, a facial image is defined as an image area extracted from the entire image of one frame in which a certain person appears, where the face of that person is the face of that person.

[0061] The hue value of the skin-colored part of the face in each face image is calculated. In Figures 4A and 4B, the numbers shown near the face images are hue values. In this case, since the hue values ​​of the skin-colored part of the face in a given face image are not necessarily the same, the average hue value of the skin-colored part of the face may be used as the hue value of the face in that face image.

[0062] [Step S3] Based on the assumption that "the facial color of the same person is almost the same under the same lighting environment," for each program, among all folders for that program, only folders containing five or more valid facial images and with a degree of variation (standard deviation) of the hue values ​​of the facial images of less than 0.8 are deemed valid. Folders other than those mentioned above are deemed invalid.

[0063] [Step S4] It is checked whether the facial images in the valid folders were taken under the same lighting environment. For example, if the images in the folder are from a commercial or a VTR, the folder is invalidated.

[0064] [Step S5] The valid folders under the same lighting environment are grouped together, and the degree of variation (standard deviation) of the average hue values ​​between the folders (people) in each group is calculated.

[0065] Here, we have determined that the average facial hue value has an error of ±1°, and if the degree of variation (standard deviation) of the average hue value is 1 or less, the hue of human facial skin can be said to be approximately the same regardless of race or gender.

[0066] <Analysis Results> FIG. 5 is a diagram showing the analysis results of skin color hue values ​​according to the experimental method (part 1) of the embodiment.

[0067] Figure 5 shows a summary of data on the average hue of the facial skin color of people appearing in multiple programs (eight programs, Video A to Video H) as the images to be analyzed (videos to be analyzed), obtained for each program through the above-mentioned steps S1 to S5.

[0068] 5 includes the following items: "Valid Number of People" indicating the number of folders (number of people) determined to be valid, "Japanese or Foreign," "Minimum Hue Value" indicating the smallest hue value among the hue values ​​analyzed for each program, "Maximum Hue Value" indicating the largest hue value among the hue values ​​analyzed for each program, "Range (Difference)" indicating the range (difference) between the minimum and maximum hue values, and "Standard Deviation" indicating the degree of variation in the average hue value. Note that the items are not limited to those illustrated in FIG. 5, and other items may be added.

[0069] Although it depends on the type of program, when only facial images in a studio with a nearly constant lighting environment were considered, the variation (standard deviation) in the hue value of facial skin color was approximately 1 or less, especially for videos A to E.

[0070] The analysis results in Figure 5 are the aggregated results of 18 Japanese people (folders) and 121 ethnically diverse foreign people (folders), and it was found that there is a high possibility that the hue of human facial skin is almost constant.

[0071] (A-2-2) Analysis Method (Part 2) Next, the analysis results of the skin color hue value by the analysis method (Part 2) of the embodiment will be described.

[0072] <Analysis Method> The images to be analyzed are images of a person taken under uniform lighting in a darkroom in an office. For each of the multiple images, the hue value of the human facial skin color is measured and compiled.

[0073] FIG. 6 is an explanatory diagram for explaining a method for measuring the hue value of facial skin color for each image in a second embodiment of the present invention.

[0074] [Step S11] A person is photographed in a darkroom in an office under the same lighting environment.

[0075] [Step S12] For each image to be analyzed, human face images (image areas showing faces) are extracted.

[0076] [Step S13] The hue value of the facial skin color is calculated for each image. As shown in Figure 6, a white piece of paper was photographed along with the subject during photography, and the hue value of the facial skin color was calculated using the color of the white paper as a reference, which is pure white (saturation 0).

[0077] <Analysis Results> FIG. 7 is a diagram showing the analysis results of skin color hue values ​​according to the experimental method (part 1) of the embodiment.

[0078] In Fig. 7, images of 23 Japanese people are used as the analysis target images, and for each image, the items are "pixels" indicating the size of the facial image and "hue value" indicating the hue value of the facial skin color. Note that the items are not limited to those exemplified in Fig. 7, and other items may be added.

[0079] The degree of variation (standard deviation) of the hue values ​​shown in FIG. 7 was 0.79°, and the average hue value was 17.8°.

[0080] (A-2-3) Results of Experimental Methods (1 and 2) The results of Experimental Method (1) described above show that, although it depends on the lighting environment, the standard deviation of facial skin hue is approximately 2° or less, and in particular, when the standard deviation of facial skin hue is 1° or less, it is considered a measurement error and is judged to be a good result. Therefore, the results of Experimental Method (1) support the idea that human facial skin hue is almost constant regardless of race or gender.

[0081] The results of the above-mentioned experimental method (2) showed that the standard deviation value was 0.79, which confirmed that the verification results of the experimental method (2) were correct.

[0082] Furthermore, from the results of Figures 5 and 7, the human skin color hue value can be set to approximately 15° to 25°, preferably approximately 15° to 19°, more preferably approximately 17° to 18°, and even more preferably approximately 17.8°.

[0083] (A-3) Image Processing Operation Next, the image processing operation for color correcting a patient image in the remote medical care support system 9 according to the embodiment will be described with reference to the drawings.

[0084] FIG. 8 is a sequence diagram showing the operation of image processing relating to color correction of a patient image in the remote medical care support system 9 according to the embodiment.

[0085] Here, an example is given in which a doctor looks at the color-corrected patient image and visually examines the symptoms of jaundice that appear in the whites of the patient's eyes.

[0086] 8, first, the imaging unit 23 is activated in the patient terminal 2, and an image of the area to be treated is captured using the imaging unit 23 (step S101). The patient terminal 2 uses the captured image as a patient image, and the communication unit 25 transmits a communication signal including the patient image data to the medical institution terminal 1 (step S102).

[0087] The patient image may be captured in advance, or the patient terminal 2 may transmit the patient image captured in advance at the time of the examination.

[0088] [Steps S103 to S104] In the medical institution terminal 1, when the communication unit 14 acquires the patient image received from the patient terminal 2 (step S103), the control unit 10 requests the medical assistance server 3 to perform color correction of the patient image as necessary (step S104).

[0089] When requesting color correction of a patient image, the communication unit 15 of the medical institution terminal 1 transmits a request signal including at least image data of the patient image to the medical assistance server 3 .

[0090] [Steps S105 to S107] In the medical support server 3, when the communication unit 33 receives a request signal including patient image data (step S105), the image processing unit 50 of the control unit 30 performs color correction by changing the hue of the entire patient image using the skin color hue value (step S106). Note that color correction of the patient image will be described in detail later.

[0091] Once the image processing unit 50 has performed color correction on the entire patient image, the communication unit 33 transmits a signal including image data of the corrected patient image to the medical institution terminal 1 (step S107).

[0092] [Steps S108 to S110] In the medical institution terminal 1, when the communication unit 14 acquires the patient image color-corrected by the medical support server 3 (step S108), the display unit 12 displays the corrected patient image (step S109). This allows a doctor to view the corrected patient image, which has been hue-corrected based on the skin hue value, and appropriately diagnose the symptoms of jaundice (step S110).

[0093] (A-4) Color Correction Processing (Eyeball Color Analysis) Hereinafter, an example of color correction processing by the image processing unit 50 of the medical support server 3 will be described with reference to the drawings.

[0094] Here, an example will be given in which the color correction process of this embodiment is applied to eyeball color analysis in which a doctor looks at the color of the whites of a patient's eyes to determine whether or not symptoms of jaundice have developed.

[0095] 9 is a flowchart showing the color correction process according to the embodiment. Note that the color correction process by the image processing unit 50 uses the biconical HSL color space as the color space of the image, but is not limited to this.

[0096] [Step S201] Depending on the environment during imaging, some images may be dark due to a low amount of external light, while others may be bright due to a high amount of external light. Therefore, the color correction unit 52 automatically adjusts the brightness of the patient image (step S201).

[0097] [Step S202] Generally, the color representation of a captured image is in the RGB color mode. However, in this embodiment, the biconic HSL color space is used as the color space to adjust the hue of the image (patient image). Therefore, the color correction unit 52 converts the RGB data into HSL data (step S202). Since existing technology can be applied to the method of converting each RGB data value into each HSL data value, a detailed description thereof will be omitted here.

[0098] If the color representation of the patient image is in a color mode other than RGB, the color mode data is converted to HSL data. Conversely, in this embodiment, the hue is corrected using the biconical HSL color space, but if another color space is used, the color mode data of the image is converted to color mode data of the other color space. In either case, it is sufficient to be able to convert the color mode data of the image to be corrected to color mode data of the color space used in the color correction process.

[0099] [Step S203] The color corrector 52 extracts an image region of the skin part shown in the patient image, and derives an average hue value using the hue value in the image region of the skin part (step S203).

[0100] [Step S204] The color correction unit 52 uses the skin color hue value as a reference value and changes the average hue value of the skin part of the patient image derived in step S203 to the skin color hue value as the reference value, thereby correcting the hue of the patient image (step S204).

[0101] In other words, a hue shift correction is performed so that the hue value of the skin-colored portion of the patient image becomes the skin-color hue value, thereby changing the hue of the entire patient image. The hue (tone) of the skin color due to the external light color in the patient image before correction is corrected to a skin-color hue value that indicates the hue value of human skin color, regardless of race or gender. In this case, the skin-color hue value can be a value obtained from the above-mentioned analysis results, such as a value between approximately 15° and 25°, a value between approximately 15° and 19°, a value between approximately 17° and 18°, or a value around 17.8°. In this embodiment, a case where 17.8° is used as the skin-color hue value is exemplified.

[0102] [Step S205] Next, in order for a doctor to examine the patient's whites to determine whether or not symptoms of jaundice are present, the color correction unit 52 recognizes the pupils of the patient in the corrected patient image and identifies the positions and ranges of the left and right eyes (step S205).

[0103] [Step S206] The color correction unit 52 generates a hue histogram of the eyeball and the skin area around the eyeball, excluding the pupil area, for each of the identified eyes (step S206).

[0104] [Step S207] The color correction unit 52 creates a graph of the hue histogram (step S207).

[0105] For example, if the color of the whites of the eyes is close to yellow, it can be determined that the person is experiencing symptoms of jaundice. In this embodiment, since a biconical HSL color space is used, the hue representing yellow is 60°, but the color of jaundice is not pure yellow but a color close to yellow (for example, a color in which yellow is mixed with red or green), and therefore, if the hue is 30° or greater, it is displayed as a possible indication of jaundice.

[0106] 10 to 13 are diagrams showing patient images before and after color correction, and hue histograms. As color correction, automatic brightness adjustment and hue correction were performed on the acquired patient images before correction.

[0107] It should be noted that the patient images shown in Figures 10 to 13 have been edited due to patient portrait rights and restrictions on drawing notation in patent applications.

[0108] As a method of implementation, we had a doctor actually visually examine multiple patients individually, and we also generated hue histograms based on the corrected patient images (Figures 10 to 13) of the same patients. Then, we verified, in the presence of the doctor, whether the results of the visual examination corresponded to the results of the hue histograms based on the patient images.

[0109] In conclusion, the visual inspection results for all patients and the hue histogram results based on the patient images (after correction) for each patient were all consistent.

[0110] In Fig. 10, the color correction unit 52 generated a hue histogram of the patient's left and right eyeballs (excluding the pupils) and the surrounding skin area in the corrected patient image. The hue histogram results in Fig. 10 show that the distribution is strong below a hue of 30°, but also appears at hues above 30°, where the skin color can be judged to be yellowish (i.e., suspected jaundice). From this, it can be inferred that jaundice is suspected, and when compared with the results of the doctor's visual examination, it was determined that jaundice was present.

[0111] The hue histogram results in Figure 11 show that a stronger distribution appears at hue angles of 30° or greater than in the case of the patient in Figure 10. In the corrected patient image in Figure 11, the whites of the patient's eyes are yellowed, and visual examination by a doctor revealed that the patient had more severe jaundice symptoms than the patient in Figure 10.

[0112] In Figure 12, the corrected patient image shows that the whites of the patient's eyes are not yellow, but the hue histogram results show that a slight distribution appears at hue angles of 30° or more. A doctor's visual examination revealed that the patient was exhibiting symptoms of jaundice. Although it was difficult to determine the symptoms of jaundice even in the corrected patient image, it was possible to determine the symptoms by referring to the hue histogram results.

[0113] The hue histogram in Figure 13 shows that a strong distribution appears above a hue of 30°, and the doctor's visual examination revealed that the patient was suffering from jaundice. The distribution appears between a hue of 53.4° and 98.4°, which indicates that depending on the patient's personality, the color may appear closer to green.

[0114] (A-5) Effects of the Embodiment As described above, according to this embodiment, the value indicating the hue of human skin color in an image is substantially the same regardless of race or gender, and the skin color hue value can be used as a reference value to perform hue correction on the image. As a result, by using the corrected image, it can be displayed on a display device such as a monitor even if the lighting environment at the time of capture was different.

[0115] Furthermore, by applying the present invention to a remote medical examination support system and displaying a patient image that has undergone hue correction on a terminal at a medical institution, doctors can perform an appropriate visual examination.

[0116] (B) Other Embodiments Although various modified embodiments have been mentioned in the above-described embodiment, the present invention can also be applied to the following modified embodiments.

[0117] (B-1) In the above-described embodiment, the present invention is applied to a remote medical care support system, but the present invention can be widely applied to any system that requires converting images of people into the hue of their original skin color. For example, the present invention can be applied to a video system that corrects the skin color of performers in television programs, commercials, etc.

[0118] (B-2) In the above-described embodiment, the medical support server performs hue correction using the skin color hue value as a reference value, and the medical institution terminal displays the corrected image. However, the medical institution terminal may perform hue correction using the skin color hue value as a reference value and display the corrected image. In this case, the medical institution terminal functions as the "image display device" of the present invention.

[0119] (B-3) In addition, if the patient terminal is equipped with an imaging unit, the imaging unit may perform hue correction on the captured image using the skin color hue value as a reference value and output the corrected image. In this case, the imaging unit or the patient terminal functions as the "imaging device" of the present invention.

[0120] 9: Remote medical care support system, NT: Network, 1: Medical institution terminal, 10: Control unit, 11: Input unit, 12: Display unit, 13: Memory unit, 14: Communication unit, 15: Communication unit, 2: Patient terminal, 20: Control unit, 21: Input unit, 22: Display unit, 23: Imaging unit, 24: Memory unit, 25: Communication unit, 3: Medical care support server, 30: Control unit, 32: Memory unit, 33: Communication unit, 50: Image processing unit, 51: Image acquisition unit, 52: Color correction unit, 53: Output unit.

Claims

1. An image processing device comprising: an image acquisition means for acquiring image data of an image showing a human; a color correction means for correcting a hue of an image area occupied by the human skin in said image so that the hue of the area becomes a value indicating the human skin hue; and an output means for outputting the corrected image to a display unit.

2. The image processing device according to claim 1, wherein the value indicating the human skin hue is any value between 15° and 25° of hue in the HSL color space.

3. The image processing device according to claim 1, characterized in that the color correction means comprises: an image area identification unit that identifies an image area in the image occupied by human skin; a hue derivation unit that derives an average hue value of the identified image area; and a correction unit that shifts and corrects the average hue value of the image area to a value indicating the human skin hue.

4. An image processing method comprising: an image acquisition means for acquiring image data of an image in which a human is shown; a color correction means for correcting a hue of an image area occupied by the human skin in said image so that the hue of the image area becomes a value indicative of the human skin hue; and an output means for outputting the corrected image to a display unit.

5. An image processing program that causes a computer to function as: an image acquisition means for acquiring image data of an image showing a human; a color correction means for correcting a hue of an image area occupied by the human skin in said image so that the hue of the area becomes a value indicating the human skin hue; and an output means for outputting the corrected image to a display unit.

6. An image display device which displays an image containing a human on a display unit, comprising: an image acquisition means for acquiring image data of the image containing the human; a color correction means for correcting a hue of an image area occupied by the human's skin in the image so that the hue of the area becomes a value indicating the human's skin hue; and an output means for outputting the corrected image to the display unit.

7. An imaging device comprising: an imaging unit; an image acquisition means for acquiring image data of an image captured by the imaging unit, the image data including a human; a color correction means for correcting a hue of an image area occupied by the human skin in the image so that the hue of the image area becomes a value indicating the human skin hue; and an output means for outputting the corrected image.

Images