Image processing apparatus, image processing method, image processing program, image display device, and imaging device

Abstract

By performing predetermined color correction on an image in which a person is shown, an image with a similar hue is provided even when the lighting environment at the time of shooting is different. 【Solution means】The image processing apparatus of the present invention includes an image acquisition means for acquiring image data of an image in which a person is shown, a color correction means for correcting the hue so that a value indicating the hue of the image area occupied by the skin portion of the person in the image becomes a value indicating the human skin hue, and an output means for outputting the corrected image to a display unit.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, an image processing program, an image display apparatus, and an imaging apparatus.

Background Art

[0002] In recent years, medical treatment using information and communication devices (so-called "online medical treatment" or "remote medical treatment") is expected to play an important role from the viewpoints of improving the quality of medical care, improving the convenience of patients, and rectifying regional disparities in medical care in remote islands and mountainous areas. In medical treatment by a doctor, visual inspection for assessing the physical condition is performed.

[0003] For example, when a doctor examines the symptoms of jaundice through online medical treatment, the doctor examines by looking at the patient's images such as the sclera and skin where the symptoms of jaundice are likely to appear. At this time, visual inspection is performed by looking at the patient image displayed on the display on the medical institution side. However, the hue and brightness differ depending on the performance of the camera that photographed the patient, the amount of external light at the time of shooting, and the way the light hits. That is, the lighting environment at the time of shooting affects. Therefore, in online medical treatment, in order for the doctor to perform correct visual inspection, it is required to correct the color of the image displayed on the display on the medical institution side.

[0004] Patent Document 1 discloses a remote medical treatment using color information such as the color of a patient's skin and tongue color. For example, when photographing a patient, a color chart is photographed at the same time, and when displaying an image, the color in the color chart displayed on the display unit is corrected so as to approach the color measured under the light source at the location where the display unit is placed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, there may be a nuisance that it is necessary to take a picture together with a pre-determined color chart.

[0007] Therefore, in order to solve the above-described problems, an object of the present invention is to provide an image having a similar hue even when the illumination environment at the time of shooting is different by performing predetermined color correction on an image in which a person is reflected.

Means for Solving the Problems

[0008] In order to solve such problems, an image processing apparatus according to a first aspect of the present invention includes: (1) an image acquisition unit that acquires image data of an image in which a person is reflected; and (2) a value indicating the hue of an image area occupied by a human skin portion in the image, Regardless of race and gender a color correction unit that corrects the hue so as to be a value indicating the human skin color hue; and (3) an output unit that outputs the corrected image to a display unit. Regardless of race and gender, the value indicating the human skin color hue is any value between 15° and 25° of the hue in the HSL color space, and the color correction means shifts and corrects the value indicating the hue of the image area occupied by the human skin part in the image to the value indicating the human skin color hue, and also corrects the hue of the entire area of the image It is characterized by that.

[0009] An image processing method according to a second aspect of the present invention includes: (1) an image acquisition unit acquires image data of an image in which a person is reflected; and (2) a color correction unit corrects the hue of an image area occupied by a human skin portion in the image so that the value indicating the hue becomes a value indicating the human skin color hue, Regardless of race and gender and (3) an output unit outputs the corrected image to a display unit. Regardless of race and gender, the value indicating the human skin color hue is any value between 15° and 25° of the hue in the HSL color space, and the color correction means shifts and corrects the value indicating the hue of the image area occupied by the human skin part in the image to the value indicating the human skin color hue, and also corrects the hue of the entire area of the image It is characterized by that.

[0010] An image processing program according to a third aspect of the present invention causes a computer to function as: (1) an image acquisition unit that acquires image data of an image in which a person is reflected; and (2) a color correction unit that corrects the hue of an image area occupied by a human skin portion in the image so that the value indicating the hue becomes a value indicating the human skin color hue, Regardless of race and gender and (3) an output unit that outputs the corrected image to a display unit. Regardless of race and gender, the value indicating the human skin color hue is any value between 15° and 25° of the hue in the HSL color space, and the color correction means shifts and corrects the value indicating the hue of the image area occupied by the human skin part in the image to the value indicating the human skin color hue, and also corrects the hue of the entire area of the image It is characterized by that.

[0011] The fourth image display device of the present invention is an image display device that displays an image in which a person is reflected, and includes: (1) an image acquisition unit that acquires image data of an image in which a person is reflected; and (2) a value indicating the hue of an image area occupied by the skin portion of the person in the image, Regardless of race and gender a color correction unit that corrects the hue so as to obtain a value indicating the skin color hue of the person; and (3) an output unit that outputs the corrected image to the display unit. Regardless of race and gender, the value indicating the human skin color hue is any value between 15° and 25° of the hue in the HSL color space, and the color correction means shifts and corrects the value indicating the hue of the image area occupied by the human skin part in the image to the value indicating the human skin color hue, and also corrects the hue of the entire area of the image It is characterized by the above.

[0012] The imaging device of the fifth aspect of the present invention is an imaging device, and includes: (1) an imaging unit; and (2) an image acquisition unit that acquires image data of an image in which a person is reflected, which is captured by the imaging unit. Regardless of race and gender (3) a color correction unit that corrects the hue so that a value indicating the hue of the image area occupied by the skin portion of the person in the image becomes a value indicating the skin color hue of the person; and (4) an output unit that outputs the corrected image. Regardless of race and gender, the value indicating the human skin color hue is any value between 15° and 25° of the hue in the HSL color space, and the color correction means shifts and corrects the value indicating the hue of the image area occupied by the human skin part in the image to the value indicating the human skin color hue, and also corrects the hue of the entire area of the image It is characterized by the above.

Effect of the Invention

[0013] According to the present invention, by performing predetermined color correction on an image in which a person is reflected, it is possible to provide an image with a similar hue even when the lighting environment at the time of shooting is different.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0015] (A) Embodiment Hereinafter, embodiments of the image processing apparatus, image processing method, image processing program, image display apparatus, and imaging apparatus according to the present invention will be described in detail with reference to the drawings.

[0016] In this embodiment, the case where the present invention is applied to a remote medical support system is exemplified. However, the present invention is not limited to the remote medical support system. The present invention can be widely applied when performing color correction of a display image when displaying an image on a display device. It can also be applied when performing color correction of image information output by an imaging device.

[0017] Note that, for example, in the notation of "A~B" indicating a numerical range, the value A indicates the lower limit value, the value B indicates the upper limit value, and both the value A and the value B are included in the numerical range.

[0018] (A-1) Configuration of the Embodiment FIG. 2 is a configuration diagram showing the overall configuration of the remote medical support system 9 according to the embodiment.

[0019] In FIG. 2, the remote medical support system 9 includes a medical institution terminal 1, a patient terminal 2, and a medical support server 3 that can be connected to the network NT.

[0020] The remote medical support system 9 causes the patient terminal 2 to transmit a patient image in which a part to be examined of the patient is shown to the medical institution terminal 1. The patient image by the doctor is displayed on the display unit 12 of the medical institution terminal 1, and the doctor performs visual inspection while looking at the displayed patient image.

[0021] Here, the color of the image displayed on the display unit 12 of the medical institution terminal 1 varies depending on the performance of the camera that captured the patient image, the lighting environment at the time of shooting, etc., that is, the amount of external light hitting the patient. Therefore, the color (especially the hue) of the displayed image may not appear appropriately, and the doctor may not be able to examine correctly.

[0022] Therefore, when displaying the patient image on the display unit 12 of the medical institution terminal 1, in response to a request from the medical institution terminal 1, the medical support server 3 corrects the color of the displayed image and transmits the corrected patient image to the medical institution terminal 1. As a result, the patient image after color correction is displayed on the display unit 12 of the medical institution terminal 1, so that the doctor can perform visual inspection correctly.

[0023] In this embodiment, the medical support server 3 and the medical institution terminal 1 can exchange information with each other, and an example is shown in which the medical support server 3 performs color correction of the patient image in response to a request from the medical institution terminal 1.

[0024] For example, the function of the medical support server 3 is on a cloud server, and the function of the medical support server 3 is provided to the medical institution terminal 1. In this case, the medical institution terminal 1 only performs input and display using a browser or a dedicated application, and data management and various processes are performed by the medical support server 3 as a cloud server.

[0025] As another example, for instance, the functions of the medical support server 3 may be provided as application software (e.g., an image processing program or the like). In this case, by installing the application software on the medical institution terminal 1, various functions such as color correction may be enabled on the medical institution terminal 1.

[0026] In this embodiment, an example is given where the medical institution terminal 1 requests the medical support server 3 for color correction instructions. However, the present invention is not limited to this. The medical institution terminal 1 on which the image processing program is installed may automatically perform color correction on the patient image and be able to display the color-corrected patient image.

[0027] <Patient terminal 2> The patient terminal 2 is a communication terminal on the patient side and is a communication terminal connectable to the network NT. The patient terminal 2 is operated by the patient himself / herself, the patient's caregiver, or the like. The patient terminal 2 may be, for example, a smartphone, a tablet terminal, a notebook personal computer, a dedicated terminal, a mobile terminal, a wearable terminal, or the like.

[0028] For example, the patient terminal 2 includes a control unit 20 that controls various functions performed on the patient terminal 2, an input unit 21 that receives information by user operation such as the input functions of a keyboard, a mouse, and a touch panel display, a display unit 22 such as a liquid crystal display that displays input information or the doctor's examination results, an imaging unit 23 represented by a camera, a storage unit 24 that stores a processing program operating on the patient terminal 2, data necessary for processing, and captured images, and a communication unit 25 that exchanges information with the network NT.

[0029] In this embodiment, an example is given where the patient terminal 2 includes an imaging unit 23 that captures the patient's image. However, it is also possible to capture the patient image with another camera and transmit the patient image to the medical institution terminal 1 during the medical examination.

[0030] <Medical institution terminal 1> The medical institution terminal 1 is operated by medical staff such as doctors and is a communication terminal that can be connected to the network NT. The medical institution terminal 1 can be applied to various terminals as long as it can display patient images for doctors to perform visual examinations. For example, it can be a smartphone, a tablet terminal, a notebook personal computer, a dedicated terminal, a mobile terminal, etc.

[0031] For example, the medical institution terminal 1 includes a control unit 10 that manages various functions performed by the medical institution terminal 1, an input unit 11 that receives information through user operations such as the input functions of a keyboard, a mouse, and a touch panel display, a display unit 12 that displays input information or patient images such as a liquid crystal display, a storage unit 13 that stores processing programs operating on the medical institution terminal 1, data necessary for processing, and patient images (including images after color correction), and a communication unit 14 that exchanges information with the network NT.

[0032] In this embodiment, the medical institution terminal 1 receives a patient image from the patient terminal 2, performs color correction on the patient image, and illustrates the case where a doctor conducts a medical examination. However, it is not limited to this, and color correction may be performed on pre-stored patient images.

[0033] <Medical treatment support server> The medical treatment support server 3 can be connected to the network NT and supports medical treatment by doctors.

[0034] Figure 1 is an internal configuration diagram showing the internal configuration of the medical treatment support server 3 according to the embodiment.

[0035] In Figure 1, the medical treatment support server 3 includes a control unit 30, a storage unit 32, and a communication unit 33.

[0036] The control unit 30 is responsible for the functions of the medical support server 3. The control unit 30 includes a CPU, ROM, RAM, EEPROM, input / output interface, etc. The CPU realizes various functions of the medical support server 3 by executing processing programs (such as image processing programs, medical support programs, etc.). As one of the functions to be executed, the control unit 30 has an image processing unit 50.

[0037] The image processing unit 50 performs predetermined image processing on the image to be examined. The image processing unit 50 includes an image acquisition unit 51, a color correction unit 52, and an output unit 53.

[0038] The image acquisition unit 51 acquires a patient image to be examined from the medical institution terminal 1 or the patient terminal 2, and provides the acquired patient image to the color correction unit 52.

[0039] The color correction unit 52 performs color correction of the patient image using the hue value indicating the human skin color.

[0040] Here, the inventor of the present application has made diligent efforts in research and has led to the conclusion that, as will be described later, regardless of race and gender, the hue value of the human skin color (hereinafter, "skin color hue value") is almost the same. For example, although the skin colors of white people and black people look different, it is considered that the difference in the amount of melanin contained in the skin and the amount of hemoglobin flowing in the blood vessels under the skin have an impact. For example, when the amount of melanin is large, the skin color is dark, and when it is small, the skin color is light. Another example is that the color tone also changes depending on the increase or decrease of hemoglobin. In this way, the brightness (lightness) and color tone (colorfulness) of the skin color can change, but the hue value of the skin color can take almost the same value regardless of race and gender.

[0041] Therefore, even if there are differences in camera performance, display performance, etc., the color correction unit 52 can create an image with a more appropriate hue by correcting the hue of the patient image using the skin color hue value as a reference value.

[0042] The output unit 53 outputs the patient image corrected by the color correction unit 52.

[0043] The storage unit 32 stores a processing program (for example, an image processing program, a medical support program, etc.) and data necessary for processing.

[0044] The communication unit 33 communicates with the medical institution terminal 1 etc. through the network NT.

[0045] (A-2) Analysis of skin color hue value First, the applicant of the present application analyzed and derived the characteristic that the hue values of human skin colors are almost the same regardless of race and gender.

[0046] Here, the human face image (image area where the face is reflected) in the analysis target image (or analysis target video) is extracted, the hue value of the face image is measured, and the result of the analysis that the skin color hue values are almost the same is explained.

[0047] When measuring the hue value, if the lighting environment at the time of shooting is different, the brightness of the face of the human being to be analyzed is different. Therefore, below, an analysis method (the first one) for deriving the skin color hue value using a TV program as the analysis target image and an analysis method (the second one) for deriving the skin color hue value using an image taken under the same lighting environment are performed, and the analysis results for each experimental method are explained.

[0048] Note that the analysis of the skin color hue value was performed by analyzing the color space of the analysis target image using image processing software on a computer.

[0049] In this embodiment, the double-cone type HSL color space, which is said to be close to the human vision, is used as the color space representation system of the image, but it is not limited to this, and other color space representation systems may be used.

[0050] As shown in FIG. 3(A), the double-cone type HSL color space is a double-cone type color space composed of three components: hue, saturation, and luminance.

[0051] As illustrated in FIG. 3(B), "hue" can be represented by an angle in the range of 0° to 360°. 0° represents red, 60° represents yellow, 120° represents green, 180° represents cyan, 240° represents blue, and 300° represents magenta.

[0052] "Luminance" can be represented by 256 numbers from "0 to 255" for the distance in the height direction of the vertical axis shown in FIG. 3(A). The value "0" represents pure black, the value "128" represents medium brightness, and the value "255" represents pure white.

[0053] "Chroma" can be represented by 256 numbers from "0 to 255" for the radial distance from the vertical axis passing through the center on the plane. The value "0" represents no color (black and white), and the value "255" represents the maximum vividness (pure color).

[0054] (A-2-1) Analysis method (Part 1) <Analysis method> The image (video) to be analyzed is a TV program. Using multiple TV programs, for each program, each face of multiple people shown in the program is extracted, and the hue values of each face of multiple people are measured and aggregated.

[0055] Note that depending on the TV program, the video may switch, such as studio video and on-location video, and the lighting environment may change, so the absolute value of the skin color hue value of the face may be different. In other words, under the same lighting environment, the face colors of the same person can be said to be almost the same. Therefore, during measurement, instead of the absolute value of the hue value, the variation in the skin color hue values of the faces of multiple people under the same lighting environment was measured and aggregated.

[0056] [Step S1] For each program, extract the face images (image areas where faces are shown) of people shown in each program. Group the face images that are considered to be of the same person into one folder and store them for each program. Note that the judgment of the same person was made by humans, but it may also be automatically discriminated by a computer.

[0057] [Step S2] For all folders, analyze the face images in each folder to obtain the hue values of each face.

[0058] FIG. 4 is an explanatory diagram for explaining the hue values of face images acquired for each program in the analysis process of the skin color hue values of the embodiment.

[0059] FIG. 4(A) is a group of folders storing a human face image and the hue value of the face image shown in "Program A", and FIG. 4(B) is a group of folders storing a human face image and the hue value of the face image shown in "Program B".

[0060] As illustrated in FIGS. 4(A) and 4(B), the group of folders is divided for each program. The face images of the same person shown in the program are stored in one folder. For example, for a face image, an image region occupied by the face of the person in the entire image of one frame in which the person is shown is extracted, and this is regarded as the face image here.

[0061] The hue value of the skin color part of the face in each face image is obtained. In FIGS. 4(A) and 4(B), the numbers shown near the face images are the hue values. At this time, in a certain face image, the hue values of the skin color parts of the face are not necessarily the same, so the average value of the hue values of the skin color parts of the face may be used as the hue value of the face in the face image.

[0062] [Step S3] Based on the assumption that "the colors of the same person are almost the same under the same lighting environment", for each program, among all the folders of each program, only the folders in which the number of valid face images in the folder is 5 or more and the value of the degree of variation (standard deviation) of the hue values of the face images is less than 0.8 are regarded as valid. Folders other than the above are regarded as invalid.

[0063] [Step S4] Check whether the face images in the valid folders are taken under the same lighting environment. For example, if the images in the folder are those in the CM or VTR images, the folder is made invalid.

[0064] [Step S5] Group together the valid folder groups in the same lighting environment, and calculate the degree of variation (standard deviation) of the average hue values among the folders (persons) within each group.

[0065] Here, regarding the value of the average hue value of the face, with an error of ±1°, if the degree of variation (standard deviation) of the average hue value is 1 or less, it was determined that regardless of race or gender, the hue of the human facial skin is almost the same.

[0066] <Analysis Results> FIG. 5 is a diagram showing the analysis results of skin color hue values by the experimental method (Part 1) in the embodiment.

[0067] FIG. 5 shows a summary of data on the average hue values of the facial skin colors of humans appearing in each of a plurality of programs (8 programs from Video A to Video H) used as analysis target images (analysis target videos), and obtained by the above steps S1 to S5 for each program.

[0068] FIG. 5 has items such as "Valid Number of People" indicating the number of folders (number of people) determined to be valid, "Japanese or Foreigner Classification", "Minimum Hue Value" indicating the smallest value among the hue values analyzed for each program, "Maximum Hue Value" indicating the largest value among the hue values analyzed for each program, "range (difference)" indicating the range (difference) between the minimum hue value and the maximum hue value, and "Standard Deviation" indicating the degree of variation of the average hue value. Note that not limited to the items illustrated in FIG. 5, other items may be added.

[0069] Depending on the type of program, especially for Videos A to E, when only face images with a substantially constant studio lighting environment were targeted, the variation (standard deviation) of the hue values of the facial skin color was almost 1 or less.

[0070] The analysis results in FIG. 5 are the aggregated results in which 18 Japanese people (folders) and 121 foreigners (folders) with diverse races are valid, and it was found that there is a high possibility that the hue of the human facial skin is almost constant.

[0071] (A-2-2) Analysis Method (Part 2) Next, the analysis results of the skin color hue value by the analysis method (Part 2) of the embodiment will be described.

[0072] <Analysis method> The image to be analyzed is an image of a human taken under the same illumination in a dark room of an office. For each of the plurality of images, the hue values of the facial skin color of the human are measured and aggregated.

[0073] FIG. 6 is an explanatory diagram for explaining a method of measuring the hue value of the facial skin color for each image in the implementation method (Part 2) of the embodiment.

[0074] [Step S11] Take a picture of a human in the same illumination environment in a dark room of an office.

[0075] [Step S12] For each image to be analyzed, extract the facial image (image area where the face is reflected) of the human.

[0076] [Step S13] For each image, obtain the hue value of the facial skin color. Here, as shown in FIG. 6, at the time of shooting, a white paper was photographed together with the subject, and the hue value of the facial skin color was obtained with the color of the white paper as a reference of pure white (chroma 0).

[0077] <Analysis result> FIG. 7 is a diagram showing the analysis results of the skin color hue value by the experimental method (Part 1) in the embodiment.

[0078] FIG. 7 takes the images of 23 Japanese people as the images to be analyzed, and for each image, the items are "pixel" indicating the size of the facial image and "hue value" indicating the hue value of the facial skin color. Note that not limited to the items illustrated in FIG. 7, other items may be added.

[0079] The value of the degree of variation (standard deviation) of the hue values illustrated in FIG. 7 was 0.79°, and the hue average value was 17.8°.

[0080] (A-2-3) Results of the experimental methods (Part 1 and Part 2) From the results of the above-described experimental method (Part 1), due to the difference in the lighting environment, the value of the standard deviation of the facial muscle hue was about 2° or less. In particular, when the value of the standard deviation of the facial muscle hue was 1° or less, it was judged as a good result with measurement error. From this, the results of the experimental method (Part 1) supported that the hue of the human facial muscles was almost constant regardless of race or gender.

[0081] From the results of the above-described experimental method (Part 2), since the value of the standard deviation was 0.79, it supported that the verification results of the experimental method (Part 2) were correct.

[0082] Also, from the results of FIGS. 5 and 7, the human skin color hue value can be about 15° to 25°, preferably about 15° to 19°, more preferably about 17° to 18°, and even more preferably about 17.8°.

[0083] (A-3) Image processing operation Next, in the telemedicine support system 9 according to the embodiment, the operation of image processing for color-correcting a patient image will be described with reference to the drawings.

[0084] FIG. 8 is a sequence diagram showing the operation of image processing related to color correction of a patient image in the telemedicine support system 9 according to the embodiment.

[0085] Here, an example is illustrated in which a doctor views the color-corrected patient image and visually examines the symptoms of jaundice appearing in the patient's sclera.

[0086] [Steps S101~S102] In FIG. 8, first, in the patient terminal 2, the imaging unit 23 is activated, and the part to be examined is imaged using the imaging unit 23 (step S101). The patient terminal 2 uses this imaged image as a patient image, and the communication unit 25 transmits a communication signal including the patient image data to the medical institution terminal 1 (step S102).

[0087] Note that the patient image may have been taken in advance, or the patient terminal 2 may be configured to transmit the patient image taken in advance during the examination.

[0088] [Steps S103 - S104] When the communication unit 14 of the medical institution terminal 1 acquires the patient image received from the patient terminal 2 (step S103), the control unit 10 requests color correction of the patient image from the medical support server 3 as necessary (step S104).

[0089] When requesting color correction of the patient image, in the medical institution terminal 1, the communication unit 15 transmits a request signal including at least the image data of the patient image to the medical support server 3.

[0090] [Steps S105 - S107] When the communication unit 33 of the medical support server 3 receives a request signal including patient image data (step S105), the image processing unit 50 of the control unit 30 performs color correction by changing the hue of the entire patient image using the skin color hue value (step S106). Details of the color correction of the patient image will be described later.

[0091] When the image processing unit 50 performs color correction on the entire patient image, the communication unit 33 transmits a signal including the image data of the corrected patient image to the medical institution terminal 1 (step S107).

[0092] [Steps S108 - S110] When the communication unit 14 of the medical institution terminal 1 acquires the corrected patient image color - corrected by the medical support server 3 (step S108), the display unit 12 displays the corrected patient image (step S109). As a result, the doctor can appropriately examine the symptoms of jaundice by looking at the corrected patient image whose hue has been corrected based on the skin color hue value (step S110).

[0093] (A - 4) Color Correction Process (Eye Color Analysis) Hereinafter, an example of the color correction process by the image processing unit 50 of the medical support server 3 will be described with reference to the drawings.

[0094] Here, an example is given of a case where the color correction process of this embodiment is applied to the eyeball color analysis in which a doctor examines the color of the sclera of a patient to determine whether or not the symptoms of jaundice are present.

[0095] FIG. 9 is a flowchart showing the color correction process according to the embodiment. Note that the color correction process by the image processing unit 50 is exemplified for the case of using the double-cone type HSL color space as the color space of the image, but it is not limited to this.

[0096] [Step S201] Depending on the environment at the time of shooting, there are dark images due to a small amount of external light, and there are also bright images due to a large amount of external light. Therefore, the color correction unit 52 automatically adjusts the brightness of the patient image (step S201).

[0097] [Step S202] Generally, the RGB color mode is used for the color representation of the captured image. On the other hand, in this embodiment, since the double-cone type HSL color space is used as the color space to adjust the hue of the image (patient image), the color correction unit 52 converts the RGB data into HSL data (step S202). Since the method of converting each RGB data value into each HSL data value can apply existing techniques, detailed description here is omitted.

[0098] Note that when the color representation of the patient image is in a color mode other than RGB, the color mode data is converted into HSL data. Conversely, in this embodiment, the double-cone type HSL color space is used to correct the hue. When using another color space, the color mode data of the image is converted into the color mode data of the other color space. In any case, it is sufficient that the color mode data of the image to be corrected can be converted into the color mode data of the color space used in the color correction process.

[0099] [Step S203] The color correction unit 52 extracts the image area of the skin part shown in the patient image, and derives the hue average value using the hue value in the image area of the skin part (step S203).

[0100] [Step S204] The color correction unit 52 uses the skin color hue value as a reference value, and changes the hue average value of the skin part of the patient image derived in step S203 to the skin color hue value as the reference value to correct the hue of the patient image (step S204).

[0101] That is, hue shift correction is performed so that the hue value of the skin color part in the patient image becomes the skin color hue value, and the hue of the entire patient image is changed. The hue (color tone) of the skin color due to the external light color in the patient image before correction is hue-corrected so as to become the skin color hue value indicating the hue value of human skin color regardless of race and gender. At this time, as the skin color hue value, a value obtained from the above analysis result, for example, any value of about 15° to 25°, or any value of about 15° to 19°, or any value of about 17° to 18°, or a value of about 17.8° can be used. In this embodiment, the case of using 17.8° as the skin color hue value is illustrated.

[0102] [Step S205] Next, in order for the doctor to examine whether the patient has jaundice symptoms by looking at the color of the sclera of the patient, the color correction unit 52 recognizes the pupil part of the patient in the corrected patient image and specifies the positions and ranges of the left and right eyes (step S205).

[0103] [Step S206] The color correction unit 52 generates a hue histogram of the eyeball and the skin part around the eyeball excluding the pupil part of each of the specified eyes (step S206).

[0104] [Step S207] The color correction unit 52 graphs the hue histogram (step S207).

[0105] For example, when the color of the sclera is close to yellow, it can be determined that the symptoms of jaundice are present. In this embodiment, since the double-cone type HSL color space is used, the hue indicating yellow is 60°, but the color of jaundice is not pure yellow, but a color near yellow (for example, a color in which red or green is mixed with yellow). Therefore, when the hue is 30° or more, it is displayed as having a possibility of jaundice.

[0106] <Example> Figs. 10 to 13 are diagrams showing the patient images before and after color correction and the hue histogram. As color correction, automatic adjustment of brightness and hue correction were performed on the acquired patient image before correction.

[0107] Note that due to the relationship of the patient's portrait right and the restrictions on the drawing notation related to the patent application, it should be noted that the patient images shown in Figs. 10 to 13 are processed.

[0108] As an implementation method, doctors were actually asked to individually examine a plurality of patients, and a hue histogram based on the corrected patient images (Figs. 10 to 13) of the same patient was generated. Then, whether the results of the physical examination and the results of the hue histogram based on the patient images correspond to each other was verified under the presence of doctors.

[0109] As a conclusion, all the results of the physical examination of all patients and the results of the hue histogram based on the patient images (after correction) of each patient were all in agreement.

[0110] In Fig. 10, the color correction unit 52 generated a hue histogram of the eyeballs (excluding the pupil part) of the patient's left and right eyes and the surrounding skin part in the corrected patient image. From the results of the hue histogram in Fig. 10, it can be seen that it is strongly distributed at less than 30° of hue, but a distribution also appears at 30° or more of hue where the color of the skin part is yellowish (that is, suspected of jaundice). From this, it can be inferred that there is a suspicion of jaundice, and when compared with the results of the physical examination by doctors, it was found to be jaundice.

[0111] From the results of the hue histogram in Fig. 11, it can be seen that a strong distribution appears at a hue of 30° or more, compared to the patient in Fig. 10. In the corrected patient image of Fig. 11, the sclera of the patient has turned yellow, and from the visual inspection results by the doctor, it was found that the jaundice symptoms are stronger than those of the patient in Fig. 10.

[0112] In Fig. 12, although the sclera part of the patient has not turned yellow in the corrected patient image, it can be seen from the results of the hue histogram that a slight distribution appears at a hue of 30° or more. From the doctor's visual inspection results, it was found that the patient has jaundice symptoms. Although it was difficult to judge the jaundice symptoms even in the corrected patient image, it could be judged by referring to the hue histogram results.

[0113] From the results of the hue histogram in Fig. 13, it can be seen that a strong distribution appears at a hue of 30° or more, and from the doctor's visual inspection results, it was found that jaundice symptoms are present. Since the distribution appears at a hue of 53.4° to 98.4°, it can be seen that depending on the individuality of the patient, it may appear as a color close to green.

[0114] (A-5) Effects of the Embodiment As described above, according to this embodiment, in an image, the value indicating the hue of human skin color is almost the same regardless of race or gender, and the image can be hue-corrected using the skin color hue value as a reference value. As a result, by using the corrected image, it can be displayed on a display device such as a display even when the lighting environment at the time of shooting is different.

[0115] In addition, by applying the present invention to a telemedicine support system and displaying the corrected patient image after hue correction on the medical institution side terminal, appropriate visual inspection by a doctor becomes possible.

[0116] (B) Other Embodiments Although various modified embodiments have been mentioned in the above-described embodiment, the present invention can also be applied to the following modified embodiments.

[0117] (B-1) In the above-described embodiment, the case where the present invention is applied to a remote medical support system has been exemplified. However, the present invention can be widely applied to any system that needs to convert an image of a human being into the hue of the original human skin color. For example, the present invention can be applied to a video system that corrects the skin color of performers in a TV program or commercial.

[0118] (B-2) In the above-described embodiment, the case where the medical support server performs hue correction using the skin color hue value as a reference value and the medical institution terminal displays the corrected image has been exemplified. However, the medical institution terminal may perform hue correction using the skin color hue value as a reference value and display the corrected image. In this case, the medical institution terminal functions as the "image display device" of the present invention.

[0119] (B-3) Also, when an imaging unit is mounted on the patient terminal, the imaging unit may perform hue correction on the captured image using the skin color hue value as a reference value so that the corrected image can be output. In this case, the imaging unit or the patient terminal functions as the "imaging device" of the present invention.

Explanation of Reference Signs

[0120] 9: Remote medical support system, NT: Network, 1: Medical institution terminal, 10: Control unit, 11: Input unit, 12: Display unit, 13: Storage unit, 14: Communication unit, 15: Communication unit, 2: Patient terminal, 20: Control unit, 21: Input unit, 22: Display unit, 23: Imaging unit, 24: Storage unit, 25: Communication unit, 3: Medical support server, 30: Control unit, 32: Storage unit, 33: Communication unit, 50: Image processing unit, 51: Image acquisition unit, 52: Color correction unit, 53: Output unit.

Claims

1. An image acquisition means for acquiring image data of an image including a human being; a color correction means for correcting a hue of an image region occupied by human skin in said image to a value indicating a human skin hue regardless of race or sex; an output means for outputting the corrected image to a display unit; Equipped with the value indicating the human skin hue is any value between 15° and 25° of the hue in the HSL color space, The color correction means shifts and corrects a value indicating a hue of an image region occupied by a human skin part in the image to a value indicating the human skin hue, and also corrects the hue of the entire region of the image.

13. An image processing device comprising:

2. The color correction means an image area specifying unit for specifying an image area occupied by human skin in the image; a hue deriving unit that derives an average hue value of the specified image region; a correction unit that shifts and corrects the hue average value of the image region to a value that indicates the human skin hue; 2. The image processing apparatus according to claim 1, further comprising:

3. The color correction means creates a hue histogram using values ​​indicating the corrected hues of image areas other than the human skin part in the image after the hue correction, The output means outputs the hue histogram result.

2. The image processing device according to claim 1,

4. The image acquisition means acquires image data of an image including a human being; a color correction means for correcting a hue of an image region occupied by human skin in the image so that the hue of the image region becomes a value indicating a human skin hue regardless of race or sex; The output means outputs the corrected image to the display unit; the value indicating the human skin hue is any value between 15° and 25° of the hue in the HSL color space, The color correction means shifts and corrects a value indicating a hue of an image region occupied by a human skin part in the image to a value indicating the human skin hue, and also corrects the hue of the entire region of the image.

13. An image processing method comprising:

5. Computer, An image acquisition means for acquiring image data of an image including a human being; a color correction means for correcting a hue of an image region occupied by human skin in said image to a value indicating a human skin hue regardless of race or sex; an output means for outputting the corrected image to a display unit; and make it work. the value indicating the human skin hue is any value between 15° and 25° of the hue in the HSL color space, The color correction means shifts and corrects a value indicating a hue of an image region occupied by a human skin part in the image to a value indicating the human skin hue, and also corrects the hue of the entire region of the image.

2. An image processing program comprising:

6. An image display device that displays an image including a human on a display unit, An image acquisition means for acquiring image data of an image including a human being; a color correction means for correcting a hue of an image region occupied by human skin in said image to a value indicating a human skin hue regardless of race or sex; an output means for outputting the corrected image to a display unit; Equipped with the value indicating the human skin hue is any value between 15° and 25° of the hue in the HSL color space, The color correction means shifts and corrects a value indicating a hue of an image region occupied by a human skin part in the image to a value indicating the human skin hue, and also corrects the hue of the entire region of the image.

1. An image display device comprising:

7. 1. An imaging device, comprising: An imaging unit; an image acquisition means for acquiring image data of an image including a human being captured by the imaging unit; a color correction means for correcting a hue of an image region occupied by human skin in said image to a value indicating a human skin hue regardless of race or sex; an output means for outputting the corrected image; Equipped with the value indicating the human skin hue is any value between 15° and 25° of the hue in the HSL color space, The color correction means shifts and corrects a value indicating a hue of an image region occupied by a human skin part in the image to a value indicating the human skin hue, and also corrects the hue of the entire region of the image.

1. An imaging device comprising:

Images