Information processing system, information processing apparatus, information processing method, and recording medium

Abstract

An information processing system (10) comprises: an acquiring means (110) for acquiring skin color information, which is information related to the skin color of a subject; an estimating means (120) for estimating luminance information of the skin of the subject, on the basis of the skin color information; and a control means (130) for controlling, on the basis of the luminance information, at least one of a camera parameter related to the exposure when imaging the subject and illumination (19). According to such information processing system, it is possible to image a subject under an appropriate condition.

Description

Technical Field

[0001] This disclosure relates to the technical fields of information processing systems, information processing apparatuses, information processing methods, and recording media.

Background Art

[0002] As this type of system, there is known one that adjusts the exposure and light environment when imaging a living body image. For example, Patent Document 1 discloses a technique for determining the amount of light of illumination based on the relationship between illuminance and pupil size. Patent Document 2 discloses a technique for setting the exposure time when imaging an iris image.

[0003] As other related techniques, for example, Patent Document 3 discloses a technique for detecting a pupil region and acquiring luminance information of a skin region around the pupil region.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] This disclosure aims to improve the techniques disclosed in the prior art documents.

Means for Solving the Problems

[0006] One aspect of the information processing system of this disclosure includes an acquisition unit that acquires skin color information, which is information regarding the skin color of a subject; an estimation unit that estimates luminance information of the skin of the subject based on the skin color information; and a control unit that controls at least one of camera parameters and illumination related to exposure when imaging the subject based on the luminance information.

[0007] One aspect of the information processing apparatus of this disclosure includes an acquisition unit that acquires skin color information, which is information regarding the skin color of a subject; an estimation unit that estimates luminance information of the skin of the subject based on the skin color information; and a control unit that controls at least one of camera parameters and illumination related to exposure when imaging the subject based on the luminance information.

[0008] One aspect of the information processing method of this disclosure is an information processing method executed by at least one computer, which includes acquiring skin color information, which is information regarding the skin color of a subject, estimating luminance information of the skin of the subject based on the skin color information, and controlling at least one of camera parameters and illumination related to exposure when imaging the subject based on the luminance information.

[0009] One aspect of the recording medium of this disclosure has recorded thereon a computer program that causes at least one computer to execute an information processing method including acquiring skin color information, which is information regarding the skin color of a subject, estimating luminance information of the skin of the subject based on the skin color information, and controlling at least one of camera parameters and illumination related to exposure when imaging the subject based on the luminance information.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of an information processing system, an information processing apparatus, an information processing method, and a recording medium will be described with reference to the drawings.

[0012] <First Embodiment> The information processing system according to the first embodiment will be described with reference to FIGS. 1 to 3.

[0013] (Hardware Configuration) First, with reference to FIG. 1, the hardware configuration of the information processing system according to the first embodiment will be described. FIG. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0014] As shown in FIG. 1, the information processing system 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 10 may further include an input device 15 and an output device 16. Also, the information processing system 10 may include a camera 18 and an illumination 19. The above-described processor 11, RAM 12, ROM 13, storage device 14, input device 15, output device 16, camera 18, and illumination 19 are connected via a data bus 17.

[0015] Processor 11 reads a computer program. For example, processor 11 is configured to read a computer program stored in at least one of RAM 12, ROM 13, and storage device 14. Alternatively, processor 11 may read a computer program stored in a computer-readable recording medium using a recording medium reader (not shown). Processor 11 may obtain (i.e., read) a computer program from a device (not shown) disposed outside information processing system 10 via a network interface. By executing the read computer program, processor 11 controls RAM 12, storage device 14, input device 15, and output device 16. In particular, in this embodiment, when processor 11 executes the read computer program, a functional block for executing control during imaging of an object is realized within processor 11. That is, processor 11 may function as a controller that executes each control in information processing system 10.

[0016] Processor 11 may be configured, for example, as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array), DSP (Demand-Side Platform), or ASIC (Application Specific Integrated Circuit). Processor 11 may be configured of one of these, or may be configured to use a plurality in parallel.

[0017] RAM 12 temporarily stores the computer program executed by processor 11. RAM 12 temporarily stores data temporarily used by processor 11 when processor 11 is executing a computer program. RAM 12 may be, for example, D-RAM (Dynamic RAM).

[0018] The ROM 13 stores the computer programs executed by the processor 11. The ROM 13 may also store other fixed data. The ROM 13 may be, for example, a P-ROM (Programmable ROM).

[0019] The storage device 14 stores the data that the information processing system 10 stores in the long term. The storage device 14 may operate as a temporary storage device of the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0020] The input device 15 is a device that receives input instructions from the user of the information processing system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as a portable terminal such as a smartphone or a tablet.

[0021] The output device 16 is a device that outputs information regarding the information processing system 10 to the outside. For example, the output device 16 may be a display device (e.g., a display) capable of displaying information regarding the information processing system 10. Also, the output device 16 may be a speaker or the like capable of outputting information regarding the information processing system 10 as sound. The output device 16 may be configured as a portable terminal such as a smartphone or a tablet.

[0022] The camera 18 is a camera installed at a location where it can capture an image of a target (for example, an image including the face of the target or an image including the iris of the target). Here, the target is not limited to humans, and may include animals such as dogs and snakes, robots, etc. The camera 18 may be a camera mounted on a terminal (for example, a smartphone) that the target has. The camera 18 may be a camera that captures still images or a camera that captures moving images. The camera 18 may be configured as a visible light camera or may be configured as a near-infrared camera. A plurality of cameras 18 may be provided. When a plurality of cameras 18 are provided, they may be provided as different types of cameras 18. For example, a visible light camera that captures the face of the target and a near-infrared camera that captures the iris of the target may be provided. Also, the camera 18 may be provided as a depth camera or a thermal camera.

[0023] The illumination 19 is configured to be able to irradiate illumination light to the target when capturing an image with the camera 18. For example, when the camera 18 is provided as a visible light camera, the illumination 19 may be configured to irradiate visible light. Also, when the camera 18 is provided as a near-infrared camera, the illumination 19 may be configured to irradiate near-infrared light. A plurality of illuminations 19 may be provided. When a plurality of illuminations 19 are provided, they may be provided as different types of illuminations 19.

[0024] Note that in FIG. 1, an example of the information processing system 10 configured to include a plurality of devices is given, but all or some of these functions may be realized by one device (information processing device). This information processing device is configured to include, for example, only the above-described processor 11, RAM 12, and ROM 13, and for other components (that is, the storage device 14, the input device 15, the output device 16, and the camera 18), for example, an external device connected to the information processing device may be provided with them. Also, the information processing device may be one in which some of the arithmetic functions are realized by an external device (for example, an external server or the cloud, etc.).

[0025] (Functional configuration) Next, while referring to FIG. 2, the functional configuration of the information processing system 10 according to the first embodiment will be described. FIG. 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0026] As shown in FIG. 2, the information processing system 10 according to the first embodiment includes a skin color information acquisition unit 110, a luminance information estimation unit 120, and a control unit 130 as components for realizing its functions. Each of the skin color information acquisition unit 110, the luminance information estimation unit 120, and the control unit 130 may be a processing block realized by, for example, the above-described processor 11 (see FIG. 1).

[0027] The skin color information acquisition unit 110 is configured to be able to acquire skin color information regarding the skin color of the target (i.e., the imaging target by the camera 18). The skin color information acquisition unit 110 may acquire skin color information from an image of the target. For example, the skin color information acquisition unit 110 may acquire skin color information from the face of the target included in the visible light image. In this case, the skin color information acquisition unit 110 may have a function of detecting a region where the face of the target exists (i.e., the face region). Further, the skin color information acquisition unit 110 may detect a partial region of the face of the target and acquire the skin color information of that region. For example, the skin color information acquisition unit 110 may detect the cheek region or the forehead region of the target and acquire the skin color information. In this way, it is possible to prevent misdetection of the color of the hair or mouth portion of the target. Also, the skin color information acquisition unit 110 may detect the eyes and acquire the skin color information from the skin region around the eyes. The skin color information may be acquired, for example, as RGB values. The skin color information may be acquired as the average RGB value of a predetermined region (e.g., the face region) of the target. The skin color information acquired by the skin color information acquisition unit 110 is configured to be output to the luminance information estimation unit 120.

[0028] The luminance information estimation unit 120 is configured to be able to estimate the luminance information of the target skin (for example, the luminance information of the skin when the target is imaged) from the skin color information acquired by the skin color information acquisition unit 110. That is, the luminance information estimation unit 120 is configured to be able to estimate the luminance information in the case where it is assumed that imaging will be performed in the future without actually imaging the target. The luminance information may be, for example, information including pixel values or luminance values. The luminance information may be, for example, the luminance information when imaging with a visible light camera, or the luminance information when imaging with a near-infrared camera. Specific methods for estimating the luminance information from the skin color information will be described in detail in other embodiments described later. The luminance information estimated by the luminance information estimation unit 120 is configured to be output to the control unit 130.

[0029] The control unit 130 is configured to be able to control camera parameters related to exposure when imaging the target based on the luminance information estimated by the luminance information estimation unit 120. The camera parameters may include, for example, at least one of the aperture of the lens, the exposure time, and the gain. The control unit 130 may control one type of camera parameter or a plurality of types of camera parameters. The control unit 130 may control the camera parameters so that the luminance information of the target when imaged with the camera 18 becomes an appropriate value based on the estimated luminance information. For example, when the estimated luminance information is too large (i.e., too bright), the control unit 130 may control the camera parameters so that the luminance information becomes smaller. For example, the control unit 130 may reduce the aperture value, shorten the exposure time, or reduce the gain. Alternatively, when the estimated luminance information is too small (i.e., too dark), the control unit 130 may control the camera parameters so that the luminance information becomes larger. For example, the control unit 130 may increase the aperture value, lengthen the exposure time, or increase the gain.

[0030] (Flow of operations) Next, with reference to FIG. 3, the flow of operations by the information processing system 10 according to the first embodiment will be described. FIG. 3 is a flowchart showing the flow of operations by the information processing system according to the first embodiment.

[0031] As shown in FIG. 3, when the information processing system 10 according to the first embodiment operates, first, the skin color information acquisition unit 110 acquires the target skin color information (step S101). When the skin color information acquisition unit 110 acquires the skin color information from the target image, a process of imaging the target image (for example, a process of imaging the target face image) may be executed before this step.

[0032] Subsequently, the luminance information estimation unit 120 estimates the luminance information of the target skin based on the skin color information acquired by the skin color information acquisition unit 110 (step S102). Then, the control unit 130 controls the camera parameters related to the exposure when imaging the target based on the luminance information estimated by the luminance information estimation unit 120 (step S103).

[0033] Note that after the control of the camera parameters by the control unit 130, the actual imaging of the target may be executed. That is, the target may be imaged by the camera 18 whose camera parameters are controlled. In this case, the control unit 130 may have a function of outputting an imaging instruction to the camera after the control of the camera parameters. The image captured by the camera 18 may be used for, for example, biometric authentication. In this case, the biometric authentication may be a multimodal authentication that performs authentication at a plurality of parts. For example, face authentication using the face image captured when acquiring the skin color information and iris authentication using the iris image captured after the control may be executed.

[0034] (Technical Effect) Next, the technical effect obtained by the information processing system 10 according to the first embodiment will be described.

[0035] As described with reference to FIGS. 1 to 3, in the information processing system 10 according to the first embodiment, camera parameters related to exposure when imaging an object are controlled based on the luminance information estimated from the skin color information of the object. By doing so, it becomes possible to image the object under appropriate conditions. Therefore, for example, it is possible to prevent whiteout due to saturation of pixel values. Further, in this embodiment, in particular, since the camera parameters are controlled without actually imaging the object (i.e., by estimating the luminance information), it is possible to capture an appropriate image while avoiding imaging delay.

[0036] <Second Embodiment> The information processing system 10 according to the second embodiment will be described with reference to FIG. 4. Note that the second embodiment is only different from the above-described first embodiment in some configurations and operations, and the other parts may be the same as those of the first embodiment. Therefore, hereinafter, the parts different from the already-described first embodiment will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0037] (Configuration of Control Unit) The control unit 130 according to the second embodiment is configured to be able to control the illumination 19 when imaging the object based on the luminance information estimated by the luminance information estimation unit 120. Specifically, the control unit 130 may be configured to be able to control the light amount, direction, or lighting timing of the illumination 19. When a plurality of illuminations 19 are provided, the control unit 130 may be configured to be able to control each of the plurality of illuminations 19. The control unit 130 may control the illumination 19 so that the luminance information of the object when imaged by the camera 18 becomes an appropriate value based on the estimated luminance information. For example, when the estimated luminance information is too large (i.e., too bright), the control unit 130 may control the illumination 19 so that the luminance information becomes smaller. For example, the control unit 130 may control to reduce the light amount of the illumination 19, deflect the direction of the illumination 19 from the object, or control to shift the lighting timing from the imaging timing. Or, when the estimated luminance information is too small (i.e., too dark), the control unit 130 may control the illumination 19 so that the luminance information becomes larger. For example, the control unit 130 may control to increase the light amount of the illumination 19, direct the illumination 19 toward the object side, or control to bring the lighting timing closer to the imaging timing.

[0038] (Flow of operations) First, with reference to FIG. 4, the flow of operations by the information processing system 10 according to the second embodiment will be described. FIG. 4 is a flowchart showing the flow of operations by the information processing system according to the second embodiment. In FIG. 4, the same reference numerals are given to the same processes as those described in FIG. 3.

[0039] As shown in FIG. 4, when the information processing system 10 according to the second embodiment operates, first, the skin color information acquisition unit 110 acquires the skin color information of the object (step S101).

[0040] Subsequently, the luminance information estimation unit 120 estimates the luminance information of the skin of the object based on the skin color information acquired by the skin color information acquisition unit 110 (step S102). Then, the control unit 130 controls the illumination 19 when imaging the object based on the luminance information estimated by the luminance information estimation unit 120 (step S201).

[0041] (Technical effect) Next, the technical effect obtained by the information processing system 10 according to the second embodiment will be described.

[0042] As described with reference to FIG. 4, in the information processing system 10 according to the second embodiment, the illumination 19 at the time of imaging the object is controlled based on the luminance information estimated from the skin color information of the object. By doing so, it becomes possible to image the object under appropriate conditions. Therefore, for example, it is possible to prevent white bleeding and the like caused by saturation of pixel values. Further, in this embodiment, in particular, since the illumination 19 is controlled without actually imaging the object (that is, by estimating the luminance information), it is possible to capture an appropriate image while avoiding the delay of imaging.

[0043] <Third Embodiment> The information processing system 10 according to the third embodiment will be described with reference to FIG. 5. Note that the third embodiment is different from the first and second embodiments described above in some configurations and operations, and the other parts may be the same as those of the first and second embodiments. For this reason, hereinafter, the parts different from the embodiments already described will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0044] (Configuration of the control unit) The control unit 130 according to the third embodiment is configured to be able to control the camera parameters and the illumination 19 related to the exposure when imaging the object based on the luminance information estimated by the luminance information estimation unit 120. That is, the control unit 130 according to the third embodiment is configured to be able to execute both the control of the camera parameters described in the first embodiment and the control of the illumination 19 described in the second embodiment. Note that the control unit 130 may be configured to be able to execute the cooperative control of the camera parameters and the illumination 19. Specifically, the control unit 130 may execute control to compensate for the shortage caused by the control of one of the camera parameters and the illumination 19 with the control of the other. For example, the control unit 130 may control the camera parameters so that the luminance information becomes smaller, and further control the illumination 19 so that the luminance information becomes smaller. Alternatively, the control unit 130 may execute control to offset the excess caused by the control of one of the camera parameters and the illumination 19 with the control of the other. For example, the control unit 130 may control the camera parameters so that the luminance information becomes smaller while controlling the illumination 19 so that the luminance information becomes slightly larger.

[0045] (Flow of operations) First, with reference to FIG. 5, the flow of operations by the information processing system 10 according to the third embodiment will be described. FIG. 5 is a flowchart showing the flow of operations by the information processing system according to the third embodiment. Note that in FIG. 5, the same reference numerals are given to the same processes as those shown in FIG. 3.

[0046] As shown in FIG. 5, when the information processing system 10 according to the third embodiment operates, first, the skin color information acquisition unit 110 acquires the skin color information of the object (step S101).

[0047] Subsequently, the luminance information estimation unit 120 estimates the luminance information of the skin of the object based on the skin color information acquired by the skin color information acquisition unit 110 (step S102). Then, the control unit 130 controls the camera parameters and the illumination 19 when imaging the object based on the luminance information estimated by the luminance information estimation unit 120 (step S301).

[0048] (Technical Effect) Next, the technical effect obtained by the information processing system 10 according to the third embodiment will be described.

[0049] As described with reference to FIG. 5, in the information processing system 10 according to the third embodiment, the camera parameters and the illumination 19 when imaging the subject are controlled based on the luminance information estimated from the skin color information of the subject. By doing so, it becomes possible to image the subject under appropriate conditions. Therefore, for example, it is possible to prevent whiteout due to saturation of pixel values. Further, in this embodiment, in particular, since the camera parameters and the illumination 19 are controlled without actually imaging the subject (that is, by estimating the luminance information), it is possible to capture an appropriate image while avoiding imaging delay. And since both the camera parameters and the illumination 19 are controlled, a higher degree of freedom of control can be performed compared to the case where only one of them is controlled.

[0050] <Fourth Embodiment> The information processing system 10 according to the fourth embodiment will be described with reference to FIGS. 6 and 7. Note that the fourth embodiment is only different from the first to third embodiments described above in some configurations and operations, and the other parts may be the same as those of the first to third embodiments. Therefore, hereinafter, the parts different from the embodiments already described will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0051] (Functional Configuration) First, with reference to FIG. 6, the functional configuration of the information processing system 10 according to the fourth embodiment will be described. FIG. 6 is a block diagram showing the functional configuration of the information processing system according to the fourth embodiment. Note that in FIG. 6, the same reference numerals are given to the elements similar to those shown in FIG. 2.

[0052] As shown in FIG. 6, the information processing system 10 according to the fourth embodiment includes, as components for realizing its functions, a skin color information acquisition unit 110, a luminance information estimation unit 120, and a control unit 130. In particular, the control unit 130 according to the fourth embodiment includes a threshold value storage unit 131.

[0053] The threshold value storage unit 131 is configured to be able to store a first threshold value and a second threshold value used when controlling at least one of the camera parameters and the illumination 19. The first threshold value and the second threshold value are threshold values indicating the upper limit and the lower limit of the target value of the luminance information. The control unit 130 according to the fourth embodiment is configured to be able to control at least one of the camera parameters and the illumination 19 so that the luminance information is between the first threshold value and the second threshold value. The first threshold value and the second threshold value may be values calculated by a prior simulation or the like. For example, the first threshold value and the second threshold value may be set as values such that the pixel values of the target image do not saturate. Note that a plurality of sets of the first threshold value and the second threshold value may be prepared. In this case, the first threshold value and the second threshold value actually used may be changed (selected) according to the skin color information and the luminance information of the target. For example, if the luminance information of the target is within a predetermined first range, a pair of the first threshold value A and the second threshold value A may be selected, and if the luminance information of the target is within a predetermined second range, a pair of the second threshold value B and the second threshold value B may be selected. Alternatively, the first threshold value and the second threshold value may be values that can be appropriately set by the user (for example, values input by the user's operation). Alternatively, the first threshold value and the second threshold value may be values appropriately calculated based on information about the imaging environment, information about the target, and the like. In this case, the threshold value storage unit 131 may be configured to have a function of calculating the first threshold value and the second threshold value.

[0054] (Flow of operations) Next, with reference to FIG. 7, the flow of operations of the information processing system 10 according to the fourth embodiment will be described. FIG. 7 is a flowchart showing the flow of operations of the information processing system according to the fourth embodiment. In FIG. 7, the same reference numerals are given to the same processes as those shown in FIG. 3.

[0055] As shown in FIG. 7, when the information processing system 10 according to the fourth embodiment operates, first, the skin color information acquisition unit 110 acquires the target skin color information (step S101). Then, the luminance information estimation unit 120 estimates the luminance information of the target skin based on the skin color information acquired by the skin color information acquisition unit 110 (step S102).

[0056] Subsequently, the control unit 130 reads the first threshold value and the second threshold value from the threshold value storage unit 131 (step S401). Then, the control unit 130 controls at least one of the camera parameters and the illumination 19 when imaging the target so that the luminance information of the target skin is between the first threshold value and the second threshold value (step S402).

[0057] (Technical Effect) Next, the technical effect obtained by the information processing system 10 according to the fourth embodiment will be described.

[0058] As described with reference to FIGS. 6 and 7, in the information processing system 10 according to the fourth embodiment, at least one of the camera parameters and the illumination 19 when imaging the target is controlled so that the luminance information of the target skin is between the first threshold value and the second threshold value. In this way, it becomes possible to control at least one of the camera parameters and the illumination 19 more easily and accurately.

[0059] <Fifth Embodiment> The information processing system 10 according to the fifth embodiment will be described with reference to FIGS. 8 and 9. Note that the fifth embodiment is only different from the first to fourth embodiments described above in some configurations and operations, and the other parts may be the same as those of the first to fourth embodiments. Therefore, hereinafter, the parts different from the embodiments already described will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0060] (Functional Configuration) First, while referring to FIG. 8, the functional configuration of the information processing system 10 according to the fifth embodiment will be described. FIG. 8 is a block diagram showing the functional configuration of the information processing system according to the fifth embodiment. In FIG. 8, the same reference numerals are given to the same elements as those shown in FIG. 2.

[0061] As shown in FIG. 8, the information processing system 10 according to the fifth embodiment includes a skin color information acquisition unit 110, a luminance information estimation unit 120, and a control unit 130 as components for realizing its functions. In particular, the luminance information estimation unit 120 according to the fifth embodiment includes a reflectance acquisition unit 121.

[0062] The reflectance acquisition unit 121 is configured to be able to acquire the skin reflectance corresponding to the skin color information acquired by the skin color information acquisition unit 110. The "skin reflectance" here indicates the ratio of the light irradiated on the skin to the light reflected from the skin. The skin reflectance may be acquired, for example, for each wavelength of the irradiated light. Specifically, in addition to the skin reflectance in the visible light region, the skin reflectance in the near-infrared region may be acquired. A more specific method for acquiring the skin reflectance will be described in detail in other embodiments described later. Note that the luminance information estimation unit 120 according to the fifth embodiment is configured to be able to estimate the luminance information of the target skin based on the skin reflectance acquired by the reflectance acquisition unit 121.

[0063] (Flow of operations) Next, while referring to FIG. 9, the flow of operations by the information processing system 10 according to the fifth embodiment will be described. FIG. 9 is a flowchart showing the flow of operations by the information processing system according to the fifth embodiment. In FIG. 9, the same reference numerals are given to the same processes as those shown in FIG. 3.

[0064] As shown in FIG. 9, when the information processing system 10 according to the fifth embodiment operates, first, the skin color information acquisition unit 110 acquires the target skin color information (step S101). Then, the reflectance acquisition unit 121 acquires the skin reflectance of the target based on the skin color information acquired by the skin color information acquisition unit 110 (step S501).

[0065] Subsequently, the luminance information estimation unit 120 estimates the luminance information of the target skin based on the skin reflectance acquired by the reflectance acquisition unit 121 (step S502). Then, the control unit 130 controls at least one of the camera parameters and the illumination 19 when imaging the target based on the luminance information estimated by the luminance information estimation unit 120 (step S503).

[0066] (Technical Effect) Next, the technical effect obtained by the information processing system 10 according to the fifth embodiment will be described.

[0067] As described with reference to FIGS. 8 and 9, in the information processing system 10 according to the fifth embodiment, the luminance information of the target skin is estimated based on the skin reflectance. In this way, the luminance information of the target skin can be accurately estimated. Therefore, it becomes possible to more appropriately control at least one of the camera parameters and the illumination 19.

[0068] <Sixth Embodiment> The information processing system 10 according to the sixth embodiment will be described with reference to FIGS. 10 to 12. Note that the sixth embodiment is only different from the above-described fifth embodiment in some configurations and operations, and the other parts may be the same as those of the first to fifth embodiments. For this reason, in the following, the parts different from the embodiments already described will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0069] (Functional Configuration) First, the functional configuration of the information processing system 10 according to the sixth embodiment will be described with reference to FIG. 10. FIG. 10 is a block diagram showing the functional configuration of the information processing system according to the sixth embodiment. In FIG. 10, the same reference numerals are given to the same elements as those shown in FIG. 8.

[0070] As shown in FIG. 10, an information processing system 10 according to the sixth embodiment includes, as components for realizing its functions, a skin color information acquisition unit 110, a luminance information estimation unit 120, a control unit 130, and a reflectance model storage unit 140. That is, the information processing system 10 according to the sixth embodiment further includes a reflectance model storage unit 140 in addition to the configuration of the fifth embodiment (see FIG. 8). The reflectance model storage unit 140 may be realized by, for example, the above-described storage device 14 (see FIG. 1).

[0071] The reflectance model storage unit 140 is configured to be able to store a reflectance model that calculates the skin reflectance from the skin color information acquired by the skin color information acquisition unit 110. The reflectance model may be, for example, pre-generated based on data acquired from a plurality of living bodies. Also, the reflectance model may be a model generated by machine learning. Specific examples of the reflectance model will be described in detail later. Note that the reflectance acquisition unit 121 according to the sixth embodiment is configured to be able to calculate the skin reflectance using the reflectance model read from the reflectance model storage unit 140.

[0072] (Flow of operations) Next, with reference to FIG. 11, the flow of operations of the information processing system 10 according to the sixth embodiment will be described. FIG. 11 is a flowchart showing the flow of operations of the information processing system according to the sixth embodiment. In FIG. 11, the same reference numerals are given to the same processes as those shown in FIG. 9.

[0073] As shown in FIG. 11, when the information processing system 10 according to the sixth embodiment operates, first, the skin color information acquisition unit 110 acquires the target skin color information (step S101). Then, the reflectance acquisition unit 121 calculates the reflectance of the target skin using the reflectance model stored in the reflectance model storage unit 140 (step S601). That is, the reflectance acquisition unit 121 calculates the reflectance of the target skin based on the skin color information and the reflectance model.

[0074] Subsequently, the luminance information estimation unit 120 estimates the luminance information of the target skin based on the skin reflectance calculated by the reflectance acquisition unit 121 (step S502). Then, the control unit 130 controls at least one of the camera parameters and the illumination 19 when imaging the target based on the luminance information estimated by the luminance information estimation unit 120 (step S503).

[0075] (Example of reflectance calculation) Next, with reference to FIG. 12, a calculation example of the skin reflectance by the information processing system 10 according to the sixth embodiment (that is, a calculation example using a reflectance model) will be specifically described. FIG. 12 is a graph showing an example of the skin reflectance calculated by the information processing system according to the sixth embodiment. Note that the vertical axis of the graph shown in FIG. 12 is "skin reflectance", and the horizontal axis is "(light) wavelength".

[0076] First, assume that the skin color information acquisition unit 110 acquires the average RGB value from the target face image (visible light image) as the skin color information. When the RGB value is gamma-corrected, the RGB value when γ = 1.0 may be calculated by de-gamma processing.

[0077] The reflectance acquisition unit 121 calculates the tristimulus values XYZ from the above-described RGB values. The tristimulus values XYZ can be calculated using, for example, the following equation (1).

[0078] [Equation] Note that the conversion matrix aij may use a matrix defined in the standard RGB space sRGB.

[0079] Subsequently, the reflectance acquisition unit 121 estimates the skin reflectance S(λ) from the calculated tristimulus values XYZ. Specifically, the characteristic parameters (a, b, c) are calculated using the following equation (2).

[0080] [Equation] Note that in the above formula (2), si(λ) (i = 1 to 3) are basis vectors, s0(λ) is the average vector, x(λ), y(λ), z(λ) are metameric functions, and d65(λ) is the spectral distribution of the standard white D65, all of which are known information.

[0081] Subsequently, the reflectance acquisition unit 121 obtains the skin reflectance by substituting the calculated characteristic parameters (a, b, c) into the reflectance model. Specifically, the reflectance S(λ) is obtained by substituting the characteristic parameters (a, b, c) into the following formula (3). The skin reflectance S(λ) may be obtained from the visible light region to the near-infrared light region as shown in FIG. 12.

[0082] (Technical Effect) Next, the technical effect obtained by the information processing system 10 according to the sixth embodiment will be described.

[0083] As described with reference to FIGS. 10 to 12, in the information processing system 10 according to the sixth embodiment, the reflectance of the target skin is calculated using the reflectance model. In this way, the skin reflectance can be calculated easily and accurately. Therefore, the luminance information of the skin can be estimated accurately, and it is possible to control at least one of the camera parameters and the illumination 19 more appropriately. In particular, in the information processing system 10 according to the sixth embodiment, in multimodal authentication using the face and iris, when the camera for photographing the face captures in visible light and the camera for photographing the iris captures in near-infrared light, the luminance information of the skin in the near-infrared light region can be obtained from the skin color information acquired from the face image captured in the visible light region, and based on this luminance information in the near-infrared light region, at least one of the camera parameters and the illumination of the camera for photographing the iris can be controlled appropriately.

[0084] <Seventh Embodiment> The information processing system 10 according to the seventh embodiment will be described with reference to FIGS. 13 and 14. Note that the seventh embodiment is only different from the first to sixth embodiments described above in some configurations and operations, and the other parts may be the same as those of the first to sixth embodiments. For this reason, in the following, the parts different from the embodiments already described will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0085] (Functional configuration) First, with reference to FIG. 13, the functional configuration of the information processing system 10 according to the seventh embodiment will be described. FIG. 13 is a block diagram showing the functional configuration of the information processing system according to the seventh embodiment. In FIG. 13, the same reference numerals are given to the elements similar to those shown in FIG. 2.

[0086] As shown in FIG. 13, the information processing system 10 according to the seventh embodiment includes, as components for realizing its functions, a skin color information acquisition unit 110, a luminance information estimation unit 120, a control unit 130, and a movement amount detection unit 150. That is, the information processing system 10 according to the seventh embodiment further includes a movement amount detection unit 150 in addition to the configuration of the first embodiment (see FIG. 2).

[0087] The movement amount detection unit 150 is configured to be able to detect the movement amount of the target. More specifically, the movement amount detection unit 150 may be configured to be able to detect the movement amount of the target per unit time (in other words, the movement speed). The movement amount detection unit 150 may be configured to detect the movement amount of the target using various sensors. Alternatively, the movement amount detection unit 150 may capture a plurality of images of the target and detect the movement amount of the target based on the difference between those images (specifically, the change in the position of the target in the image). The movement amount of the target detected by the movement amount detection unit 150 is output to the control unit 130.

[0088] The control unit 130 according to the seventh embodiment is configured to be able to control at least one of the camera parameters and the illumination 19 based on the movement amount detected by the movement amount detection unit 150 in addition to the luminance information estimated by the luminance information estimation unit 120. For example, the control unit 130 may change the control amount according to the detected movement amount of the object. That is, the control amount of the camera parameters or the illumination 19 may be increased or decreased according to the magnitude of the movement amount of the object. Further, the control unit 130 may change the type of parameter to be controlled according to the detected movement amount of the object. For example, when the movement amount of the object is large, it is not preferable to increase the shutter speed (exposure time) (there is a risk of blurring), so other parameters may be controlled. Also, when the movement amount of the object is large, the control may be performed such that the shutter speed is shortened and the light amount of the illumination 19 is increased. Further, when the movement amount of the object is small, the control may be performed such that the shutter speed is increased and the light amount of the illumination 19 is decreased.

[0089] (Flow of operations) Next, with reference to FIG. 14, the flow of operations of the information processing system 10 according to the seventh embodiment will be described. FIG. 14 is a flowchart showing the flow of operations of the information processing system according to the seventh embodiment. In FIG. 14, the same reference numerals are given to the same processes as those shown in FIG. 3.

[0090] As shown in FIG. 14, when the information processing system 10 according to the seventh embodiment operates, first, the skin color information acquisition unit 110 acquires the skin color information of the object (step S101). Then, the luminance information estimation unit 120 estimates the luminance information of the skin of the object based on the skin color information acquired by the skin color information acquisition unit 110 (step S102).

[0091] Subsequently, the movement amount detection unit 150 detects the movement amount of the object (step S701). The detection of the movement amount may be performed before or after steps S101 and S102 described above, or may be performed simultaneously and in parallel.

[0092] Thereafter, the control unit 130 controls at least one of the camera parameters and the illumination 19 when imaging the object based on the luminance information estimated by the luminance information estimation unit 120 and the movement amount detected by the movement amount detection unit (step S2702).

[0093] (Technical effect) Next, the technical effect obtained by the information processing system 10 according to the seventh embodiment will be described.

[0094] As described with reference to FIGS. 13 and 14, in the information processing system 10 according to the seventh embodiment, at least one of the camera parameters and the illumination 19 when imaging the object is controlled in consideration of the movement amount of the object. In this way, at least one of the camera parameters and the illumination 19 can be controlled more appropriately as compared with the case where the movement amount of the object is not considered.

[0095] <Eighth Embodiment> The information processing system 10 according to the eighth embodiment will be described with reference to FIG. 15. Note that the eighth embodiment is different from the first to seventh embodiments described above only in some operations, and the other parts may be the same as those of the first to seventh embodiments. For this reason, hereinafter, the parts different from the embodiments already described will be described in detail, and the description of the other overlapping parts will be omitted as appropriate.

[0096] (Flow of operations) First, with reference to FIG. 15, the flow of operations by the information processing system 10 according to the eighth embodiment will be described. FIG. 15 is a flowchart showing the flow of operations by the information processing system according to the eighth embodiment. Note that in FIG. 15, the same reference numerals are given to the same processes as those shown in FIG. 3.

[0097] As shown in FIG. 15, when the information processing system 10 according to the eighth embodiment operates, first, the skin color information acquisition unit 110 acquires the target skin color information (step S101). Then, the luminance information estimation unit 120 estimates the luminance information of the target skin based on the skin color information acquired by the skin color information acquisition unit 110 (step S102).

[0098] Subsequently, the control unit 130 controls at least one of the camera parameters and the illumination 19 when imaging the target based on the luminance information estimated by the luminance information estimation unit 120 (step S801). And in particular, in the information processing system 10 according to the present embodiment, it is determined whether or not all imaging to be executed after the control is completed has ended (step S802). That is, it is determined whether to end the imaging of the current target or to continue imaging the current target image. Whether to end the imaging of the target may be determined based on the number of images captured so far (for example, whether the number of captured images has reached a predetermined number) or the position of the target (for example, whether the target has moved outside the imaging range).

[0099] If it is determined that the imaging of the target is to be ended (step S802: YES), the series of operations will end as it is. On the other hand, if it is determined that the imaging of the target is not to be ended (step S802: NO), the processing starts again from step S101. As a result, when capturing a plurality of images of the target, at least one of the camera parameters and the illumination 19 is controlled each time an image is captured. For example, when the target moves closer to the camera 18, as the target approaches, the shutter speed may be controlled to be shortened. Also, in such a case, in order to reduce the influence caused by the shortening of the shutter speed, the illumination 19 may be controlled to gradually become brighter.

[0100] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the eighth embodiment will be described.

[0101] As described with reference to FIG. 15, in the information processing system 10 according to the eighth embodiment, at least one of the camera parameters and the illumination 19 is controlled for each imaging. By doing so, since the control is performed as needed according to the situation, it becomes possible to image the target under more appropriate conditions.

[0102] <Ninth Embodiment> The information processing system 10 according to the ninth embodiment will be described with reference to FIG. 16. Note that the ninth embodiment describes a specific example of the control operation of the eighth embodiment described above, and other parts may be the same as those of the first to seventh embodiments. For this reason, in the following, parts different from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate.

[0103] (Example of Control Operation) First, with reference to FIG. 16, a specific example of the control operation (that is, control of camera parameters and illumination) by the information processing system 10 according to the ninth embodiment will be described. FIG. 16 is a table showing an example of the control operation by the information processing system according to the ninth embodiment.

[0104] As shown in FIG. 16, in the information system 10 according to the ninth embodiment, the control unit 130 controls the exposure time (shutter speed) of the camera 18 and the illumination 19 based on the luminance information estimated by the luminance information estimation unit 120. Specifically, the control unit 130 according to the ninth embodiment is capable of executing a first control of shortening the exposure time and brightening the illumination 19, and a second control of lengthening the exposure time and dimming the illumination 19. In the first control, considering the influence on the image due to shortening the exposure time, the brightness of the illumination 19 may be controlled so that the image does not become too dark. In the second control, considering the influence on the image due to lengthening the exposure time, the brightness of the illumination 19 may be controlled so that the image does not become too bright. Note that the brightness of the illumination 19 may be controlled by adjusting not only the light amount but also the direction and lighting timing of the illumination 19, as described in the other embodiments above.

[0105] The first control and the second control are executed for each imaging. Therefore, when the first control is executed for each imaging, as the imaging progresses, the exposure time gradually becomes shorter while the illumination 19 is controlled to gradually become brighter. Also, when the second control is executed for each imaging, as the imaging progresses, the exposure time gradually becomes longer while the illumination 19 is controlled to gradually become darker. Note that which of the first control and the second control is to be executed may be changed during the imaging. For example, after the first control is executed, the second control may be executed. Also, after the second control is executed, the first control may be executed. The timing for switching between the first control and the second control may be determined by, for example, the luminance information estimated by the information estimation unit 120.

[0106] (Technical Effect) Next, the technical effect obtained by the information processing system 10 according to the ninth embodiment will be described.

[0107] As described with reference to FIG. 16, in the information processing system 10 according to the ninth embodiment, based on the luminance information, the exposure time of the camera 18 and the brightness of the illumination 19 are controlled. By doing so, since cooperative control of the camera parameters and the illumination 19 is performed, it becomes possible to image the subject under more appropriate conditions.

[0108] A program that operates the configuration of the embodiment so as to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and a processing method executed by a computer is also included in the scope of each embodiment. That is, a computer-readable recording medium is also included in the scope of each embodiment. Also, of course, the recording medium on which the above program is recorded and the program itself are included in each embodiment.

[0109] As the recording medium, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a non-volatile memory card, or a ROM can be used. Further, not limited to those that execute processing with only the program recorded on the recording medium, those that operate on the OS and execute processing in cooperation with the functions of other software and expansion boards are also included in the scope of each embodiment. Furthermore, the program itself may be stored in a server so that a part or all of the program can be downloaded from the server to the user terminal.

[0110] <Supplementary Note> Regarding the embodiments described above, it can be further described as follows in the supplementary note below, but is not limited thereto.

[0111] (Supplementary Note 1) The information processing system described in Supplementary Note 1 includes an acquisition means for acquiring skin color information, which is information regarding the skin color of the target, an estimation means for estimating the luminance information of the skin of the target based on the skin color information, and a control means for controlling at least one of the camera parameters and the lighting regarding the exposure when imaging the target based on the luminance information.

[0112] (Supplementary Note 2) In the information processing system described in Supplementary Note 2, the control means controls at least one of the camera parameters and the lighting so that the luminance information of the skin of the target is between a first threshold value and a second threshold value. It is the information processing system described in Supplementary Note 1.

[0113] (Supplementary Note 3) In the information processing system described in Supplementary Note 3, the estimation means estimates the luminance information based on the reflectance of the skin of the target corresponding to the skin color information. It is the information processing system described in Supplementary Note 1 or 2.

[0114] (Supplementary Note 4) The information processing system described in Supplementary Note 4 further includes storage means for storing a reflectance model that calculates the reflectance of the skin from the skin color information, and the estimation means estimates the luminance information based on the reflectance of the skin calculated using the reflectance model, which is the information processing system described in Supplementary Note 3.

[0115] (Supplementary Note 5) The information processing system described in Supplementary Note 5 further includes detection means for detecting the amount of movement of the object, and the control means controls at least one of the camera parameters and the illumination based on the luminance information and the amount of movement, which is the information processing system described in any one of Supplementary Notes 1 to 4.

[0116] (Supplementary Note 6) The information processing system described in Supplementary Note 6 is such that when the control means captures an image of the object a plurality of times, it controls at least one of the camera parameters and the illumination for each imaging, which is the information processing system described in any one of Supplementary Notes 1 to 5.

[0117] (Supplementary Note 7) The information processing system described in Supplementary Note 7 is such that the control means performs first control to shorten the exposure time and brighten the illumination when imaging the object for each imaging based on the luminance information, or second control to lengthen the exposure time and darken the illumination when imaging the object for each imaging, which is the information processing system described in Supplementary Note 6.

[0118] (Supplementary Note 8) The information processing apparatus described in Supplementary Note 8 includes acquisition means for acquiring skin color information which is information regarding the color of the skin of an object, estimation means for estimating the luminance information of the skin of the object based on the skin color information, and control means for controlling at least one of the camera parameters related to the exposure and the illumination when imaging the object based on the luminance information.

[0119] (Supplementary Note 9) The information processing method described in Supplementary Note 9 is an information processing method executed by at least one computer, which acquires skin color information that is information regarding the skin color of a subject, estimates the luminance information of the skin of the subject based on the skin color information, and controls at least one of a camera parameter and illumination related to exposure when imaging the subject based on the luminance information.

[0120] (Supplementary Note 10) The recording medium described in Supplementary Note 10 is a recording medium on which a computer program is recorded that causes at least one computer to execute an information processing method of acquiring skin color information that is information regarding the skin color of a subject, estimating the luminance information of the skin of the subject based on the skin color information, and controlling at least one of a camera parameter and illumination related to exposure when imaging the subject based on the luminance information.

[0121] (Supplementary Note 11) The computer program described in Supplementary Note 11 is a computer program that causes at least one computer to execute an information processing method of acquiring skin color information that is information regarding the skin color of a subject, estimating the luminance information of the skin of the subject based on the skin color information, and controlling at least one of a camera parameter and illumination related to exposure when imaging the subject based on the luminance information.

[0122] This disclosure can be appropriately changed within a range not contrary to the gist or idea of the invention that can be read from the claims and the entire specification, and an information processing system, an information processing apparatus, an information processing method, and a recording medium involving such changes are also included in the technical idea of this disclosure.

Explanation of Reference Numerals

[0123] 10 Information processing system 11 Processor 18 Camera 19 Illumination 110 Skin color information acquisition unit 120 Luminance information estimation unit 121 Reflectance acquisition unit 130 Control Unit 131 Threshold Memory Unit 140 Reflectance Model Memory Unit 150 Movement Detection Unit

Claims

1. An acquisition means for acquiring skin color information, which is information regarding the color of the skin of an object, from an image captured in the visible light region; A storage means for storing a reflectance model for calculating the reflectance of the skin from the skin color information; An estimation means for estimating the luminance information of the skin of the object in the near-infrared light region based on the reflectance of the skin calculated using the reflectance model; A control means for controlling at least one of a camera parameter and illumination related to exposure when imaging the object in the near-infrared light region based on the luminance information; An information processing system comprising the above.

2. The control means controls at least one of the camera parameter and the illumination so that the luminance information of the skin of the object is between a first threshold value and a second threshold value. The information processing system according to claim 1.

3. The information processing system further comprises a detection means for detecting the amount of movement of the object, The control means controls at least one of the camera parameter and the illumination based on the luminance information and the amount of movement. The information processing system according to claim 1 or 2.

4. When imaging the object a plurality of times, the control means controls at least one of the camera parameter and the illumination for each imaging. The information processing system according to any one of claims 1 to 3.

5. The control means performs a first control of shortening the exposure time and brightening the illumination when imaging the object for each imaging, or a second control of lengthening the exposure time and dimming the illumination when imaging the object for each imaging, based on the luminance information. The information processing system according to claim 4.

6. An acquisition means for acquiring skin color information, which is information regarding the color of the skin of an object, from an image captured in the visible light region; A storage means for storing a reflectance model for calculating the reflectance of the skin from the skin color information; Estimation means for estimating the luminance information of the target skin in the near-infrared light region based on the reflectance of the skin calculated using the reflectance model; Control means for controlling at least one of the camera parameters and illumination related to the exposure when imaging the target in the near-infrared light region based on the luminance information; An information processing apparatus comprising the above.

7. An information processing method executed by at least one computer, comprising: Obtaining skin color information, which is information regarding the color of the target skin, from an image captured in the visible light region; Calculating the reflectance of the skin from the skin color information using a stored reflectance model; Estimating the luminance information of the target skin in the near-infrared light region based on the reflectance of the skin calculated using the reflectance model; Controlling at least one of the camera parameters and illumination related to the exposure when imaging the target in the near-infrared light region based on the luminance information. An information processing method.

8. Causing at least one computer to: Obtain skin color information, which is information regarding the color of the target skin, from an image captured in the visible light region; Calculate the reflectance of the skin from the skin color information using a stored reflectance model; Estimate the luminance information of the target skin in the near-infrared light region based on the reflectance of the skin calculated using the reflectance model; Control at least one of the camera parameters and illumination related to the exposure when imaging the target in the near-infrared light region based on the luminance information. A computer program for causing a computer to execute the information processing method.

Images