System, information processing device, and method for operating system

Abstract

This system comprises: an imaging device that captures an image of an object; an information processing device that detects a target portion in the object on the basis of the captured image of the object; and a projection device that projects colored light to the target portion. The information processing device instructs the color of the colored light to the projection device so that light reflected at the target portion exhibits a target color on the basis of information about a reflected spectrum at the target portion and information about a light source spectrum of the projection device.

Description

System, information processing device and system operation method CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority from Japanese Patent Application No. 2023-141792, filed on August 31, 2023, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to a system, an information processing device, and a method of operating the system.

[0003] Techniques have been proposed for simulating makeup instead of actually applying makeup to a user's face using cosmetics. For example, projection mapping has been proposed, in which colored light simulating makeup is projected onto the user's face using a projection device. Patent Literature 1 discloses a technique related to color correction of colored light in projection mapping.

[0004] Japanese Patent Application Laid-Open No. 2017-147638

[0005] In the technology of projecting colored light onto an object, there is room for improvement in the accuracy of expressing a desired color.

[0006] The following describes an information processing device and the like that can improve the accuracy of color representation of colored light projected onto an object.

[0007] In order to solve the above problems, the system disclosed herein includes an imaging device that images an object, an information processing device that detects a target portion of the object based on the captured image of the object, and a projection device that projects colored light onto the target portion, and the information processing device instructs the projection device on the color of the colored light so that the reflected light at the target portion exhibits a target color, based on information on the reflection spectrum of the target portion and information on the light source spectrum of the projection device.

[0008] The information processing device of the present disclosure is an information processing device having a communication unit and a control unit that communicates via the communication unit, wherein the control unit acquires an image of an object from an imaging device, detects a target portion of the object based on the image, and, based on information on the reflection spectrum of the target portion and information on the light source spectrum of the projection device, instructs the projection device to select a color of colored light that will cause reflected light at the target portion to exhibit a target color, and causes the projection device to project the colored light onto the target portion.

[0009] The system operation method disclosed herein is a method of operation of a system having an imaging device, a projection device, and an information processing device, in which the imaging unit images an object, the information processing device detects a target portion of the object based on the captured image of the object, and based on information on the reflection spectrum of the target portion and information on the light source spectrum of the projection device, instructs the projection device on a color of colored light that will cause reflected light at the target portion to exhibit a target color, and the projection device projects the colored light onto the target portion.

[0010] According to the information processing device and the like of the present disclosure, it is possible to improve the accuracy of color representation of colored light projected onto an object.

[0011] It is a diagram showing an example of the configuration of an information processing system.It is a flowchart showing an example of the operation procedure of an information processing device.It is a diagram showing an example of a target portion.

[0012] Hereinafter, an embodiment of the present invention will be described.

[0013] [System Configuration] FIG. 1 is a diagram illustrating an example configuration of an embodiment of the present invention. The information processing system 1 includes an information processing device 10, an imaging device 12, and a projection device 13, which are communicatively connected to each other via a network 11. The information processing device 10 is, for example, one or more personal computers. The information processing device 10 may also include a tablet terminal device, a smartphone, or the like. The imaging device 12 is a camera installed in a position capable of capturing an image of a user 15. The projection device 13 has a light source and an optical system for emitting colored light and projects a makeup expression onto the face of the user 15 using the colored light. The user 15 is illuminated by ambient light from a lighting device 14. The lighting device 14 is installed in a room, such as a store or studio, where the user 15 performs makeup simulations, and has a light source for emitting ambient light. The network 11 is, for example, a local area network (LAN) within a store, office, or the like. The network 11 may include the Internet, an ad hoc network, a metropolitan area network (MAN), a mobile communication network, or other networks.

[0014] An information processing system 1 supports a simulation of cosmetic makeup on the face of a user 15. The information processing system 1 includes an imaging device 12 that captures an image of an object, an information processing device 10 that detects a target area on the object based on the captured image of the object, and a projection device 13 that projects colored light onto the target area. Here, when the object is the face of the user 15, the target area is, for example, the eyes, cheeks, lips, etc., to which a cosmetic film is to be applied. Since the skin color of the target area varies from user to user and the cosmetic film transmits and reflects light to a degree corresponding to its color, the color exhibited by the target area with the cosmetic film applied (hereinafter referred to as the makeup area), i.e., the spectrum of the light reflected from the makeup area, varies depending on the combination of the skin color of the target area and the color of the cosmetic film. To represent the color of the makeup area, i.e., the target color, the information processing device 10 instructs the projection device 13 on the color of colored light (hereinafter referred to as the adjustment color) that will cause the target area to exhibit the target color, based on the reflection spectrum of the target area and the light source spectrum of the projection device 13. In this way, by determining the adjusted color of the colored light according to the skin color of the target area, the characteristics of the projection device 13, etc., it is possible to improve the accuracy when expressing the target color of different makeup areas using colored light depending on the combination of the skin color of the user 15 and the color of the makeup film.

[0015] Next, the configuration of the information processing device 10 will be described.

[0016] The information processing device 10 has a communication unit 101, a storage unit 102, a control unit 103, an input unit 105, and an output unit 106. When the information processing device 10 is configured with two or more computers that can communicate with each other, these components are appropriately arranged in the two or more computers.

[0017] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, an interface that supports a wired or wireless LAN standard and connects to a nearby router device. The communication interface may include a module that supports short-range wireless communication such as Bluetooth (registered trademark), or a mobile communication standard such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The communication unit 101 receives information used in the operation of the information processing device 10 and transmits information obtained by the operation of the information processing device 10. The information processing device 10 is connected to a network 11 by the communication unit 101 and communicates with other devices via the network 11 or by a direct peer-to-peer connection, etc.

[0018] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these, that function as a main storage device, an auxiliary storage device, or a cache memory. The semiconductor memory is, for example, a random access memory (RAM) or a read-only memory (ROM). The RAM is, for example, a static RAM (SRAM) or a dynamic RAM (DRAM). The ROM is, for example, an electrically erasable programmable read-only memory (EEPROM). The storage unit 102 stores information used in the operation of the control unit 103 and information obtained by the operation of the control unit 103.

[0019] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for specific processing. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 103 executes information processing related to the operation of the information processing device 10 while controlling each unit of the information processing device 10.

[0020] The functions of the information processing device 10 are realized by a processor included in the control unit 103 executing a control program. The control program is a program for causing the processor to function as the control unit 103. Alternatively, some or all of the functions of the information processing device 10 may be realized by a dedicated circuit included in the control unit 103. Alternatively, the control program may be stored in a non-transitory recording / storage medium readable by the control unit 103, and read by the control unit 103 from the medium.

[0021] The input unit 105 includes one or more input interfaces. Examples of the input interface include physical keys, capacitive keys, a pointing device, a touch screen integrated with a display, a microphone for accepting voice input, a camera for capturing captured images, etc. Furthermore, the input interface may include a scanner or camera for scanning image codes, or an IC card reader. The input unit 105 accepts an operator's operation to input information used in the operation of the information processing device 10 and sends the input information to the control unit 103.

[0022] The output unit 106 includes one or more output interfaces. The output interface is, for example, a display or a speaker. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 106 outputs information obtained by the operation of the information processing device 10 to a user, an operator, or the like.

[0023] Next, the configurations of the imaging device 12, the projection device 13, and the lighting device 14 will be described.

[0024] The imaging device 12 includes one or more cameras and a control device for the cameras. The control device includes a processor for controlling the operation of the cameras and a communication module for sending captured images to the information processing device 10 via the network 11 or peer-to-peer. The camera may be, for example, a monocular camera or a stereo camera. In particular, when the purpose is to detect the position of a face, it is preferable to use an infrared camera and a hot mirror, which allows for coaxial placement without affecting the light from the projection device 13. The imaging device 12 captures images of the face of the user 15 using the camera at any frame rate, for example, several hundred frames per second, and sends the captured images to the information processing device. The frame rate is preferably 300 fps or higher, more preferably 400 fps or higher, and even more preferably 500 fps or higher.

[0025] The projection device 13 has one or more light sources, optical systems, and controllers for these. The controller has a processor that controls the operation of the light sources and optical systems, and a communication module for communicating with the information processing device 10 via the network 11 or peer-to-peer. The light source is a lamp, laser, LED, or the like, which emits light including visible light in the range of 380 nm to 780 nm. The optical system is a transmissive liquid crystal system, a reflective liquid crystal system, a DLP system, an RGB-LED system, or the like. The projection device 13 emits colored light in a pattern according to information and instructions received from the information processing device 10 at an arbitrary frame rate, for example, several hundred frames per second. For example, the frame rate is 300 Hz or higher, preferably 400 Hz or higher, and more preferably 500 Hz or higher or 1000 Hz or higher.

[0026] The lighting device 14 is installed on or near the ceiling of a store, studio, etc. The lighting device 14 has, for example, a group of LED elements with different peak wavelengths as a light source, and is configured to be able to irradiate ambient light including visible light by selectively turning on and off the group of LED elements.

[0027] [Operation of Information Processing Device 10] Fig. 2 is a flowchart for explaining an example of the operation of the information processing device 10. The procedure in Fig. 2 is executed by the control unit 103 of the information processing device 10 in response to an operation by an operator when simulating makeup on a user's face using colored light from the projection device 13.

[0028] In step S20, the control unit 103 acquires a captured image of the face of the user 15. The control unit 103 acquires the captured image from the imaging device 12. The captured image is generated by the imaging device 12 at an arbitrary frame rate and sent to the information processing device 10.

[0029] In step S21, the control unit 103 extracts target parts from the captured image of the face of the user 15. The control unit 103 extracts the target parts by any image processing including, for example, pattern matching. As shown in FIG. 3 , target parts such as the eyes 30, cheeks 31, and lips 32 are extracted from the face 33 of the user 15. The control unit 103 also derives the spatial coordinates of each target part.

[0030] In step S22, the control unit 103 acquires information about the reflectance spectrum of the target portion. The reflectance spectrum of the target portion may be the reflectance spectrum for each target portion, or may be a representative reflectance spectrum for the face including the target portion. The reflectance spectrum is, for example, spectral reflectance. For example, an operator measures the spectral reflectance of the target portion or any portion of the face using a color measurement device. The measurement results are input by the operator to the information processing device 10, or are sent from the color measurement device to the information processing device 10 via the network 11 or via a peer-to-peer connection. Alternatively, the operator may visually classify the skin color of the face including the target portion and input classification identification information to the information processing device 10. For example, the storage unit 102 may previously store a table associating skin color classifications with spectral reflectances, and the control unit 103 may use the input classification identification information to read out corresponding spectral reflectance information from the storage unit 102. The number of such classifications may be, for example, 3 to 40. Alternatively, the control unit 103 may derive the spectral reflectance of the target portion using the color of the target portion extracted from the captured image of the user's face, i.e., the gradation values ​​of a color system such as RGB or XYZ in the captured image. For example, the storage unit 102 may previously store a table, model, or the like that associates gradation values ​​of a color system that indicates the color of the target portion with spectral reflectance, and the control unit 103 reads information on the spectral reflectance that corresponds to the color of the target portion from the storage unit 102. Alternatively, the control unit 103 may derive the spectral reflectance from the color of the target portion using an arbitrary algorithm.

[0031] A table or model associating the color of a target area with its spectral reflectance is generated in advance as follows. For example, a colorimeter measures the skin color and spectral reflectance of a user under standard illumination, such as white light, and then performs cluster analysis using hierarchical cluster analysis, k-means, or the like on the data set associating the skin color and spectral reflectance to associate a representative value, such as the average, for each cluster. Such cluster analysis and association of representative values ​​for each cluster are performed by an arbitrary information processing device. A contact-type colorimeter can be used to measure skin color. It is also possible to perform color measurement by projecting standard illumination using a projection device 13. In this case, illuminating the user's face with standard illumination from the front can illuminate the entire face uniformly. Furthermore, when illuminating the user's face from the front, the spectral reflectance varies depending on the direction of the normal to the skin surface. Therefore, an arbitrary information processing device can correct the spectral reflectance according to the angle of the skin surface relative to the front.

[0032] In step S23, the control unit 103 acquires cosmetic film information. The cosmetic film information includes information on the type of cosmetic film to be applied to the target area and the color of the cosmetic film. The types of cosmetic film include eye shadow, blush, lipstick, etc. The cosmetic film color information includes a gradation value of any color system that indicates the color of the cosmetic film and a reflectance spectrum. The reflectance spectrum may include the spectral reflectance, diffusion coefficient, absorption coefficient, etc. of the cosmetic film. The user or operator inputs desired cosmetic film information into the information processing device 10, and the control unit 103 acquires the cosmetic film information. For example, multiple pieces of cosmetic film information are pre-stored in the memory unit 102, and the control unit 103 displays selectable combinations of cosmetic film type and color, and the user selects the desired cosmetic film by any operation, thereby acquiring the cosmetic film information.

[0033] The reflectance spectrum of a cosmetic film is derived, for example, as follows. First, a cosmetic film is applied at multiple thicknesses to a base with known spectral reflectance, and an operator measures the spectral reflectance of the cosmetic film under each condition using a colorimetric device. Then, based on the difference between the measurement results, an arbitrary information processing unit derives the reflectance spectrum of the cosmetic film. The base is, for example, a bioskin plate or the like having a texture similar to that of skin. Furthermore, it is preferable that color measurement be performed at multiple angles and with a certain degree of spatial resolution. The information processing unit includes an estimation model using the Kubelka-Munk theory, an estimation model using machine learning, and the like. The estimation model using machine learning is generated by machine learning training data that associates the difference in spectral reflectance between the color of the cosmetic film and the base with the difference in the measured spectral reflectance of the cosmetic film, and estimates the spectral reflectance from the color of the cosmetic film. Such information processing unit is implemented by an arbitrary information processing unit.

[0034] In step S24, the control unit 103 acquires information about the light source spectrum (hereinafter referred to as the ambient light spectrum) of the lighting device 14 that emits ambient light. The ambient light spectrum is the spectral intensity of light output from the light source of the lighting device 14. For example, the control unit 103 reads out information about the spectral intensity of the lighting device 14 that is stored in advance in the storage unit 102. Alternatively, for example, an operator measures the spectral intensity of the ambient light using a spectroscopic measurement device. The measurement results are input by the operator to the information processing device 10, or are sent from the spectroscopic measurement device to the information processing device 10 via the network 11 or via a peer-to-peer connection.

[0035] In step S25, the control unit 103 acquires information about the light source spectrum (hereinafter referred to as the projection light spectrum) of the projection device 13 that projects colored light. The projection light spectrum is the spectral intensity of a standard color output by the light source of the projection device 13. The standard color is a monochromatic color that is arbitrarily set and is projected onto an area where no makeup is applied. For example, the control unit 103 reads out information about the spectral intensity of the standard color from information about the spectral intensity of each light source of the projection device 13 that is stored in advance in the storage unit 102. Alternatively, for example, an operator measures the spectral intensity of the projection light using a spectroscopic measurement device. The measurement results are input by the operator to the information processing device 10, or are sent from the spectroscopic measurement device to the information processing device 10 via the network 11 or via a peer-to-peer connection.

[0036] In step S26, the control unit 103 determines the adjusted color of the colored light to be projected onto the target site based on the reflection spectrum of the target site, the reflection spectrum of the cosmetic film, the ambient light spectrum, and the projection light spectrum, using the following method.

[0037] First, the reflected light at the makeup area when a cosmetic film is applied to the target area, that is, the target color spectrum X(λ) (λ is the wavelength of light), is estimated by the following formula 1: X(λ)=R(λ)(P BG (λ)+E(λ)) where R(λ) is the spectral reflectance of the makeup area, P BG (λ) is the spectral intensity of the standard color by the projection device 13, and E(λ) is the spectral intensity of the ambient light.

[0038] Based on the above formula 1, when the number of light sources of the projection device 13 is a, the target color of the makeup area can be expressed as an a-dimensional vector X, and the output of the projection device 13 can be uniquely determined. For example, in a projector that uses LED light sources of three colors, R, G, and B, the number of light sources is three. In this case, the i-th (1≦i≦a) element X of the vector X is ｉ The following equation 2 is derived for the wavelength λ in the visible light region (360 nm to 830 nm): where P(λ) is the spectral intensity of the projection device 13, E(λ) is the spectral intensity of the ambient light, R(λ) is the spectral reflectance of the target area, and S i(λ) is a color matching function (RGB, XYZ, etc.) corresponding to the reflectance spectrum of the cosmetic film.

[0039] Based on the above equation 2, the intensity of the jth (1≦j≦a) light source in the projection device 13 at its maximum output is expressed as I ｊ (where 0≦I ｊ ≦1), the following equation 3 is derived. where k ji、 e i are as follows: k ji =ΣP max,j (λ)R(λ)S i (λ) e i =ΣE(λ)R(λ)S i (λ) Also, P max,j (λ) is the maximum output of the jth light source.

[0040] The following equation 4 is derived from the above equation 3.

[0041] In the above equation 4, the control unit 103 derives, as the value of the adjusted color of colored light, a matrix I indicating the intensity of each light source such that reflected light from the target area exhibits the target color X. The derivation of the adjusted color is performed by the control unit 103 performing the calculations of the above equations 1 to 4 according to an arbitrary algorithm. Note that if the adjusted color has a saturation exceeding an arbitrary reference value, the control unit 103 may correct the value of matrix I so that colored light of a color that contrasts with the adjusted color is irradiated around the adjusted color. Alternatively, if the normal to the target area onto which the adjusted color is irradiated forms an angle with the optical axis of the projection device 13 that exceeds an arbitrary reference value, for example, if the target area is the cheek, the control unit 103 may correct the value of matrix I so as to reduce the luminance of the adjusted color.

[0042] In step S27, the control unit 103 sends an instruction to the projection device 13 to project colored light. The instruction includes information specifying the adjustment color. Based on the spatial coordinates of the projection device 13 and the spatial coordinates of the target area stored in advance in the storage unit 102, the control unit 103 identifies light sources for projecting the adjustment color onto the target area and generates information specifying the intensity of each light source based on the results derived in step S26.

[0043] When the projection device 13 projects colored light of the adjusted color onto the target area in response to an instruction from the information processing device 10, the target area takes on the target color, and makeup is simulated on the face of the user 15.

[0044] 2 , it is possible to improve the accuracy of expressing different target colors of makeup areas using colored light depending on the combination of the user 15's skin color and the color of the makeup film. For example, when trying on cosmetics, it is important to accurately reproduce the color of the makeup. However, because the color of the skin after applying the cosmetics and the color obtained as a result of projecting an image from a projector may change depending on the user's skin color, color correction is necessary to accurately reproduce the makeup color. Color correction is usually difficult for objects with complex shapes and moving objects such as faces, since the reflectance spectrum of skin varies from user to user. However, this embodiment makes it possible to easily and accurately express desired colors even for objects such as faces, which are dynamic and have different colors for each individual.

[0045] 2 may be executed in response to, for example, the movement of the face of the user 15. For example, when the control unit 103 detects a temporal displacement of the face of the user 15 in the captured image, the control unit 103 executes steps S21 and S27. In this way, even if the face of the user 15 moves after the adjustment color is derived once, it is possible to execute a makeup simulation by following the movement of the face.

[0046] Furthermore, steps S21 to S26 may be performed for each target site. When multiple target sites are detected in step S21 and the cosmetic film information acquired in step S23 targets different types of target sites, the control unit 103 may perform steps S25 to S26 for each target site. Since the reflectance spectrum differs depending on the type of target site, it becomes possible to more accurately determine the adjustment color.

[0047] Furthermore, the present embodiment also includes a case where the information processing device 10 operates in a stand-alone state without being connected to the network 11 in part of the procedure shown in FIG.

[0048] In the above description, an example has been given in which the subject of the makeup simulation is a user. However, the subject may be an animal other than a person, such as a pet. This embodiment is also applicable when projection mapping is performed as part of a show or other production other than a makeup simulation. Alternatively, this embodiment is also applicable when projection mapping is performed on part or all of an inanimate object, such as a vehicle or a mechanical structure, as the subject.

[0049] In the above, the processing / control program that defines the operation of the information processing device 10 may be stored on a cloud server or the like and downloaded to the information processing device 10 via the network 11, or may be stored on a non-transitory recording / storage medium that is readable by a computer and read by the information processing device 10 from the medium.

[0050] Although the embodiments have been described above based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means, step, etc. can be rearranged so as not to be logically inconsistent, and multiple means, steps, etc. can be combined or divided into one.

[0051] Some embodiments of the present disclosure are exemplified below. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Supplementary Note 1] A system comprising: an imaging device that images an object; an information processing device that detects a target portion of the object based on the captured image of the object; and a projection device that projects colored light onto the target portion, wherein the information processing device instructs the projection device on a color of the colored light such that the reflected light from the target portion exhibits a target color, based on information on the reflection spectrum of the target portion and information on the light source spectrum of the projection device. [Supplementary Note 2] The system in Supplementary Note 1, wherein the information processing device determines the color of the colored light by taking into account information on the light source spectrum of ambient light illuminating the object. [Supplementary Note 3] The system in Supplementary Note 1 or 2, wherein the information processing device receives input information on the color classification of the target portion and determines the reflection spectrum based on the classification. [Supplementary Note 4] In any of Supplements 1 to 3, the information processing device sets the target color for each target portion based on information about the cosmetic film and information about the color of the target portion, and determines the color of the colored light for each target portion. [Supplementary Note 5] In any of Supplements 1 to 4, a system used for checking the finish of a cosmetic product without applying the cosmetic product to the object. [Supplementary Note 6] An information processing device having a communication unit and a control unit that communicates via the communication unit, wherein the control unit acquires a captured image of the object from the imaging device, detects a target portion on the object based on the captured image, and, based on information about the reflection spectrum of the target portion and information about the light source spectrum of the projection device, instructs a projection device to select a color of colored light such that reflected light from the target portion exhibits the target color, and causes the projection device to project the colored light onto the target portion. [Supplementary Note 7] In Supplementary Note 6, the control unit determines the color of the colored light by taking into account information about the light source spectrum of ambient light illuminating the object. [Supplementary Note 8] The information processing device according to Supplementary Note 6 or 7, wherein the control unit receives input of information on a color classification of the target portion, and determines the reflectance spectrum based on the classification.[Supplementary Note 9] The information processing device in any of Supplements 6 to 8, wherein the control unit sets the target color for each target portion based on information about the cosmetic film and information about the color of the target portion, and determines the color of the colored light for each target portion. [Supplementary Note 10] The information processing device in any of Supplements 6 to 9, wherein the information processing device is used to check the finish of a cosmetic product without applying the cosmetic product to the object. [Supplementary Note 11] A method for operating a system having an imaging device, a projection device, and an information processing device, wherein the imaging device images an object, the information processing device detects a target portion on the object based on the captured image of the object, and instructs the projection device on a color of colored light that will cause reflected light from the target portion to exhibit the target color based on information about the reflection spectrum of the target portion and information about the light source spectrum of the projection device, and the projection device projects the colored light onto the target portion. [Supplementary Note 12] A system operation method as set forth in Supplementary Note 11, wherein the information processing device determines the color of the colored light by taking into account information on the light source spectrum of ambient light illuminating the object. [Supplementary Note 13] A system operation method as set forth in Supplementary Note 11 or 12, wherein the information processing device receives input information on the color classification of the target area and determines the reflectance spectrum based on the classification. [Supplementary Note 14] A system operation method as set forth in any of Supplements 11 to 13, wherein the information processing device sets the target color for each target area based on information on a cosmetic film and information on the color of the target area, and determines the color of the colored light for each target area. [Supplementary Note 15] A system operation method as set forth in any of Supplements 11 to 14, wherein the information processing device determines the color of the colored light for each target area.

[0052] 10: Information processing device 11: Network 12: Imaging device 13: Projection device 14: Lighting device 101: Communication unit 102: Storage unit 103: Control unit 105: Input unit 106: Output unit

Claims

1. A system comprising: an imaging device that images an object; an information processing device that detects a target portion of the object based on the captured image of the object; and a projection device that projects colored light onto the target portion, wherein the information processing device instructs the projection device on the color of the colored light so that the reflected light from the target portion exhibits a target color, based on information on the reflection spectrum of the target portion and information on the light source spectrum of the projection device.

2. The system according to claim 1, wherein the information processing device determines the color of the colored light by taking into account information on the light source spectrum of the ambient light that illuminates the object.

3. The system according to claim 1, wherein the information processing device receives input of information on a color classification of the target portion and determines the reflectance spectrum based on the classification.

4. A system according to claim 1, wherein the information processing device sets the target color for each target area based on cosmetic film information and color information of the target area, and determines the color of the colored light at the target area.

5. The system according to claim 1, which is used to check the finish of a cosmetic product without applying the cosmetic product to the object.

6. An information processing device having a communication unit and a control unit that communicates via the communication unit, wherein the control unit obtains an image of an object from an imaging device, detects a target portion of the object based on the image, and, based on information on the reflection spectrum of the target portion and information on the light source spectrum of the projection device, instructs the projection device to select a color of colored light such that the reflected light from the target portion exhibits a target color, and causes the projection device to project the colored light onto the target portion.

7. An information processing device according to claim 6, wherein the control unit determines the color of the colored light by taking into account information on a light source spectrum of the ambient light that illuminates the object.

8. An information processing device according to claim 6, wherein the control unit receives input of information on a color classification of the target portion, and determines the reflectance spectrum based on the classification.

9. An information processing device according to claim 6, wherein the control unit sets the target color for each target site based on information about the cosmetic film and information about the color of the target site, and determines the color of the colored light at the target site.

10. An information processing device according to claim 6, which is used to check the finish of the cosmetic product without applying the cosmetic product to the object.

11. A method of operating a system having an imaging device, a projection device, and an information processing device, wherein the imaging device captures an image of an object, the information processing device detects a target portion of the object based on the captured image of the object, and based on information on the reflection spectrum of the target portion and information on the light source spectrum of the projection device, instructs the projection device on a color of colored light that will cause the reflected light at the target portion to exhibit a target color, and the projection device projects the colored light onto the target portion.

12. A method for operating a system according to claim 11, wherein the information processing device determines the color of the colored light by taking into account information on a light source spectrum of the ambient light that illuminates the object.

13. A method of operating a system according to claim 11, wherein the information processing device receives input of information on a color classification of the target portion, and determines the reflectance spectrum based on the classification.

14. A method of operating a system according to claim 11, wherein the information processing device sets the target color for each target area based on information about the cosmetic film and information about the color of the target area, and determines the color of the colored light at the target area.

15. A method according to claim 11, wherein the system is used to check the finish of a cosmetic product without applying the cosmetic product to the object.

Images