Coating control device, coating device, coating control method, and recording medium

The coating control device uses an imaging unit and recognition algorithm to accurately position the coating head on the skin by comparing images, addressing the lack of precise positioning in existing technologies and enhancing cosmetic application accuracy.

JP7577431B2Active Publication Date: 2024-11-05SHISEIDO CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2018557762
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2016-12-20
Filing Date
2017-12-18
Publication Date
2024-11-05
Estimated Expiration
2037-12-18

AI Technical Summary

Technical Problem

Existing technologies, such as the makeup device disclosed in Patent Document 1, do not provide a method for accurately positioning a coating head on the skin during the application of cosmetics, necessitating a way to recognize the head's position for precise application.

Method used

A coating control device that includes a storage unit for a first image of the skin, an imaging unit connected to a coating head, and a recognition unit to determine the position of a second image within the first image, using correlation calculations to align the coating head accurately.

Benefits of technology

Enables precise positioning of the coating head on the skin, ensuring accurate application of cosmetics by comparing the second image with the stored first image, thereby improving application accuracy and consistency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007577431000002
    Figure 0007577431000002
  • Figure 0007577431000003
    Figure 0007577431000003
  • Figure 0007577431000004
    Figure 0007577431000004
Patent Text Reader

Abstract

An application control device according to one embodiment of the present invention includes a memory unit that stores a first image of a predetermined area of ​​skin, an acquisition unit that acquires a second image of the skin of an area smaller than the area of ​​the first image from an imaging unit connected to an application head capable of applying cosmetics to the skin, and a recognition unit that recognizes the position of the second image in the first image.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a coating control device, a coating device, a coating control method, and a recording medium. [Background technology]

[0002] There is a known technology for applying cosmetics to the skin automatically using a printing device such as an inkjet, instead of applying the cosmetics manually using a finger, sponge, etc. Patent Document 1 discloses a makeup device equipped with a head that ejects cosmetic ink and a moving device that can move the head. The makeup device of Patent Document 1 measures data such as unevenness and brightness distribution of the surface to be made up (skin), and determines the positions of blemishes, wrinkles, etc. on the surface to be made up based on the data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-271654 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not specifically disclose a method for positioning the head relative to the target area, such as spots or wrinkles. The moving device in Patent Document 1 can move the head along the skin, but in order to position it, it is necessary to know where on the skin the head is located.

[0005] The present invention has been made in consideration of such problems, and aims to provide an application control device, an application device, an application control method, and a recording medium that are capable of recognizing the position of an application head on the skin. [Means for solving the problem]

[0006] An application control device according to one embodiment of the present invention comprises a memory unit that stores a first image of a predetermined area of ​​the skin, an acquisition unit that acquires a second image of the skin of an area smaller than the area of ​​the first image from an imaging unit connected to an application head capable of applying cosmetic to the skin, and a recognition unit that recognizes the position of the second image in the first image.

[0007] An application control method according to one embodiment of the present invention includes the steps of storing a first image of a predetermined area of ​​skin, acquiring a second image of skin of an area smaller than the area of ​​the first image from an imaging unit connected to an application head capable of applying cosmetic to the skin, and recognizing the position of the second image in the first image.

[0008] A recording medium according to one embodiment of the present invention records a program that causes a computer to execute the steps of storing a first image of a predetermined area of ​​skin, acquiring a second image of skin of an area smaller than the area of ​​the first image from an imaging unit connected to an application head capable of applying cosmetic to the skin, and recognizing the position of the second image in the first image. Effect of the Invention

[0009] According to the present invention, there are provided an application control device, an application device, an application control method, and a recording medium capable of recognizing the position of an application head on the skin. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of a coating system according to a first embodiment. [Diagram 2] 1 is a block diagram of a coating device and a coating control device according to a first embodiment. [Diagram 3] FIG. 2 is a functional block diagram of the coating control device according to the first embodiment. [Figure 4] 4 is a flowchart showing a coating control method according to the first embodiment. [Diagram 5] FIG. 4 is a diagram for explaining correlation processing according to the first embodiment. [Figure 6] 5 is a flowchart showing details of a coating process according to the first embodiment. [Figure 7] 5A and 5B are a schematic view and a cross-sectional view of a coating apparatus according to a second embodiment. [Figure 8] FIG. 13 is a schematic diagram showing the configuration of a coating system according to a third embodiment. [Figure 9] FIG. 11 is a block diagram of a coating device and a coating control device according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [First embodiment] FIG. 1 is a schematic diagram showing the configuration of an application system 10 according to this embodiment. The application system 10 includes an application device 100, an imaging unit 200, and an application control device 300. The application device 100 is a device for applying cosmetics (makeup material) to the skin, and is held by a user. The application device 100 has a prismatic housing, and an application head 101 is provided on one end surface of the housing. The shape of the application device 100 is not limited as long as it is a shape that is easy for a user to hold, and may be cylindrical or hemispherical. The application device 100 may also include a gripping member such as a handle.

[0012] The application head 101 is composed of, for example, an inkjet head, and has a plurality of nozzles for ejecting the cosmetic material. The plurality of nozzles are arranged two-dimensionally, and are capable of applying the cosmetic material to a predetermined area of ​​the skin. A cosmetic material tank 102 is attached to the application device 100, and the cosmetic material is supplied from the cosmetic material tank 102 to the application head 101. The cosmetic material tank 102 may be provided inside the application device 100. The cosmetic material may be a liquid concealer for concealing blemishes, freckles, pores, etc. on the skin.

[0013] An imaging unit 200 is provided on the side (top) of the housing of the coating device 100 in the same orientation as the coating head 101. The imaging unit 200 includes a lens, an imaging element, etc., and can capture an image (second image) of a narrow area of ​​the skin to which coating is performed by the coating head 101. The imaging unit 200 is connected to the coating head 101, and the position of the imaging unit 200 relative to the coating head 101 is fixed. The imaging unit 200 may be formed integrally with the coating head 101.

[0014] The coating device 100 and the imaging unit 200 are controlled by a coating control device 300. The coating control device 300 is connected to the coating device 100 and the imaging unit 200 via a wired connection such as a USB (Universal Serial Bus) cable or a wireless connection such as Bluetooth (registered trademark) or WiFi. The coating control device 300 may be built into the coating device 100. The coating control device 300 pre-stores an image (first image) of a wide range of skin including spots, freckles, pores, etc. to be coated. The coating control device 300 can grasp the position of the coating head 101 on the skin by comparing the second image acquired from the imaging unit 200 with the first image. The coating control device 300 includes a display 301, on which various information such as an image of the skin and the state of the coating head 101 is displayed.

[0015] When a user holds application device 100 and brings application head 101 close to the skin, application control device 300 recognizes the position of application head 101 on the skin and displays the current position of application head 101 on display 301. The user moves application head 101 along the skin while checking display 301, and when application head 101 arrives at a position where a coating target is located, it automatically applies the cosmetic.

[0016] 2 is a block diagram of the coating device 100 and the coating control device 300 according to this embodiment. The coating device 100 includes a coating head 101, a cosmetic tank 102, a moving mechanism 103, an operation unit 104, a distance sensor 105, and a motion sensor 106. The coating control device 300 includes a display 301, an image processing circuit 302, a gap control circuit 303, a head control circuit 304, a CPU 305, a RAM 306, a ROM 307, a storage device 308, and a speaker 309.

[0017] The application head 101 is, for example, a piezoelectric inkjet head, and is composed of a nozzle, a pressure chamber, a piezoelectric element, a drive circuit, etc. The pressure chamber is filled with a cosmetic material, and when a voltage is applied from the drive circuit to the piezoelectric element, the volume of the pressure chamber changes due to the deformation of the piezoelectric element. This causes the cosmetic material to be ejected in droplets from the nozzle. Note that the application head 101 may also be a thermal inkjet head that heats the cosmetic material with a heater and ejects the cosmetic material by the pressure of the generated bubbles. The application head 101 operates based on a control signal from a head control circuit 304.

[0018] The cosmetic tank 102 contains the cosmetic and supplies the cosmetic to the application head 101. The cosmetic tank 102 may be a cartridge type that is easily replaceable. The cosmetic is a liquid having a predetermined viscosity that can be discharged from the application head 101, and may include concealer, foundation, blush, powder, eye shadow, and the like. A plurality of cosmetic tanks 102 may be provided so as to contain a plurality of cosmetics of different types or shades. For example, four cosmetic tanks 102 may be provided so as to be able to apply four colors of cosmetic, and four nozzle groups corresponding to each color may be provided on the application head 101.

[0019] The moving mechanism 103 is composed of an actuator, a guide member, etc., and can drive the application head 101 to move forward and backward in the longitudinal direction of the application device 100, that is, in a direction perpendicular to the skin when the application head 101 faces the skin. The moving mechanism 103 controls the position of the application head 101 in accordance with a control signal from a gap control circuit 303.

[0020] The operation unit 104 includes operation members such as a power switch, a menu button, and an application button for executing application, and is used by the user to give instructions to the applicator 100. The application control device 300 controls the operation of the applicator 100 in response to user instructions input from the operation unit 104. The application button is preferably disposed in a position where the user can easily operate it while holding the applicator 100, for example, in a position where the user can touch the application button with their finger when holding the applicator 100. This allows the user to operate the application button by feel even when the applicator 100 is moved to a part (such as the cheek) that cannot be directly seen.

[0021] The distance sensor 105 is, for example, an infrared sensor, an ultrasonic sensor, or the like, and irradiates an object with a detection wave such as infrared rays or ultrasonic waves and receives the reflected wave. The distance sensor 105 can detect the distance to the object based on the time from irradiating the detection wave to receiving the reflected wave. In addition, a plurality of distance sensors 105 are provided around the application head 101, and can also detect the inclination of the application head 101 with respect to the skin. Based on the detection signal from the distance sensor 105, the application control device 300 can keep the distance between the skin and the application head 101 constant, and control the application head 101 so as not to eject the cosmetic when the application head 101 is inclined with respect to the skin, for example.

[0022] The motion sensor 106 includes an acceleration sensor, a gyro sensor, and a geomagnetic sensor, and detects the movement, rotation, and other movements of the application head 101. The acceleration sensor is composed of, for example, a capacitance detection element, and can detect the acceleration applied to the application head 101. The gyro sensor is composed of, for example, a piezoelectric vibration element, and has a function of detecting the orientation of the application head 101. The geomagnetic sensor can grasp the orientation of the application head 101 by detecting geomagnetism. Based on the detection signal from the motion sensor 106, the application control device 300 can control the application head 101 so as not to eject cosmetic material, for example, when the application head 101 is moving quickly.

[0023] The imaging unit 200 includes an optical system, an imaging element, and an A / D (Analog / Digital) converter. The optical system includes an optical filter, a fixed lens, and a focus lens, and focuses light from a subject (skin) on the imaging surface of the imaging element to form an image of the subject. A polarizing filter can be attached to the optical system, and specular reflection can be reduced. The imaging element is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and includes a plurality of pixels arranged two-dimensionally, a color filter, and a microlens. The plurality of pixels may include pixels for imaging and pixels for focus detection. The imaging element also has an electronic shutter function that controls charge accumulation time. Each of the plurality of pixels outputs a pixel signal based on the incident light from the optical system. The A / D converter is composed of a comparison circuit, a latch circuit, etc., and converts the analog pixel signal from the imaging element into digital RAW data. The imaging unit 200 can output not only still images, but also moving images at a predetermined frame rate.

[0024] The display 301 is configured, for example, by a liquid crystal display or an organic EL (Electro Luminescence) display. The display 301 displays various information such as an image from the imaging unit 200, an image stored in the storage device 308, status information of the coating head 101, a menu screen, etc. based on data from the CPU 305. The display 301 may be a touch panel and can also function as the operation unit 104.

[0025] The image processing circuit 302 includes a numerical calculation circuit, and performs demosaic processing on the RAW data from the imaging unit 200 to generate image data (RGB image) having color values ​​of R (red), G (green), and B (blue) for each pixel. The image processing circuit 302 also has a function of performing digital image processing such as white balance adjustment, gamma correction, edge enhancement, gradation conversion, noise reduction, and compression on the image data.

[0026] The gap control circuit 303 controls the distance (gap) between the skin and the application head 101 by outputting a control signal to the movement mechanism 103. The gap control circuit 303 can control the position of the application head 101 so as to maintain a constant distance from the skin, based on a detection signal from the distance sensor 105. The head control circuit 304 outputs a control signal indicating information about the nozzle that ejects the cosmetic material, the application amount, etc., to the application head 101, based on instructions from the CPU 305.

[0027] The CPU (Central Processing Unit) 305 includes a CPU core, a cache memory, and the like, and performs overall control of each part of the coating control device 300. The RAM (Random Access Memory) 306 is, for example, a DRAM (Dynamic RAM), and is used as a work area for the CPU 305, a program load area, and the like. The RAM 306 temporarily stores data required for processing by the CPU 305, image data generated by the image processing circuit 302, image data read from the storage device 308, and the like. The ROM (Read Only Memory) 307 is, for example, an EEPROM (Electrically Erasable Programmable ROM), and stores various setting files, basic programs such as an OS (Operating System), and a control program for controlling the operation of the coating device 100.

[0028] The storage device (storage unit) 308 is, for example, a flash memory or a hard disk, and stores LAW data from the imaging unit 200, image data generated by the image processing circuit 302, etc. The storage device 308 can also store image data acquired by an external imaging device. The storage device 308 may be a portable storage medium, and can be configured to be detachable from the coating control device 300 via a memory card slot, a USB connector, etc.

[0029] The speaker 309 includes a piezoelectric vibration unit, a drive circuit, and the like, and outputs a sound wave signal based on data from the CPU 305. The speaker 309 can reproduce voice messages, sound effects, and the like, and is used to notify the user of the operating state of the coating apparatus 100, for example.

[0030] 3 is a functional block diagram of a coating control device 300 according to this embodiment. The coating control device 300 has the functions of a storage unit 308, an acquisition unit 310, a recognition unit 311, and a calculation unit 312. The functions of the coating control device 300 are realized by a CPU 305 reading a predetermined control program stored in a ROM 307 into a RAM 306 and executing the program.

[0031] The storage unit 308 stores a first image of a predetermined area of ​​the skin. The first image is an image of a wide area of ​​the skin including spots, freckles, pores, etc. to be applied. The first image can be captured using a dedicated imaging device or a general digital camera, and is stored in advance in the storage unit 308. The first image can also be captured using the imaging unit 200.

[0032] The acquisition unit 310 acquires a second image from the imaging unit 200. The second image is an image of the skin in a range smaller than the range of the first image. The acquisition unit 310 acquires the second image at predetermined time intervals and transfers the second image to the recognition unit 311. The imaging unit 200 is connected to the coating head 101, and the position of the coating head 101 can be obtained from the position of the imaging unit 200.

[0033] The recognition unit 311 compares the second image from the acquisition unit 310 with the first image stored in the storage unit 308, and recognizes the position of the second image in the first image. The recognition unit 311 is capable of performing correlation calculations on the image data, and can extract from the first image an area having the highest correlation with the second image.

[0034] The calculation unit 312 divides the first image or the second image into a plurality of segments and calculates the amount of cosmetic required to be applied for each segment. Any method for calculating the amount of cosmetic application may be used, and for example, the amount of cosmetic application may be calculated so that the luminance distribution of the skin after application is uniform. The application head 101 is capable of applying the cosmetic to the skin according to the amount of cosmetic application from the calculation unit 312.

[0035] FIG. 4 is a flowchart of the application control method according to this embodiment. Here, an example of applying cosmetics to blemishes on the cheek will be described. First, CPU 305 reads an image (first image) including multiple blemishes from storage device 308 and stores it in RAM 306 (step S401). The first image is an image capturing a wide area of ​​the cheek (e.g., 3 cm long x 4 cm wide). The first image may be captured using a polarizing filter. Furthermore, CPU 305 displays the first image on display 301. CPU 305 may identify the positions of multiple blemishes from the first image and display icons or the like on display 301 to indicate the positions of the blemishes to the user. The positions of the blemishes can be identified based on, for example, image features (e.g., brightness distribution).

[0036] Next, the user holds the applicator 100 and brings the applicator head 101 close to the skin. If the applicator head 101 is already close to the skin, the user moves the applicator head 101 along the skin (step S402). While watching the display 301, the user can move the applicator head 101 in the direction of the blemish. The imaging unit 200 captures moving images of the skin at a predetermined frame rate.

[0037] CPU 305 acquires a second image from imaging unit 200 (step S403). The second image is one frame of a moving image, and is an image of a small area of ​​the cheek (e.g., 1 cm long x 1 cm wide). Like the first image, the second image may also be captured using a polarizing filter. CPU 305 generates image data for the second image in image processing circuit 302, and stores the image data in RAM 306. Note that the second image may be a single still image, and the data format of the second image is not particularly limited.

[0038] Next, the CPU 305 calculates the correlation between the first image and the second image (step S404). Specifically, the CPU 305 calculates the value of the evaluation function F(τ, υ) represented by the following equation (1).

Equation

[0039] As shown in FIG. 5, f(x, y) represents the pixel value at the coordinates (x, y) of the second image 502, and g(x, y) represents the pixel value at the coordinates (x, y) of the first image 501. In the first image 501 and the second image 502, the hatched portions represent defects.

[0040] The origin O of the first image 501 and the second image 502 is set to the pixel at the lower left corner, respectively. The size of the second image 502 is M pixels in vertical direction × N pixels in horizontal direction, and the size of the first image 501 is larger than that of the second image 502, which is M1 pixels in vertical direction × N1 pixels in horizontal direction (M < M1 and N < N1). The parameters τ, υ represent the shift amount of the window region 501a in the first image. The window region 501a has the same size as the second image 502. By changing the values of the parameters τ, υ, the window region 501a can be scanned within the first image 501. The evaluation function F(τ, υ) corresponds to the sum of products of corresponding pixel values between the second image 502 and the window region 501a. The evaluation function F(τ, υ) is not limited to Equation (1), and may be the sum of absolute values of differences, the sum of squares of pixel value differences, the correlation coefficient, etc. for corresponding pixel values.

[0041] The pixel value can be the luminance value of the image data, each color value (R value, G value, B value) of an RGB image, a value obtained by appropriately weighting and adding each color value, a value expressed in a color system other than RGB, etc. The image correlation can be calculated using a specific wavelength component, and it is preferable to use, for example, a wavelength component (B signal) of 540 nm or less.

[0042] In step S404, CPU 305 calculates the value of evaluation function F(τ, υ) while changing parameters τ and υ within the ranges of 0≦τ≦N1-N and 0≦υ≦M1-M, respectively. That is, CPU 305 scans the entire area of ​​first image 501 while shifting the position of window area 501a by one pixel vertically or horizontally, and calculates the correlation between second image 502 and window area 501a.

[0043] Next, CPU 305 recognizes the position of the second image in the first image (step S405). CPU 305 determines that the correlation is higher as the value of evaluation function F(τ, υ) is larger, and recognizes the position (τ, υ) of window region 501a when evaluation function F(τ, υ) is at its maximum value as the position of second image 502. CPU 305 calculates the position of application head 101 based on the position of second image 502, and displays position information of application head 101 on display 301. For example, CPU 305 displays a frame indicating an area that application head 101 can currently apply, superimposed on the first image.

[0044] Next, the CPU 305 judges whether or not there is a coating target (stain) at the position of the coating head 101 (step S406). For example, the CPU 305 refers to the position of the stain identified in step S401 and judges whether or not the stain is included in the area that the coating head 101 can currently coat. If the CPU 305 judges that there is no stain at the position of the coating head 101 (NO in step S406), it returns to step S402 and waits until the coating head 101 is moved to another position by the user. If the CPU 305 judges that there is a stain at the position of the coating head 101 (YES in step S406), it executes the coating process of FIG. 6 (step S407). The CPU 305 may inform the user that the coating process will be executed using the display 301 and the speaker 309.

[0045] After executing the application process, CPU 305 judges whether application has been completed for all application targets (step S408). That is, CPU 305 judges whether application has been completed for all blemishes included in the first image. If CPU 305 judges that application has not been completed (NO in step S408), it returns to step S402 and waits until application head 101 is moved to another position by the user. If CPU 305 judges that application has been completed (YES in step S408), it displays a message on display 301 indicating that application has been completed, and ends the process.

[0046] FIG. 6 is a flowchart showing details of the application process (step S407) according to this embodiment. First, the CPU 305 calculates the amount of application of the cosmetic based on the second image acquired from the imaging unit 200 (step S601). For example, the CPU 305 calculates the amount of application required for each of a plurality of segments obtained by dividing the second image. The segment can be a square pixel block consisting of a plurality of pixels, and the CPU 305 calculates the amount of application independently for each segment. The CPU 305 may calculate the amount of application for the target segment based on a plurality of segments including the target segment and its surrounding segments. The amount of application of the cosmetic may be calculated based on the first image in step S401 of FIG. 4. The CPU 305 can calculate the amount of application required for each of a plurality of segments obtained by dividing the first image, similar to the second image.

[0047] Next, CPU 305 adjusts the distance between application head 101 and the skin (step S602). That is, CPU 305 outputs a control signal based on the detection value by distance sensor 105 to movement mechanism 103 via gap control circuit 303. Movement mechanism 103 moves application head 101 forward and backward, thereby keeping the gap with the skin constant. If the size of the gap exceeds the range that can be adjusted by movement mechanism 103, CPU 305 can prompt the user to move application head 101 closer to or slightly farther away from the skin using display 301, speaker 309, etc. Note that the gap adjustment process (step S602) may be executed in parallel with the above-mentioned application amount calculation process (step S601).

[0048] Next, CPU 305 causes application head 101 to eject the cosmetic (step S603). That is, CPU 305 outputs control signals, such as identification information of the nozzle that ejects the cosmetic, and the amount of application, to application head 101 via head control circuit 304. Application head 101 ejects the cosmetic from the nozzle in response to the control signal from head control circuit 304. Note that CPU 305 may permit ejection of the cosmetic only when an execution instruction is received from the user. For example, CPU 305 may perform the cosmetic ejection process (step S603) when the application button is operated by the user, and may return to the flowchart of FIG. 4 without performing the cosmetic ejection process if the application button is not operated for a predetermined period of time.

[0049] Furthermore, the CPU 305 may acquire status information such as the movement and posture of the application head 101 based on detection signals from the distance sensor 105 and the motion sensor 106. The CPU 305 can control the timing of ejection based on the status information of the application head 101. For example, when the amount of shaking of the application head 101 or the inclination with respect to the skin exceeds a predetermined range, the CPU 305 warns the user using the display 301 and the speaker 309, and delays the timing of ejection until the application head 101 assumes an appropriate posture with respect to the skin.

[0050] According to this embodiment, the position of the application head 101 on the entire skin can be accurately recognized by comparing the second image acquired from the imaging unit 200 connected to the application head 101 with the first image capturing the entire skin including the application target. This makes the positional relationship between the application head 101 and the application target on the skin clear, and enables the user to guide the application head 101 to the application target without hesitation.

[0051] Furthermore, the first image and the second image are images of the same part of the skin captured at different sizes, but they differ not only in size but also in the timing of capturing images, the swelling of the skin, the skin tone, the degree of specular reflection, etc. For this reason, it may be difficult to detect the position of the second image in the first image by simply matching the images, but according to this embodiment, the position of the second image in the first image can be accurately detected by using the correlation between the first image and the second image.

[0052] [Second embodiment] Next, a coating apparatus 700 according to a second embodiment of the present invention will be described. The coating apparatus 700 according to this embodiment has a similar configuration to the coating apparatus 100 according to the first embodiment, and therefore differences from the first embodiment will be mainly described.

[0053] FIG. 7(a) is a schematic diagram showing the appearance of the applicator 700, and FIG. 7(b) is a cross-sectional view taken along line A-A' in FIG. 7(a). The applicator 700 is provided with a guide member (holding section) 710 on an end surface (hereinafter referred to as a head surface) on which the applicator head 101 and the imaging section 200 are provided. The guide member 710 is a hollow rectangular column and is made of a transparent or semi-transparent synthetic resin or the like. The guide member 710 is disposed along the edge of the head surface and is fixed to a housing 711 of the applicator 700. The tip portion of the guide member 710 is bent inward so as to make it easier to press the skin. The user holds the applicator 700 and lightly presses the tip portion of the guide member 710 against the skin. The guide member 710 holds the applicator head 101 at a constant distance D in the z-axis direction from the skin.

[0054] According to this embodiment, since the guide member 710 is provided on the head surface of the coating device 700, there is no need to adjust the gap using the movement mechanism 103. In addition, by pressing the guide member 710 against the skin, it is possible to suppress swelling of the skin and improve the flatness of the skin. This makes it possible to improve the coating accuracy of the coating head 101.

[0055] [Third embodiment] Next, a coating system 80 according to a third embodiment of the present invention will be described. Fig. 9 is a schematic diagram showing the configuration of the coating system 80 according to this embodiment. The coating system 80 includes a coating device 800, an imaging unit 200, and a coating control device 900. The coating device 800 includes a coating head 801 and a robot arm 810, and the imaging unit 200 is connected to the coating head 801. The coating head 801 is attached to the tip of the robot arm 810.

[0056] The robot arm 810 is a multi-joint arm type robot, and by changing the posture of the arm, the position and orientation of the application head 801 can be freely changed. For example, the robot arm 810 can move the application head 801 in any direction along the skin based on a drive command from the application control device 900. The application control device 900 is connected to the robot arm 810, and the application control device 900 can control the operation of the robot arm 810 using the drive command. The application control device 900 includes a display 301, and various information such as an image of the skin and the state of the application head 801 is displayed on the display 301.

[0057] 9 is a block diagram of a coating device 800 and a coating control device 900 according to this embodiment. The coating device 800 includes a coating head 801, a robot arm 810, a cosmetic tank 102, an operation unit 104, and a distance sensor 105. The coating control device 900 includes a robot controller 910, a display 301, an image processing circuit 302, a head control circuit 304, a CPU 305, a RAM 306, a ROM 307, a storage device 308, and a speaker 309.

[0058] The application head 801 is configured similarly to the application head 101 according to the first embodiment, but in the application head 801, a plurality of nozzles are arranged one-dimensionally (for example, in a line). The robot arm 810 can apply the cosmetic by sweeping the application head 801 in a direction perpendicular to the direction in which the plurality of nozzles are arranged. The application device 800 may include a plurality of cosmetic tanks 102 each containing a different shade of cosmetic, and the application head 801 may be provided with a plurality of nozzle rows corresponding to each color. The robot controller 910 outputs a drive command generated based on an instruction from the CPU 305 to the robot arm 810.

[0059] The cosmetic tank 102, the operation unit 104, the distance sensor 105, the display 301, the image processing circuit 302, the head control circuit 304, the CPU 305, the RAM 306, the ROM 307, the storage device 308, and the speaker 309 are the same as those in the first embodiment, and therefore the description thereof will be omitted. The operation unit 104 may be provided on a base supporting the robot arm 810, on the display 301, or on the like. The storage device 308 may also store in advance the shape of the skin associated with the first image in the form of three-dimensional coordinates or the like. The robot controller 910 can control the robot arm 810 so as to hold the application head 801 at a certain distance from the skin based on the shape of the skin.

[0060] 4, the present embodiment differs in that in step S402, the robot arm 810 moves the coating head 801. That is, the CPU 305 determines the direction and distance to move the coating head 101 based on the position of the second image in the first image recognized in step S405, and drives the robot arm 810 via the robot controller 910.

[0061] According to this embodiment, CPU 305 recognizes the position of application head 801 on the skin and can move application head 801 to the position of the application target using robot arm 810. Even if the user moves while application head 801 is moving, CPU 305 can recognize the position of application head 801 in real time, and therefore can instantly correct the movement direction and movement amount of application head 801.

[0062] [Other embodiments] The present invention is not limited to the above-mentioned embodiment, and can be modified without departing from the spirit of the present invention. For example, when calculating the correlation between the first image and the second image (step S404), feature points such as spots, moles, and pores may be extracted from each image, and the correlation may be calculated only for the feature points. This makes it possible to significantly reduce the amount of calculation of the evaluation function F(τ, υ).

[0063] The control process of the above-described embodiment can also be applied to a cosmetic device that acts on a specific location on the skin, such as injecting medicine using a needle or irradiating a mole or the like on the skin with a laser. [Explanation of symbols]

[0064] 100 Coating device 101 Coating head 200 Imaging unit 300 Coating control device 308 Storage device (storage unit) 310 Acquisition Department 311 Recognition part 312 Calculation Unit 501 1st image 502 Second Image

Claims

1. a storage unit that stores a first image of a predetermined area of ​​the skin captured by the first imaging unit; an acquisition unit that is connected to an application head capable of applying a cosmetic to skin and that acquires a second image of the skin in an area smaller than the area of ​​the first image from a second imaging unit different from the first imaging unit; a recognition unit that recognizes a position of the second image in the first image; a processing unit that identifies a position of an application target included in the first image based on a luminance distribution of the first image, identifies a position of the application head based on a position of the second image, and determines whether the application target is included in an area to which the cosmetic can be applied by the application head based on the identified position of the application target and the identified position of the application head; A coating control device comprising:

2. The coating control device according to claim 1 , wherein the recognition unit recognizes the position by extracting an area from the first image that has the highest correlation with the second image.

3. The coating control device according to claim 2 , wherein the recognition unit calculates the correlation by using specific wavelength components of the first image and the second image.

4. The coating control device according to claim 3 , wherein the specific wavelength component has a wavelength of 540 nm or less.

5. The coating control device according to claim 1 , wherein the first image and the second image are captured using a polarizing filter.

6. A calculation unit that divides the second image into a plurality of segments and calculates a required application amount for each of the segments, The application control device according to claim 1 , wherein the application head applies the cosmetic according to the application amount.

7. The application control device according to claim 1 , further comprising a holder for holding the application head so as to maintain a constant distance from the skin.

8. The application control device according to claim 1 , wherein the application head is an inkjet head that ejects the cosmetic in the form of droplets.

9. The coating head; The second imaging unit; The coating control device according to any one of claims 1 to 8, A coating device comprising:

10. storing a first image of a predetermined area of ​​the skin captured by the first imaging unit; acquiring a second image of the skin in an area smaller than the area of ​​the first image from a second imaging unit that is connected to an application head capable of applying a cosmetic to the skin and is different from the first imaging unit; recognizing a position of the second image in the first image; Identifying a position of an application target included in the first image based on a luminance distribution of the first image; determining a position of the application head based on a position of the second image; determining whether the application target is included in an area to which the cosmetic can be applied by the application head based on the identified position of the application target and the identified position of the application head; The coating control method includes:

11. storing a first image of a predetermined area of ​​the skin captured by the first imaging unit; acquiring a second image of the skin in an area smaller than the area of ​​the first image from a second imaging unit that is connected to an application head capable of applying a cosmetic to the skin and is different from the first imaging unit; recognizing a position of the second image in the first image; Identifying a position of an application target included in the first image based on a luminance distribution of the first image; determining a position of the application head based on a position of the second image; determining whether the application target is included in an area to which the cosmetic can be applied by the application head based on the identified position of the application target and the identified position of the application head; A recording medium on which a program for causing a computer to execute the above is recorded.

Citation Information

Patent Citations

  • Cosmetic-application system and method using liquid delivery method

    JP2006271654A

  • Makeup method

    JP2008206752A

  • Image positioning method, image positioning device, image processing system, control program, and computer-readable storage medium recording the program

    JP2008234362A

  • Skin treatment equipment, skin treatment method, and program

    JP2015159975A

  • Image pickup device, imaging apparatus, and imaging information recognition system

    JP2016127333A