Makeup support apparatus, cosmetic customization system and program

The makeup support device and system allow users to visualize and order customized cosmetics by simulating makeup on their faces, addressing the challenge of selecting suitable products for individual facial features.

JP2024160004AInactive Publication Date: 2024-11-08SHACHIHATA IND
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2024151174
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Individuals struggle to visualize the finish of cosmetic stamps and find commercially available stamps that do not fit their facial features, making it difficult to choose suitable makeup products.

Method used

A makeup support device and system that includes face image acquisition, makeup pattern selection, simulation image generation, and order information transmission to customize cosmetics based on user-specific facial features and preferences.

Benefits of technology

Enables users to simulate and order customized cosmetics that match their facial features, providing tailored makeup solutions without actual application.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160004000001_ABST
    Figure 2024160004000001_ABST
Patent Text Reader

Abstract

To provide a makeup support apparatus configured to allow a user to confirm a final condition of makeup without actually using cosmetics and to order cosmetics suitable for the user on the basis of a result of the confirmation, a cosmetic customization system and a program.SOLUTION: A terminal device 1 with eyebrow makeup application-related data installed thereon generates a simulation image SM obtained by applying eyebrow makeup based on an eyebrow sample (eyebrow pattern) selected by a user on eyebrow parts of a face included in a face image displayed on a display unit 12 (main display area 12a). The simulation image SM is displayed on the display unit 12 (main display area 12a). An eyebrow makeup stamp based on the simulation result (simulation image SM) displayed on the display unit 12 can be ordered from the terminal device 1.SELECTED DRAWING: Figure 10
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a makeup support device, a custom-made cosmetics system, and a program. [Background technology]

[0002] Conventionally, stamps (cosmetic stamps) used for applying makeup to facial features (eyebrows, mouth, cheeks, etc.) have been known (see Patent Document 1). This type of cosmetic stamp is suitable for those who want to apply makeup easily and in a shorter time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-213339 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, for those who have never used cosmetic stamps or have little experience using them, it is difficult to imagine the final look of the makeup created by the cosmetic stamps (the final image).Facial features, bone structure, and other facial characteristics vary widely, so there is also the problem that commercially available (ready-made) cosmetic stamps may not necessarily suit oneself.

[0005] To provide a makeup support device, a cosmetics order-making system, and a program for enabling a user to check the finish of makeup without actually using the cosmetics and to order cosmetics suited to the user based on the checked results, the device being devised in view of the above circumstances. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention employs the following configuration. (1) The makeup support device according to the first aspect of the present invention is A face image acquisition means for acquiring an image including a face; a makeup pattern acquisition means for acquiring a makeup pattern from a plurality of makeup patterns prepared in advance; a simulation image generating means for generating a simulation image in which makeup based on the makeup pattern acquired by the makeup pattern acquiring means is applied to a face included in the image acquired by the face image acquiring means; a simulation image display means for displaying the simulation image generated by the simulation image generating means; a first order information generating means for generating order information for a cosmetic product based on a makeup pattern included in the simulation image displayed by the simulation image display means; a first transmission means for transmitting the order information generated by the first order information generation means to an ordering party of the cosmetics; a makeup condition recognition means for recognizing a state of makeup applied to a face included in an image acquired by the face image acquisition means when the face includes actual makeup; a second order information generating means for generating order information for cosmetics having specifications corresponding to the actual makeup, based on the state of makeup recognized by the makeup state recognizing means; and a second transmission means for transmitting the order information generated by the second order information generation means to a destination of the order for the cosmetics. The gist of the present invention is as follows. (2) The makeup support device of the present invention is the makeup support device of the above-mentioned means 1, a face size calculation means for calculating an actual size of a face included in the image acquired by the face image acquisition means, The order information includes information regarding the size calculated by the face size calculation means. The gist of the present invention is as follows. (3) The cosmetic custom-made system according to the third aspect of the present invention is The makeup support device according to the first or second means, an order receiving device that receives an order for cosmetics from a user who uses the makeup support device, The makeup support device and the order receiving device are configured to be connectable via a network, The order receiving device includes a receiving means for receiving the order information transmitted by the first transmitting means or the second transmitting means, The cosmetics for the user are manufactured based on the order information received by the receiving means. The gist of the present invention is as follows. (4) The cosmetic custom-made system of the present invention is the cosmetic custom-made system of the above-mentioned means 3, The cosmetic product is an eyebrow makeup stamp that can be pressed against the eyebrows of the face to apply makeup to the eyebrows. The gist of the present invention is as follows. (5) The program of the fifth aspect of the present invention is a program for causing a computer to function as each of the means of the makeup support device of the first or second aspect. The gist of the present invention is as follows.

[0007] In addition, a makeup support device according to another aspect of the present invention is A face image acquisition means for acquiring an image including a face; a makeup pattern acquisition means for acquiring a makeup pattern from a plurality of makeup patterns prepared in advance; a simulation image generating means for generating a simulation image in which makeup based on the makeup pattern acquired by the makeup pattern acquiring means is applied to a face included in the image acquired by the face image acquiring means; a simulation image display means for displaying the simulation image generated by the simulation image generation means; The gist of the present invention is as follows.

[0008] According to this, a simulation image is generated and displayed in which makeup based on the makeup pattern acquired by the makeup pattern acquisition means is applied to a face included in an image acquired by the facial image acquisition means (also called a "facial image"), and the user can check the finished state of the makeup applied to his or her face through this simulation image. Here, the pattern of makeup applied to the facial image (face in the image) is acquired from a plurality of makeup patterns prepared in advance, so the user can check various makeup patterns. This makes it possible to check the finished state of makeup according to various makeup patterns through the simulation image.

[0009] The makeup support device according to another aspect of the present invention is the makeup support device according to the first aspect of the present invention, a feature point extraction means for extracting feature points from a face included in the image acquired by the face image acquisition means; a makeup pattern presenting means for selecting and presenting a makeup pattern suitable for a face from which the feature points are extracted, based on the feature points extracted by the feature point extracting means, from among the plurality of makeup patterns; The makeup pattern acquisition means is capable of acquiring the makeup pattern presented by the makeup pattern presentation means. The gist of the present invention is as follows.

[0010] According to this, a makeup pattern suitable for a facial image (a face in an image) is selected from a plurality of makeup patterns and presented based on feature points extracted from the facial image, and the presented makeup pattern is acquired by a makeup pattern acquisition means, making it possible to generate and display a simulation image with a makeup pattern suitable for the user's face. This makes it possible to improve the functionality of the makeup support device by adding a function that recommends a makeup pattern that is thought to suit the user (a so-called "recommendation function").

[0011] The makeup support device according to another aspect of the present invention is the makeup support device according to the first or second aspect of the present invention, The makeup pattern is an eyebrow makeup pattern applied to the eyebrow area of ​​the face. The gist of the present invention is as follows.

[0012] This makes it possible to generate and display a simulation image in which eyebrow makeup is applied based on an eyebrow makeup pattern, thereby enabling eyebrow makeup to be simulated.

[0013] A makeup support device according to another aspect of the present invention is a makeup support device according to any one of the other aspects 1 to 3, an order information generating means for generating order information for a cosmetic product based on a makeup pattern included in the simulation image displayed by the simulation image display means; a transmission means for transmitting the order information generated by the order information generation means to a destination of an order for cosmetics. The gist of the present invention is as follows.

[0014] According to this, cosmetics order information is generated based on the makeup pattern included in the displayed simulation image, i.e., the makeup pattern applied to the face (facial image) in the displayed simulation image (the makeup pattern used as the basis for generating the simulation image), and the order information is sent to the cosmetics ordering destination. This allows a user who has performed a makeup simulation to order cosmetics that suit them (custom-made) based on the results of the simulation.

[0015] The makeup support device of another means 5 related to the present invention is the makeup support device of the other means 4, a makeup condition recognition means for recognizing a state of makeup applied to a face included in an image acquired by the face image acquisition means when the face includes actual makeup; and a second order information generating means for generating order information for the cosmetic product based on the state of the cosmetic product recognized by the cosmetic product state recognizing means, The transmission means is capable of transmitting the order information generated by the second order information generation means to an ordering destination of the cosmetics. The gist of the present invention is as follows.

[0016] According to this, when a face with actual makeup applied is included in a face image, the state of the makeup applied is recognized, and cosmetic order information is generated based on the recognized state of makeup, and the order information is sent to the cosmetics ordering destination. This allows a user to order cosmetics to their liking (custom-made) based on an image of a face with actual makeup applied (also called a "made-up face image"). Note that the state of makeup refers to the state of the makeup, such as the position, shape, and color of the makeup.

[0017] The makeup support device according to another aspect of the present invention is the makeup support device according to the other aspect of the present invention, a face size calculation means for calculating an actual size of a face included in the image acquired by the face image acquisition means, The order information includes information regarding the size calculated by the face size calculation means. The gist of the present invention is as follows.

[0018] According to this, the actual size of the face included in the image acquired by the face image acquisition means is calculated, and information about the calculated size is included in the cosmetics order information, making it possible to order cosmetics suited to the size of the user's face.

[0019] A makeup support device according to another aspect of the present invention is a makeup support device according to any one of the other aspects 4 to 6, The cosmetic product is an eyebrow makeup stamp that can be pressed against the eyebrows of the face to apply makeup to the eyebrows. The gist of the present invention is as follows.

[0020] This allows users to simulate makeup using eyebrow makeup stamps (eyebrow makeup) and order eyebrow makeup stamps based on the results of the simulation. This allows users to order eyebrow makeup stamps that suit them (custom-made eyebrow makeup stamps).

[0021] The cosmetic custom-made system according to another embodiment 8 of the present invention is as follows: Any one of the makeup support devices according to the other means 4 to 7; an order receiving device that receives an order for cosmetics from a user who uses the makeup support device, The makeup support device and the order receiving device are configured to be connectable via a network, The order receiving device includes a receiving means for receiving the order information transmitted by the transmitting means, The cosmetics for the user are manufactured based on the order information received by the receiving means. The gist of the present invention is as follows.

[0022] According to this, a system is constructed in which a makeup support device used by a user and an order receiving device that receives orders for cosmetics are connected via a network, an order for cosmetics (order information) from a user using the makeup support device is received by the order receiving device, and cosmetics for the user are manufactured based on the order (order information). This makes it possible to realize a custom-made cosmetics system that allows users to easily simulate makeup (check the finish) and order cosmetics that suit them.

[0023] A program according to another aspect of the present invention is a program for causing a computer to function as each of the means of the makeup support device according to any one of the other aspects 1 to 7. The gist of the present invention is as follows. Effect of the Invention

[0024] According to the present invention as described above, it is possible to provide a makeup support device, a cosmetics custom-made system, and a program that enable a user to check the finish of makeup without actually using the cosmetics and to order cosmetics that suit the user based on the checked results. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a configuration diagram of a system according to an embodiment. [Diagram 2] FIG. 2 is a configuration diagram of a terminal device. [Diagram 3] FIG. 2 is a configuration diagram of a sales company device. [Figure 4] FIG. 2 is a configuration diagram of a manufacturer device. [Diagram 5] FIG. 4 is a functional block diagram of a processing unit of the terminal device. [Figure 6] 13 is a flowchart of an eyebrow makeup application execution process. [Figure 7] 13 is a flowchart of an eyebrow makeup application execution process. [Figure 8] 13 is a flowchart of an order management process. [Figure 9] 1A is an explanatory diagram of a first process display, and FIG. 1B is an explanatory diagram of a second process display. [Figure 10] (C) is an explanatory diagram of the third step display, and (D) is an explanatory diagram of the fourth step display. [Figure 11] 1 is a table showing eyebrow shape patterns. [Figure 12] FIG. 13 is a diagram showing an example of the configuration of an eyebrow stamp. [Figure 13] 13 is a flowchart of an eyebrow makeup application execution process 2 according to the second modified example. [Figure 14] 13 is a flowchart of an eyebrow makeup application execution process 2 according to the second modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Next, an embodiment of the present invention will be described using an example. EXAMPLES

[0027] This embodiment shows an apparatus that enables simulation of eyebrow makeup for applying makeup to the eyebrows of the face, and a system related to ordering eyebrow makeup stamps using the apparatus.

[0028] [Overall system configuration] As shown in Fig. 1, the system according to the present embodiment includes a terminal device 1 used by a user, a sales company device 2 provided at a sales company that sells eyebrow cosmetic stamps, a manufacturer device 3 provided at a manufacturer that manufactures eyebrow cosmetic stamps, and a network such as the Internet 4. The sales company and the manufacturer may be the same or different.

[0029] The terminal device 1 is a mobile terminal device such as a smartphone or tablet used by a user who simulates eyebrow makeup or orders (purchases) eyebrow makeup stamps. The terminal device 1 can also be configured as a notebook or desktop personal computer (PC). The terminal device 1 is also referred to as a "user device."

[0030] The sales business device 2 is a device such as a computer managed by a business that sells products over the Internet (online shopping) and provides various services associated therewith, and has a server function for operating a website and providing various services. The products referred to here include eyebrow cosmetic stamps. The sales business device 2 is also called a "service providing server."

[0031] The manufacturer device 3 is a device such as a computer managed by a manufacturer of products (eyebrow cosmetic stamps in this embodiment) sold by a sales company, and has a server function for managing manufacturing requests (orders) from the sales company (seller device 2) and information about manufactured products. The manufacturer device 3 is also called a "manufacturing management server." The manufacturer device 3 can include a manufacturing device for the product.

[0032] Each of the seller device 2 and the manufacturer device 3 may be configured with one or more computers or other devices. If the seller and manufacturer are the same business, the seller device 2 and the manufacturer device 3 may be installed under the management of the same business, or the seller device 2 and the manufacturer device 3 may be configured with a single device.

[0033] [Terminal device configuration] Next, the configuration of the terminal device 1 will be described with reference to Fig. 2. As shown in the figure, the terminal device 1 includes a processing unit 10, a storage unit 11, a display unit 12, an operation input unit 13, a communication unit 14, a camera (imaging unit) 15, a battery (power supply unit) 16, and the like.

[0034] The processing unit 10 executes various processes for controlling the operation of the terminal device 1, and is configured with a CPU (Central Processing Unit) etc. The processing unit 10 is also referred to as a "control unit" (the same applies to processing units 20 and 30 described later).

[0035] The memory unit 11 stores the OS (Operating System) that operates the terminal device 1, various application programs that can be executed by the terminal device 1, and various other data necessary for the operation of the terminal device 1, and is composed of a non-volatile memory such as a ROM (Read Only Memory), a volatile memory such as a RAM (Random Access Memory), etc.

[0036] The application programs (application software) stored in the memory unit 11 include programs such as an eyebrow makeup simulation application (also called an "eyebrow makeup application") provided by the sales business device 2, and a Web application (also called a "Web application") that connects to a Web server and enables browsing, operation, etc. of Web sites (Web pages).

[0037] The storage unit 11 also stores data related to the eyebrow makeup app, such as a plurality of eyebrow makeup patterns (a plurality of makeup patterns) provided by the sales company device 2. The eyebrow makeup patterns include an eyebrow shape pattern that indicates the shape of the eyebrows (also called "eyebrow shape") and an eyebrow color pattern that indicates the color of the eyebrows (also called "eyebrow color").

[0038] The display unit 12 displays various images, screens, characters, etc., and is configured, for example, with a liquid crystal display, etc. The display unit 12 has a touch panel function. The display unit 12 is also called a "display panel."

[0039] The operation input unit 13 is composed of buttons, switches, etc. that are operated by the user of the terminal device 1 to input various instructions. The instruction input by the operation input unit 13 includes input by buttons, icons, keyboards, input areas, etc. displayed on the display unit 12 equipped with a touch panel function. A signal (operation signal, instruction signal) based on the instruction input by the operation input unit 13 is input to the processing unit 10.

[0040] The communication unit 14 performs data communication with other devices via the network 4, for example, performing wireless communication in accordance with the IEEE802.11 standard. The other devices include the sales company device 2.

[0041] Camera 15 becomes available when a camera function is activated under the control of processing unit 10, and captures an image of a subject such as a person or a landscape based on an operation (instruction) of a user who uses terminal device 1. Lenses 15a (see FIG. 9(A)) of camera 15 are provided on the front and back surfaces of terminal device 1.

[0042] The battery 16 is composed of a rechargeable battery or the like, and supplies power to each of the components of the mobile terminal 1, such as the processing unit 10, memory unit 11, display unit 12, operation input unit 13, communication unit 14, camera 15, etc.

[0043] [Configuration of the sales company device] Next, the configuration of the sales company device 2 will be described with reference to Fig. 3. As shown in the figure, the sales company device 2 includes a processing unit 20, a storage unit 21, a display unit 22, an operation unit 23, a communication unit 24, and the like.

[0044] The processing unit 20, like the processing unit 10 of the terminal device 1, is configured with a CPU and the like, and executes various processes for controlling the operation of the sales company device 2 (service providing server).

[0045] The memory unit 21 stores the OS (Operating System) that operates the sales company device 2, various application programs, and various other data necessary for the operation of the sales company device 2, and is composed of a ROM (Read Only Memory), which is a non-volatile memory, a RAM (Random Access Memory), which is a volatile memory, a hard disk drive (HDD), which is an auxiliary storage device, etc.

[0046] The data stored in the storage unit 21 includes data necessary for operating websites and providing various services, etc. Specifically, the data includes a program for operating websites such as a website providing an eyebrow makeup app and a website selling eyebrow makeup stamps, a program for the eyebrow makeup app that is installed and executed on the terminal device 1, and a plurality of types of eyebrow makeup patterns that can be used in the eyebrow makeup app.

[0047] In this embodiment, an eyebrow makeup pattern database DB1 that manages (stores) a plurality of types of eyebrow makeup patterns is provided, and the database DB1 is stored in the storage unit 21. The eyebrow makeup pattern database DB1 stores a plurality of eyebrow shape patterns (six types of patterns P1 to P6 in this embodiment) with different eyebrow shapes, for example, as shown in Fig. 11. In addition, data (eyebrow color patterns) of a plurality of eyebrow colors (three colors, gray, natural brown, and dark brown in this embodiment) that can be set (used) for each eyebrow shape is also stored.

[0048] The types and number of eyebrow makeup patterns (eyebrow shape patterns, eyebrow color patterns) managed in the eyebrow makeup pattern database DB1 can be changed (updated) or increased or decreased by the seller operating the seller device 2. Therefore, for example, it is possible to add a trendy eyebrow makeup pattern after the fact, or delete an eyebrow makeup pattern that is losing popularity after the fact.

[0049] The types and number of eyebrow makeup patterns are not limited to those shown in this embodiment, and more patterns may be provided.

[0050] The storage unit 21 also stores a customer information database DB2 and an order information database DB3.

[0051] The customer information database DB2 is a database that manages (stores) information on users (customers) who have ordered (purchased) eyebrow makeup stamps using the eyebrow makeup application.

[0052] The order information database DB3 is a database that manages (stores) information about the order contents when ordering eyebrow makeup stamps using the eyebrow makeup application (also referred to as "order information").

[0053] A user of the terminal device 1 can operate the terminal device 1 to connect to the sales provider device 2 (service providing server), download data required for the operation of the eyebrow makeup app, such as the program of the eyebrow makeup app and eyebrow makeup patterns (also referred to as "eyebrow makeup app related data"), and install the data in the terminal device 1, thereby executing (using) the eyebrow makeup app on the terminal device 1. In addition, update programs for the eyebrow makeup app and update data for eyebrow makeup patterns can also be connected to the sales provider device 2, downloaded, and installed (updated) in the terminal device 1.

[0054] In addition, when the sales business device 2 is configured to include two or more (multiple) computers (servers), the various data exemplified as being stored in the memory unit 21 may be distributed and managed (stored) in the multiple computers.

[0055] The display unit 22 displays various images, screens, characters, etc., and is configured with a liquid crystal display or the like.

[0056] The operation unit 23 is operated to input various data and instructions, and is composed of a keyboard, a mouse, and the like.

[0057] The communication unit 24, like the communication unit 14 of the terminal device 1, performs data communication with other devices and the like via the network 4. The other devices include the terminal device 1 and the manufacturer device 3.

[0058] [Manufacturer device configuration] Next, the configuration of the manufacturer device 3 will be described with reference to Fig. 4. As shown in the figure, the manufacturer device 3 includes a processing unit 30, a storage unit 31, a display unit 32, an operation unit 33, a communication unit 34, and the like.

[0059] The processing unit 30, the storage unit 31, the display unit 32, the operation unit 33, and the communication unit 34 are basically configured in the same manner as the processing unit 20, the storage unit 21, the display unit 22, the operation unit 23, and the communication unit 24 of the sales business device 2. Therefore, detailed explanations of the configurations similar to those of the sales business device 2 will be omitted.

[0060] In addition to various data necessary for the operation of the manufacturer device 3, the storage unit 31 stores an order management database DB4 and a product management database DB5.

[0061] The order management database DB4 is a database that manages information (also referred to as "order information") based on a manufacturing request (order information) when order information sent from the terminal device 1 to the sales company device 2 is sent to the manufacturer device 3 and a manufacturing request for a product (in this embodiment, an eyebrow makeup stamp) is accepted upon receipt of the order information by the manufacturer device 3.

[0062] The product management database DB5 is a database that manages information (also referred to as "product information") regarding manufacturing requests (order information) from the sales business device 2, i.e., goods (products) manufactured based on the order information managed in the order management database DB4.

[0063] [Eyebrow makeup app execution process] Next, the eyebrow makeup application execution process (S100) executed by the terminal device 1 in which the eyebrow makeup application related data is installed will be described with reference to the flowcharts shown in Fig. 6 and Fig. 7. In this specification, "S" in S100 and the like ("S" in the flowchart) means a step.

[0064] The eyebrow makeup application execution process (S100) is executed (started) by the processing unit 10 when the user activates the eyebrow makeup application by operating the terminal device 1. The eyebrow makeup application can be activated, for example, by tapping an eyebrow makeup application icon (not shown) displayed on the display unit 12 of the terminal device 1 in which eyebrow makeup application related data has been installed.

[0065] The processing unit 10 of the terminal device 1 in which the eyebrow makeup application related data is installed functions as each of the means 101 to 112 shown in Fig. 5 by starting (executing) the eyebrow makeup application. This terminal device 1 corresponds to an example of the "makeup support device" of the present invention. Note that the instruction input by the user (instruction input by the operation input unit 13) while executing (using) the eyebrow makeup application is detected by the instruction input detection means 101.

[0066] As shown in Fig. 6, in the eyebrow makeup application execution process (S100), first, an application screen is displayed on the display unit 12 of the terminal device 1 (S101), and the camera 15 is started (S103). The display content (also referred to as "first step display") on the display unit 12 (application screen) at this time is shown in Fig. 9(A).

[0067] As shown in the figure, the display unit 12 on which the application screen is displayed is divided into a main display area 12a and a sub-display area 12b having a smaller display range (area size) than the main display area 12a. That is, the application screen is roughly divided into the main display area 12a and the sub-display area 12b. In this embodiment, the screen configuration is such that the sub-display area 12b is disposed below the main display area 12a. Note that the configuration of the application screen, including the display contents described below, is not limited to this embodiment, and can be set appropriately according to the size (screen size) and shape of the display unit (display screen) of the terminal device (user device).

[0068] In the first step display, an image captured by the camera 15 through the lens 15a (here, an image including the user's face) is displayed in the main display area 12a. In addition, an OK button BT1 for inputting an image capture instruction and a selection button BT2 for inputting an instruction to select data to be used in the eyebrow makeup simulation from data such as photographs stored in the storage unit 11 (also called a "selection instruction") are displayed in the sub-display area 12b.

[0069] In the first step display, the user can tap the OK button BT1 or the selection button BT2 to select whether to perform the eyebrow makeup simulation based on the face imaged at that time or based on a face imaged in the past.

[0070] Returning to FIG. 6, next, in S105, in the first step display, it is determined whether or not an image capture instruction has been input by tapping the OK button BT1 (S105). As a result, when it is determined that an image capture instruction has been input (YES in S105), an image of an image capture target (here, the user's face) is captured by the camera 15 (S107). The processing of S107 is performed by the processing unit 10 functioning as the image capture means 102.

[0071] On the other hand, if it is determined that an image capture instruction has not been input (NO in S105), it is determined whether or not a selection instruction has been input by tapping the selection button BT2 (S109). As a result, if it is determined that a selection instruction has not been input (NO in S109), the process returns to S105, and if it is determined that a selection instruction has been input (YES in S109), data selection processing is performed (S111).

[0072] The data selection process (S111) is a process in which a list of data such as photographs stored in the memory unit 11 is displayed on the display unit 12 (not shown), and one piece of data is selected from the list based on a selection operation (selection instruction input) by the user.

[0073] Following the process of S107 or S111, in S113, an image including a face (face image) is acquired from the captured data (captured data) or selected data (selected data) (S113), and the acquired face image is displayed in the main display area 12a (S115), and an eyebrow shape sample icon SA1 and an eyebrow color sample icon SA2 based on the eyebrow makeup pattern stored in the storage unit 11 are displayed in the sub-display area 12b (S117). The eyebrow shape sample icon SA1 and the eyebrow color sample icon SA2 are collectively referred to as "eyebrow makeup samples".

[0074] The processes of S113, S115, and S117 are performed by the processing unit 10 functioning as the face image acquiring means 103, the face image display means 104, and the makeup pattern display means 105, respectively. The eyebrow makeup samples displayed in the sub-display area 12b indicate the eyebrow makeup patterns stored in the storage unit 11.

[0075] FIG. 9B shows the display content (also called "second step display") of the display unit 12 (application screen) when the face image and eyebrow makeup sample are displayed.

[0076] As shown in the figure, the face image FA displayed in the main display area 12a is displayed as a still image showing the front face of the user. Also, the eyebrow shape sample icon SA1 displayed in the sub-display area 12b shows an eyebrow shape based on the eyebrow makeup pattern (eyebrow shape pattern) stored in the storage unit 11.

[0077] In this embodiment, three eyebrow shape sample icons SA1 corresponding to three of the six patterns P1 to P6 shown in FIG. 11 are displayed in the secondary display area 12b, and the user can select a preferred eyebrow shape by tapping any one of the eyebrow shape sample icons SA1.

[0078] In addition, by swiping left and right on the display portion of the eyebrow shape sample icon SA1 on the display unit 12 (display panel), the eyebrow shape sample icon SA1 can be scrolled left and right within the sub-display area 12b, thereby changing the type of eyebrow shape sample icon SA1 displayed in the sub-display area 12b, i.e., the type of eyebrow makeup pattern (eyebrow shape).

[0079] FIG. 9B shows a state in which eyebrow shape sample icons SA1 corresponding to P4 to P6 out of the six patterns P1 to P6 shown in FIG. 11 are displayed.

[0080] In the second step display, the eyebrow color sample icon SA2 displayed in the secondary display area 12b indicates the eyebrow color based on the eyebrow makeup pattern (eyebrow color pattern) stored in the storage unit 11. In this embodiment, three eyebrow color sample icons SA2 corresponding to three types of eyebrow colors (gray, natural brown, dark brown) are displayed in the secondary display area 12b, and the user can select a preferred eyebrow color by tapping any one of the eyebrow color sample icons SA2.

[0081] In this embodiment, the eyebrow color sample icon SA2 is not active (enabled) until the eyebrow shape sample icon SA1 is tapped to select the eyebrow shape, and the eyebrow color sample icon SA2 is active (enabled) when the eyebrow shape is selected. This is to prioritize the selection of the eyebrow shape over the selection of the eyebrow color.

[0082] Returning to Fig. 6, next, in S119, it is determined whether or not an instruction to select an eyebrow makeup sample has been input in the second step display (S119). Here, it is determined whether or not both an instruction to select an eyebrow shape by tapping the eyebrow shape sample icon SA1 and an instruction to select an eyebrow color by tapping the eyebrow color sample icon SA2 have been input (whether or not the selection of eyebrow shape and eyebrow color has been completed).

[0083] If it is determined in S119 that an instruction to select an eyebrow makeup sample has not been input (NO in S119), the process waits until an instruction to select an eyebrow makeup sample is input. On the other hand, if it is determined that an instruction to select an eyebrow makeup sample has been input (YES in S119), the process reads and acquires an eyebrow makeup pattern corresponding to the selected eyebrow makeup sample from the storage unit 11 (S121). The process of S121 is performed by the processing unit 10 functioning as the makeup pattern acquisition means 106.

[0084] Next, in S123, a simulation image is generated in which makeup (virtual makeup) based on the eyebrow makeup pattern acquired in S121 is applied to the eyebrow portion of the face image FA being displayed in the main display area 12a, i.e., the face image acquired in S113 (S123). The eyebrow makeup simulation image can be generated, for example, by combining the face image FA being displayed in the display area 12a (the face image acquired in S113) with an eyebrow image based on the eyebrow makeup pattern (eyebrow shape pattern, eyebrow color pattern) acquired in S121. The processing of S123 is performed by the processing unit 10 functioning as the simulation image generating means 107.

[0085] Next, in S125, the simulation image generated in S123 is displayed in the main display area 12a (S125). The display content (also referred to as "third step display") on the display unit 12 (application screen) at this time is shown in FIG.

[0086] As shown in the figure, a simulation image SM is displayed in the main display area 12a, and an eyebrow shape sample icon SA1 and an eyebrow color sample icon SA2 are displayed in the sub-display area 12b, similar to the second step display (see FIG. 9(B)).

[0087] The simulation image SM shown in Fig. 10(C) shows a case where eyebrow makeup based on pattern P5 is applied to the eyebrow portion of the face image FA shown in Fig. 9(B). The processing of S125 is performed by the processing unit 10 functioning as the simulation image display means 108.

[0088] In the third step display, in addition to the simulation image SM, the main display area 12a displays guide lines GU, which serve as a guide for adjusting the position and balance of eyebrows after makeup (makeup eyebrows), a guide setting button BT3 that allows the guide lines GU to be switched between hidden (OFF) and displayed (ON), and a decision button BT4 for inputting a decision instruction to finalize the results of the eyebrow makeup simulation. By default, the guide lines GU are displayed, and by tapping the guide setting button BT3, the user can select whether to hide (OFF) or display (ON) the guide lines GU.

[0089] The guide lines GU consist of one vertical guide line GU1 extending vertically (vertically) between the left and right eyebrows (middle), and two horizontal guide lines GU2, GU3 extending horizontally (horizontally) above and below the eyebrows.

[0090] The user can move the eyebrow image included in the simulation image SM by touching with a finger a portion of the simulation image SM corresponding to the eyebrow portion (eyebrow image) of the simulation image SM on the display unit 12 (display panel) on which the simulation image SM is displayed and dragging and dropping the portion. Also, the user can enlarge or reduce the size (size) of the eyebrow image included in the simulation image SM by touching with a finger a portion of the simulation image SM corresponding to the eyebrow portion (eyebrow image) and pinching out or in. That is, an instruction to adjust the eyebrow image is input by a touch operation such as dragging and dropping, pinching out, or pinching in. At this time, if the guide lines GU are displayed, the position, balance, etc. of the eyebrow image can be adjusted using the guide lines GU as a reference (guideline). Such display (display control) of the guide lines GU is achieved by the processing unit 10 functioning as the guide display means 109.

[0091] In addition, when adjusting the eyebrow image, the adjustable range of the eyebrow image may be limited (restricted) within the range of the guide line GU. For example, in FIG. 10(C), the eyebrow image R corresponding to the right eyebrow may be movable within the range of two horizontal guide lines GU2 and GU3 on the left side of the vertical guide line GU1, and the eyebrow image L corresponding to the left eyebrow may be movable within the range of two horizontal guide lines GU2 and GU3 on the right side of the vertical guide line GU1. This prevents the balance of the appearance of the eyebrow position and the like from becoming unnatural, and allows for a simulation of eyebrow makeup that looks good. In addition, with regard to the adjustment of the eyebrow image, in addition to adjustment (changing) the position and size of the eyebrow image, adjustment of the shape of the eyebrow such as thickness and length may also be performed by touch operation. This makes it possible to improve the quality and accuracy of the eyebrow makeup simulation.

[0092] In addition, in the third step display, by tapping the eyebrow shape sample icon SA1 or the eyebrow color sample icon SA2 displayed in the secondary display area 12b, an instruction to select an eyebrow makeup sample can be input. In other words, the eyebrow shape or eyebrow color can be changed even after the simulation image SM is displayed. Note that after the simulation image SM is displayed, both sample icons SA1 and SA2 are active (enabled), and neither is given priority.

[0093] Next, in S127 shown in Fig. 7, in the third display step (simulation image display screen), it is determined whether or not an instruction to select an eyebrow makeup sample has been input (S127). Here, it is determined whether or not an instruction to select an eyebrow shape by tapping the eyebrow shape sample icon SA1 or an instruction to select an eyebrow color by tapping the eyebrow color sample icon SA2 has been input. As a result, if it is determined that an instruction to select an eyebrow makeup sample has been input (YES in S127), the process returns to S121 (see Fig. 6) and the process from S121 is performed again.

[0094] On the other hand, if it is determined that the instruction to select the eyebrow makeup sample has not been input (NO in S127), it is determined whether or not an instruction to adjust the eyebrow image has been input (S129). The instruction to adjust the eyebrow image is input by an operation such as drag and drop, pinch out, pinch in, etc. on the part corresponding to the eyebrow image as described above.

[0095] If it is determined in S129 that an instruction to adjust the eyebrow image has been input (YES in S129), a process of moving, enlarging, or reducing the eyebrow image included in the simulation image SM in accordance with the input (eyebrow image adjustment process) is performed (S131). The process of S131 is performed by the processing unit 10 functioning as the makeup pattern adjustment means 110.

[0096] If it is determined in S129 that an instruction to adjust the eyebrow image has not been input (NO in S129), it is determined whether or not a confirmation instruction has been input by tapping the decision button BT4 (S133). As a result, if it is determined that a confirmation instruction has not been input (NO in S133), the process returns to S127.

[0097] On the other hand, if it is determined that the decision instruction has been input (YES in S133), a confirmation screen for allowing the user to confirm the decided (confirmed) result of the eyebrow makeup simulation is displayed on the display unit 12 (S135). The display content of the display unit 12 at this time (also referred to as the "fourth step display") is shown in Fig. 10(D).

[0098] As shown in the figure, in the fourth step display (confirmation screen), a simulation image SM is displayed in the main display area 12a as a result of eyebrow makeup simulation. In this case, the display contents of the main display area 12a are that the guide line GU, the guide setting button BT3, and the decision button BT4 displayed in the main display area 12a in the third step display (see FIG. 10(C)) are hidden, and the simulation image SM displayed in the third step display immediately before the transition to the fourth step display is displayed as a result of eyebrow makeup simulation. The processing of S135 related to the display of the simulation image SM in the fourth step display (confirmation screen) is also performed by the processing unit 10 functioning as the simulation image display means 108.

[0099] In addition, in the fourth step display, the secondary display area 12b displays a message such as "Are you sure?", a back button BT5 for inputting an instruction to return to the previous step display (third step display) (also called a "return instruction"), and an order button BT6 for inputting an instruction to order an eyebrow makeup stamp that can be used to apply the eyebrow makeup shown in the simulation image SM (eyebrow makeup simulation result) being displayed in the primary display area 12a (also called an "order instruction").

[0100] Returning to Fig. 7, next in S137, it is determined whether an order instruction has been input by tapping the order button BT6 (S137). As a result, if it is determined that an order instruction has not been input (NO in S137), it is determined whether a return instruction has been input by tapping the back button BT5 (S139), and if it is determined that a return instruction has been input (YES in S139), the process returns to S125 (see Fig. 6) and the process from S125 is performed again. At this time, the display content on the display unit 12 of the terminal device 1 becomes that shown in Fig. 10(C) (third step display).

[0101] On the other hand, when it is determined in S139 that the return instruction has not been input (NO in S139), it is determined whether or not an instruction to end the eyebrow makeup app has been input (S141). The instruction to end the eyebrow makeup app can be given, for example, by swiping the app screen of the eyebrow makeup app displayed on the display unit 12 of the terminal device 1 in a predetermined direction (e.g., upward) or by displaying a home screen (not shown) on the display unit 12.

[0102] If it is determined in S141 that an end instruction has not been input (NO in S141), the process returns to S137, and if it is determined that an end instruction has been input (YES in S141), an application end process is performed to end the eyebrow makeup application (S155). The eyebrow makeup application is ended in this manner without issuing an order instruction (S137) when the user only performs an eyebrow makeup simulation.

[0103] On the other hand, when it is determined in the above-mentioned S137 that an order instruction has been input (YES in S137), order information relating to an order for an eyebrow makeup stamp based on the result of this eyebrow makeup simulation is generated (S143), and an order screen (not shown) is displayed on the display unit 12 (S145). The processing of S143 is performed by the processing unit 10 functioning as the order information generating means 111.

[0104] The order screen (not shown) displays, for example, the order information generated in S143, input fields for inputting information about the user who is the orderer (purchaser) (also referred to as "user information"), and a send button for inputting an instruction to send the user information and order information (also referred to as a "send instruction"). Once the required items have been entered into the input fields, the send button becomes active (enabled), and the send instruction can be input by tapping the send button. By making the send button active (enabled) upon completion of input into the input fields, it is possible to prevent erroneous sending of information.

[0105] The order information is generated based on the eyebrow makeup pattern (eyebrow makeup pattern) included in the simulation image SM (eyebrow makeup simulation result) displayed in the main display area 12a in the fourth step display (confirmation screen). Here, if the size, shape, etc. of the eyebrow image are adjusted (changed) by the above-mentioned eyebrow image adjustment process (S131), the adjusted (changed) eyebrow makeup pattern is reflected in the generation of the order information.

[0106] The order information includes information indicating, for example, the order date, product name, product specifications (eyebrow shape, eyebrow color, size, etc.), product unit price, quantity, product cost, etc. The "product specifications" in the order information specifies the eyebrow makeup pattern (eyebrow makeup pattern) included in the simulation image SM (eyebrow makeup simulation result). The user information includes information indicating the name, address (delivery destination of the product), contact information (telephone number, email address, etc.), payment method, etc. of the user who ordered (purchased) the product (here, eyebrow makeup stamp). Note that the order information and user information may be collectively referred to as "order information." Hereinafter, the order information and user information are also collectively referred to as "order information."

[0107] Next, in S147, it is determined whether a transmission instruction has been input by tapping a transmission button (not shown) (S147). As a result, if it is determined that a transmission instruction has not been input (NO in S147), it is determined whether an instruction to end the eyebrow makeup application has been input (S151). If it is determined that an instruction has not been input (NO in S151), the process returns to S147, and if it is determined that an instruction has been input (YES in S151), application end processing is performed (S155). The eyebrow makeup application is ended without transmitting order information when the user only performs an eyebrow makeup simulation.

[0108] On the other hand, when it is determined that a transmission instruction has been input in S147 (YES in S147), the order information is transmitted to the order destination (the sales company device 2 in this embodiment) (S149). The processing in S149 is performed by the processing unit 10 functioning as the transmission means 112.

[0109] Next, in S153, it is determined whether or not a command to end the eyebrow makeup application has been input (S153). If it is determined that a command to end the eyebrow makeup application has not been input (NO in S153), the process waits until the command to end the application is input. If it is determined that the command to end the application has been input, the process to end the application is performed (S155).

[0110] [Order Management Processing] Next, the order management process (S200) executed by the sales business operator device 2 (service providing server) will be described with reference to the flowchart shown in Fig. 8. The order management process (S200) is a process executed by the processing unit 20.

[0111] 8, in the order management process (S200), first, it is determined whether or not the order information from the terminal device 1, i.e., the order information transmitted in the process of S149 described above, has been received (S201). As a result, if it is determined that the order information has not been received (NO in S201), this process is terminated without performing the processes in and after S203.

[0112] On the other hand, if it is determined that the order information has been received (YES in S201), an order reception number is issued (generated) and the order reception number is returned to the terminal device 1 that sent the order information (S203). This allows the user to confirm that the current order has been properly received. The order reception number can be issued (generated) automatically by a program. The processing of S201 is performed by the processing unit 20 functioning as a receiving means.

[0113] Next, in S205, the user information among the order information received in S201 is registered (stored) as customer information in the customer information database DB2 (S205). This enables the sales company to manage information (age, order history, etc.) about users (customers) who have ordered (purchased) eyebrow makeup stamps using the eyebrow makeup app.

[0114] The registration of information in the customer information database DB2 in S205 may be performed by a program or may be performed manually (manually) by a person in charge (operator) of the sales company.

[0115] Next, in S207, the order information received in S201 is registered (stored) in the order information database DB3 (S207). Here, the order information is linked (associated) with the order reception number issued (generated) in S203 and the customer information (user information) registered in the customer information database DB2 in S205, and registered (stored) in the order information database DB3. This enables the sales company to manage information about the eyebrow makeup stamps ordered using the eyebrow makeup app (order date, eyebrow makeup stamp specifications, order history, etc.).

[0116] The registration of information in the order information database DB3 in S207 may be performed by a program or may be performed manually (manually) by a person in charge (operator) of the sales company.

[0117] Next, in S209, a process is executed to request the manufacturer (manufacturer device 3) to manufacture the eyebrow cosmetic stamp based on the order information received in S201 (S209). This manufacturing request can be made by any method, such as transferring the order information received in S201 to the manufacturer device 3, converting the order information received in S201 into a format that can be received (accepted) by the manufacturer device 3 and sending it to the manufacturer device 3, or inputting information about the product to be requested to be manufactured (information about the eyebrow cosmetic stamp based on the order information received in S201) into a dedicated input form (manufacturing request form) that can be used between the sales company and the manufacturer.

[0118] The above is a specific processing content of the order management processing (S200) by the seller device 2 (processing unit 20), but when the manufacturing request is sent to the manufacturer device 3 by the processing of S209 and the manufacturing request is accepted by the manufacturer device 3 (manufacturer), information based on the manufacturing request is managed as order information in the order management database DB4. The order information includes information indicating, for example, the order date, the order origin (seller), the delivery destination (the address and name of the user who is the purchaser), the product name, the specifications of the product (eyebrow cosmetic stamp) (eyebrow shape, eyebrow color, size, etc.), the product unit price, quantity, product price, etc. The order information also includes information that overlaps with the order information.

[0119] The manufacturer manufactures the eyebrow cosmetic stamp based on the order information managed in the order management database DB4. Specifically, the manufacturer manufactures the eyebrow cosmetic stamp based on the eyebrow shape specified based on the order information managed in the order management database DB4, i.e., the order information (manufacturing request) from the sales company device 2. Also, the manufacturer prepares a cosmetic powder based on the eyebrow color specified based on the order information (manufacturing request) from the sales company device 2, and impregnates a sponge or the like with the prepared powder to manufacture a powder for the eyebrow cosmetic stamp (not shown). The eyebrow cosmetic stamp and the powder for the eyebrow cosmetic stamp manufactured in this manner are stored in a dedicated storage case (not shown) and delivered to the user. In this embodiment, the eyebrow cosmetic stamp ordered using the eyebrow cosmetic app are delivered as a set with the powder for the eyebrow cosmetic stamp.

[0120] In the product management database DB5, information about the products (eyebrow stamp, powder for eyebrow stamp) manufactured by the manufacturer is managed as product information. The product information includes, for example, information indicating the specifications (eyebrow shape, eyebrow color, size, etc.) of the product (eyebrow stamp, powder for eyebrow stamp), delivery date, order origin (sales company), delivery destination, etc.

[0121] When a manufacturer receives a request (additional order) from a seller or user to manufacture the same product (eyebrow cosmetic stamp, powder for eyebrow cosmetic stamp) as the product that the manufacturer manufactured earlier, the manufacturer can manufacture the same product (eyebrow cosmetic stamp, powder for eyebrow cosmetic stamp, or a set of these) based on the product information managed in the product management database DB5. Also, when a manufacturer receives an order from a seller or user later for replacement parts (consumables) for the product that the manufacturer manufactured earlier (eyebrow cosmetic stamp, powder for eyebrow cosmetic stamp), the manufacturer can manufacture the corresponding replacement parts (consumables) based on the product information managed in the product management database DB5.

[0122] A custom-made system for eyebrow makeup stamps (corresponding to an example of the "custom-made cosmetics system" of the present invention) is constructed by connecting the sales business device 2 that executes the above-mentioned order management process (S200) and the terminal device 1 that executes the above-mentioned eyebrow makeup application execution process (S100) via a network 4. In this case, the sales business device 2 corresponds to an example of the "order receiving device" of the present invention.

[0123] Here, an example of the configuration of the eyebrow cosmetic stamp will be described with reference to Fig. 12. As shown in the figure, the eyebrow cosmetic stamp 500 is composed of a main body 501 that is horizontally long (rectangular) in plan view, and a stamp member 502 that is attached to the bottom surface of the main body 501. Note that, although Fig. 12 shows the eyebrow cosmetic stamp 500 for the left eyebrow, it is actually sold as a set for the left and right eyebrows, and the eyebrow cosmetic stamp for the right eyebrow appears symmetrically to the eyebrow cosmetic stamp 500 for the left eyebrow shown in Fig. 12.

[0124] The main body 501 is made of transparent plastic etc. On both the left and right ends of the main body 501, there are provided gripping parts 501a for gripping with fingers when using the eyebrow makeup stamp 500.

[0125] The stamp member 502 is made of porous rubber, sponge, etc., and is molded into an eyebrow shape according to the eyebrow shape pattern. One side of the stamp member 502 is a stamp surface (stamp portion) 502a to be pressed against the eyebrow area of ​​the face.

[0126] When using the eyebrow makeup stamp 500, the user pinches the pinched parts 501a at both ends with the fingers, and presses (stamps) the stamp surface 502a, which has eyebrow makeup powder on it, onto the eyebrows of the face reflected in the mirror. This causes the eyebrow makeup powder on the stamp surface 502a to be applied to the eyebrows, and eyebrow makeup is applied. The main body 501 of the eyebrow makeup stamp 500 shown in this configuration example is transparent, so that the user can easily grasp the position to stamp.

[0127] [Effects of the embodiment] In the terminal device 1 (makeup support device) in which the eyebrow makeup application related data of the present embodiment described above is installed, a simulation image SM is generated in which eyebrow makeup is applied to the eyebrow portion of the face included in the face image acquired in the process of S113 (face image acquisition means 103) based on the eyebrow makeup pattern acquired in the process of S121 (makeup pattern acquisition means 106), and is displayed on the display unit 12 (main display area 12a). Therefore, the user can easily simulate eyebrow makeup by operating the terminal device 1, and can check the finish of the eyebrow makeup applied to his / her face through the simulation image SM.

[0128] Here, eyebrow makeup patterns (eyebrow makeup patterns) to be applied to the eyebrows of the face image (face in the image) are provided by the sales business device 2 (service providing server) and stored (prepared) in the storage unit 11 of the terminal device 1, and the user can select a desired pattern from the multiple eyebrow makeup patterns stored in the storage unit 11. Then, the eyebrow makeup pattern selected by the user is acquired, and a simulation image SM with eyebrow makeup based on the eyebrow makeup pattern is displayed, so that the user can try and check various makeup patterns. This makes it possible to check the finish of makeup with various makeup patterns through the simulation image SM.

[0129] In this embodiment, order information for the eyebrow makeup stamp is generated based on the eyebrow makeup pattern included in the simulation image SM displayed on the display unit 12 (main display area 12a) of the terminal device 1, i.e., the eyebrow makeup pattern applied to the face (face image) in the simulation image SM (the eyebrow makeup pattern used as the basis for generating the simulation image SM), and the order information is sent to the sales company device 2 (the ordering destination of the eyebrow makeup stamp). This allows a user who has performed a simulation of eyebrow makeup using the eyebrow makeup app (terminal device 1) to order an eyebrow makeup stamp that suits them based on the results of the simulation, i.e., to have an eyebrow makeup stamp custom-made.

[0130] In this embodiment, a terminal device 1 used by a user and a seller device 2 (order receiving device) that receives orders for eyebrow cosmetic stamps are connected via a network 4, and an order (order information) for eyebrow cosmetic stamps from a user using the terminal device 1 is received by the seller device 2, and a system is constructed in which an eyebrow cosmetic stamp for the user is manufactured by a manufacturer based on the order (order information). This makes it possible to realize a custom-made eyebrow cosmetic stamp system that allows users to easily simulate eyebrow makeup (check the finished look) and order eyebrow cosmetic stamps that suit them.

[0131] There are several possible variations of the embodiment described above. The variations will be described below. In the following description, the differences from the embodiment described above will be mainly described, and the commonalities between the embodiment and the embodiment will be omitted.

[0132] [Variation 1] Variation 1 is a configuration in which facial feature points are extracted from the facial image acquired by the aforementioned processing of S113 (facial image acquisition means) and eyebrow makeup patterns (eyebrow makeup samples) suited to the user's face are presented based on the extracted feature points (so-called "recommendation function") to the aforementioned embodiment.

[0133] The program of the eyebrow makeup application (data related to the eyebrow makeup application) according to the first modification includes an algorithm for recognizing and extracting facial feature points, such as the overall shape of the face included in the face image, and the positions and shapes of the elements (features) that constitute the face, such as the eyes, nose, mouth, etc. As a technique for extracting facial feature points, a known facial image feature point extraction technique can be used.

[0134] Moreover, the eyebrow makeup patterns (eyebrow makeup patterns managed in the eyebrow makeup pattern database DB1) stored in the storage unit 11 as data related to the eyebrow makeup application are created according to various facial features (particularly, the overall face shape, eye shape, skin color, etc.), and are stored after being classified (grouped) according to the features. In this case, the features (feature points) can be, for example, a round face, an inverted triangle face, a long face, a square face, an oval face, a square face, etc. for the face, round eyes, almond eyes, droopy eyes, upturned eyes, single eyelids, double eyelids, deep-set eyelids, etc. for the eyes, and fair skin, dark skin, etc. for the skin color.

[0135] The processing unit 10 of the terminal device 1 in which the eyebrow makeup application related data according to the first modification is installed acquires a face image by the process of S113, and then extracts facial feature points included in the face image acquired in S113. This feature point extraction process is performed by the processing unit 10 functioning as a feature point extraction means.

[0136] Next, the processing unit 10 acquires one or more eyebrow makeup patterns corresponding to the extracted facial feature points, i.e., eyebrow makeup patterns suitable for the face from which the feature points were extracted, from among the eyebrow makeup patterns stored in the storage unit 11, and displays an eyebrow makeup sample based on the acquired eyebrow makeup pattern in the secondary display area 12b. The processing unit 10 functions as a makeup pattern presenting means to display this eyebrow makeup sample, and the above-mentioned process of S117 (makeup pattern display means 105) can also be said to have a function of presenting eyebrow makeup patterns (recommendation function).

[0137] Next, when one of the eyebrow makeup samples (eyebrow makeup patterns) displayed (presented) in the secondary display area 12b is selected according to the facial feature points (S119), the processing unit 10 reads and acquires the eyebrow makeup pattern corresponding to the selected eyebrow makeup sample from the memory unit 11, generates a simulation image with eyebrow makeup applied based on the acquired eyebrow makeup pattern, and displays it in the primary display area 12a (S121 to S125).

[0138] According to the above-mentioned modification 1, eyebrow makeup patterns (eyebrow makeup samples) suited to the facial features of the user are presented, and it becomes possible to generate and display a simulation image or order eyebrow makeup stamps using an eyebrow makeup pattern selected from the presented patterns. This makes it possible to improve the functionality of the eyebrow makeup app by adding a function (recommendation function) that recommends eyebrow makeup patterns (eyebrow makeup samples) suited to the user.

[0139] The presentation of eyebrow makeup patterns according to facial feature points (recommendation function) may be performed as follows.

[0140] For example, information on currently popular eyebrow makeup and / or information on users who have used an eyebrow makeup app to perform eyebrow makeup simulations or order eyebrow makeup stamps (age, order history, facial features, etc.) may be accumulated and trained on an AI (artificial intelligence), various eyebrow makeup patterns may be created based on the learning results, and an eyebrow makeup pattern corresponding to the facial image features acquired in the process of S113 may be acquired and presented from the created eyebrow makeup patterns. In this case, the learning by the AI ​​(artificial intelligence) may be performed, for example, on the sales company device 2 (service providing server) side, and the user may operate the terminal device 1 to access the sales company device 2 (service providing server) and periodically download the latest eyebrow makeup patterns (eyebrow makeup app related data) and install them in the terminal device 1. This makes it possible to present eyebrow makeup patterns that are more suitable for the user.

[0141] [Variation 2] In the second modification, when the face in the face image acquired by the process of S113 (face image acquisition means) has actual eyebrow makeup applied, the state of the applied eyebrow makeup is recognized, and order information for the eyebrow makeup stamp is generated based on the recognition result, to the above-mentioned embodiment. Note that the second modification can also be applied to the first modification.

[0142] The program of the eyebrow makeup app (data related to the eyebrow makeup app) according to the second modification includes an algorithm for recognizing the state of eyebrow makeup when makeup (eyebrow makeup) is applied to the eyebrow portion of the face in the face image. The state of eyebrow makeup includes at least the shape (contour) and color of the eyebrows (makeup eyebrows) that constitute the eyebrow makeup. As a technology for recognizing the state of eyebrow makeup, a known face image recognition technology can be used. For example, the state of eyebrow makeup can be recognized by identifying the part where makeup is applied (e.g., the eyebrow portion) based on the difference (difference) between the natural skin color and the makeup color in the face image and extracting the shape, color, etc. of the identified part.

[0143] The processing unit 10 of the terminal device 1 having installed therein the eyebrow makeup application related data according to the second modification displays a menu screen (not shown) on the display unit 12 of the terminal device 1 to allow the user to select whether to perform eyebrow makeup simulation or eyebrow makeup state recognition when the eyebrow makeup application is started. When the eyebrow makeup simulation is selected, the processing unit 10 executes the above-mentioned eyebrow makeup application execution process (S100). On the other hand, when the eyebrow makeup state recognition is selected, the processing unit 10 executes the eyebrow makeup application execution process 2 (S1000) shown in Fig. 13 and Fig. 14. The eyebrow makeup application execution process 2 (S1000) will be described below.

[0144] [Eyebrow makeup app execution process 2] Of the eyebrow makeup application execution process 2 (S1000), the processes of S1001 to S1013 shown in Fig. 13 are similar to the processes of S101 to S113 in the above-mentioned embodiment. Note that the face image acquired in S1013 is a face image (made-up face image) in which actual makeup (at least eyebrow makeup in this case) is applied to the user's face.

[0145] In S1015 following S1013, the state of eyebrow makeup applied to the face in the face image acquired in S1013 is recognized (S1015). At this time, the shape and color of the eyebrows (makeup eyebrows) that constitute the eyebrow makeup are recognized. The processing of S1015 is performed by the processing unit 10 functioning as makeup state recognition means.

[0146] Next, in S1017, based on the state of eyebrow makeup recognized in S1015, eyebrow makeup stamp information is displayed on the display unit 12 (main display area 12a) of the terminal device 1 (S1017). The processing of S1017 is performed by the processing unit 10 functioning as an eyebrow makeup stamp information display unit.

[0147] The eyebrow stamp information includes at least information indicating the shape of the eyebrows (eyebrow shape pattern) specified based on the state of eyebrow makeup recognized in S1015, and information indicating the color of the eyebrows (eyebrow color) specified based on the state of eyebrow makeup recognized in S1015. In addition, the eyebrow stamp information may be displayed on the display unit 12 with information (image data) indicating an overall image of the eyebrow stamp having an eyebrow-shaped stamp member corresponding to the shape of the eyebrows specified based on the state of eyebrow makeup recognized in S1015, and information (image data) indicating the powder for the eyebrow stamp of the eyebrow color specified based on the state of eyebrow makeup recognized in S1015.

[0148] After the process of S1017, the processing unit 10 performs the processes of S1019 to S1037 shown in Fig. 14. The processes of S1019 to S1037 are basically the same as the processes of S137 to S155 (see Fig. 7) in the above-mentioned embodiment. However, if it is determined in S1021 of this modified example 2 that a return instruction has been input (YES in S1021), the process returns to the process of S1001 shown in Fig. 13. In other words, acquisition of a face image, recognition of the state of eyebrow makeup, etc. are redone.

[0149] In addition, in S1025 of the present modified example 2, order information for the eyebrow makeup stamp is generated based on the eyebrow makeup stamp information displayed in the process of S1017, i.e., the state of eyebrow makeup recognized in the process of S1015. This allows the user to order an eyebrow makeup stamp based on the eyebrow makeup actually applied to the user's face (eyebrow area). In this case, the user can actually apply eyebrow makeup using, for example, an eyebrow pencil or mascara, and then order and obtain an eyebrow makeup stamp that allows the user to apply the eyebrow makeup as a stamp. The process of S1025 is performed by the processing unit 10 functioning as a second order information generating means.

[0150] According to the above-mentioned modified example 2, if the face in the face image acquired by the process of S1013 has actual eyebrow makeup applied, the state of the applied eyebrow makeup is recognized, and order information for the eyebrow makeup stamp is generated based on the recognized state of the eyebrow makeup, and the order information is transmitted to the sales company device 2 (the order destination of the eyebrow makeup stamp). This allows the user to order (custom-made) the eyebrow makeup stamp of his / her preference based on the image of the face with actual eyebrow makeup applied (made-up face image).

[0151] In the second modification, the eyebrow makeup stamp information is displayed based on the state of eyebrow makeup recognized by the process of S1015 (makeup state recognition means), and the order information for the eyebrow makeup stamp is generated (S1017, S1025). However, based on the state of eyebrow makeup recognized by the process of S1015 (makeup state recognition means), an eyebrow makeup pattern corresponding to the state of eyebrow makeup may be acquired from the storage unit 11 (see S121). In this case, a simulation image based on the acquired eyebrow makeup pattern is generated and displayed, making it possible to perform a simulation based on the eyebrow makeup actually applied to the user's face (eyebrow area) (see S123, S125), and it becomes possible to order an eyebrow makeup stamp based on the result of the simulation (see S137, S143, S147, S149). In addition, it is also possible to adjust the eyebrow image as in the above embodiment (see S129, S131).

[0152] In addition, the configuration according to the present modified example 2 added to the above-mentioned embodiment can be implemented independently of the embodiment. That is, it is possible to provide an eyebrow makeup application that does not include the eyebrow makeup application execution process (S100) but includes the eyebrow makeup application execution process 2 (S1000) (another example of modified example 2). The inventions (related inventions) related to such another example of modified example 2 are shown below.

[0153] (Related invention 1) The system includes a user device (e.g., a terminal device 1) used by a user, and an order receiving device (e.g., a sales company device 2) that receives an order for a cosmetic product (e.g., an eyebrow cosmetic stamp) from the user, A cosmetics custom-made system configured so that the user device and the order receiving device can be connected via a network, The user device comprises: A facial image acquisition means for acquiring an image including a face of a user with makeup (e.g., eyebrow makeup); a makeup condition recognition means for recognizing a state of makeup applied to a face included in an image acquired by the face image acquisition means; an order information generating means for generating order information for cosmetics based on the state of makeup recognized by the makeup state recognizing means; an order information sending means for sending the order information generated by the order information generating means to a destination of an order for cosmetics, the order receiving device includes an order information receiving means for receiving the order information transmitted by the order information transmitting means, The cosmetics for the user are manufactured based on the order information received by the order information receiving means. A custom-made cosmetics system.

[0154] (Related invention 2) In the cosmetics custom-made system of the above-mentioned related invention 1, The cosmetic product is an eyebrow makeup stamp that can be pressed against the eyebrows of the face to apply makeup to the eyebrows. A custom-made cosmetics system.

[0155] [Variation 3] The third modification adds to the above-mentioned embodiment a configuration for calculating the actual size of the face included in the face image acquired by the process of S113 (face image acquisition means) and a configuration for including information on the calculated face size in the order information for the eyebrow cosmetic stamp. Note that the third modification can be applied to both the first modification and the second modification.

[0156] The program of the eyebrow makeup app (eyebrow makeup app related data) according to the third modification includes an algorithm for calculating the actual size of the face in the face image acquired in the process of S113. The size of the face can be calculated, for example, by dividing it into a vertical size and a horizontal size. Specifically, the distance of a straight line connecting the top of the head and the tip of the chin can be calculated as the vertical size of the face (vertical size), and the distance of a straight line connecting the left and right temples can be calculated as the horizontal size of the face (horizontal size).

[0157] The processing unit 10 of the terminal device 1 in which the eyebrow makeup application related data according to the modification 3 is installed acquires a face image by the process of S113, and then calculates the actual size of the face included in the face image acquired in S113. This calculation process is performed by the processing unit 10 functioning as a face size calculation means.

[0158] Next, the processing unit 10 holds the calculated information on the face size (also called "face size information") until the user inputs an order instruction, and when the order instruction is input (YES in S137), the processing unit 10 includes the face size information in the order information generated based on the input of the order instruction (S143). After that, when the user inputs a transmission instruction (YES in S147), the processing unit 10 transmits the order information including the face size information to the ordering destination of the eyebrow cosmetic stamp (the sales business device 2) (S149).

[0159] According to the above-mentioned third modification, the actual size of the face included in the face image acquired by the process of S113 (face image acquisition means) is calculated, and the information about the calculated size is included in the order information of the eyebrow cosmetic stamp. This makes it possible to order (custom-made) an eyebrow cosmetic stamp that is suitable for the size of the user's face.

[0160] Although the examples and modifications have been described above as the embodiments of the present invention, the present invention is not limited to these.

[0161] For example, the simulation image SM displayed on the display unit 12 (main display area 12a) of the terminal device 1 may be displayed as an image (3D image) that can be rotated in a three-dimensional direction based on a touch operation on the display panel. In this case, the shape of the eyebrows can be adjusted to match the unevenness of the face included in the simulation image SM. In addition, by rotating the simulation image SM in the display unit 12 to change the direction of the face, the shape of the eyebrows can be changed accordingly. In this way, a 3D simulation of eyebrow makeup is possible, making it possible to provide a highly realistic eyebrow makeup simulation.

[0162] Also, the simulation image generated during the execution of the eyebrow makeup simulation by the eyebrow makeup app and data such as the eyebrow makeup pattern (eyebrow shape pattern, eyebrow color pattern) on which the simulation image was generated may be stored as generated information in the storage unit 11 of the terminal device 1 (generated information storage means). In this way, it becomes possible to generate a simulation image generated (created) in the past again based on the generated information, and it becomes possible to check the results of the eyebrow makeup simulation again or order eyebrow makeup stamps based on the regenerated simulation image. This makes it possible to improve the convenience of the eyebrow makeup app.

[0163] Also, based on the order history of eyebrow makeup stamps using the eyebrow makeup app, statistics of eyebrow makeup patterns selected by users may be collected on the system (e.g., a service providing server) (makeup pattern statistical means), and information on the selection tendency of eyebrow makeup patterns (also referred to as "selection tendency information") may be displayed on the display unit 12 of the terminal device 1 based on the statistical results. The selection tendency information may be, for example, information showing eyebrow makeup patterns with relatively high selection rates, information showing the relationship between the selected eyebrow makeup pattern and a user demographic (age, occupation, etc.), information showing the selection rate (sales ranking) of eyebrow makeup patterns in a ranking format, etc. In this way, it is possible to inform the user of the selection tendency of eyebrow makeup patterns (popular eyebrow makeup patterns, etc.). This makes it possible to enhance the functions of the eyebrow makeup app.

[0164] In addition, in the above-mentioned embodiment and modified example, eyebrow makeup app related data is installed in the terminal device 1 used by the user, and the processing unit 10 of the terminal device 1 performs processing related to the eyebrow makeup app, such as the eyebrow makeup app execution process (S100) and the eyebrow makeup app execution process 2 (S1000), but the processing may be performed on the sales company device 2 (service providing server) side.

[0165] Specifically, for example, when a user uses the terminal device 1 to perform an eyebrow makeup simulation or to order eyebrow makeup stamps, the terminal device 1 connects to the sales company device 2 via the network 4 and accesses a website (also called an "eyebrow makeup related site") that provides services related to eyebrow makeup (eyebrow makeup simulation, sales of eyebrow makeup stamps, etc.). Then, based on an instruction from the terminal device 1 that has accessed the eyebrow makeup related site, the processing unit 20 of the sales company device 2 performs the above-mentioned eyebrow makeup application execution process (S100) and eyebrow makeup application execution process 2 (S1000), etc., and returns the execution results of each process to the terminal device 1, so that each step display (screens shown in Figs. 9 and 10, etc.) described in the above embodiment is displayed on the display unit 12 of the terminal device 1.

[0166] According to this configuration, even if eyebrow makeup application-related data is not installed on the terminal device 1, as long as the communication environment (network environment) is in place, it is possible to perform eyebrow makeup simulations and order eyebrow makeup stamps, as in the above-mentioned embodiments.

[0167] In the above-mentioned embodiment and modified example, the present invention is applied to a device and a system for applying eyebrow makeup, but the present invention can also be applied to a device and a system for applying makeup to face parts other than eyebrows, such as the mouth and cheeks. Examples of makeup patterns other than eyebrows include a pattern based on lipstick makeup (lip makeup pattern) and a pattern based on cheek makeup (cheek makeup pattern). Examples of cosmetics other than eyebrow makeup stamps include lipstick stamps and cheek stamps. The present invention can also be applied to cosmetics other than stamp-type cosmetics. [Explanation of symbols]

[0168] 1 terminal device, 2 sales business device, 3 manufacturer device, 4 network, 10 processing unit, 11 memory unit, 12 display unit, 13 operation input unit, 14 communication unit, 15 camera (imaging means), 16 battery, 20 processing unit, 21 memory unit, 22 display unit, 23 operation unit, 24 communication unit, 30 processing unit, 31 memory unit, 32 display unit, 33 operation unit, 34 communication unit, 101 instruction input detection means, 102 imaging means, 103 face image acquisition means, 104 face image display means, 105 makeup pattern display means, 106 makeup pattern acquisition means, 107 simulation image generation means, 108 simulation image display means, 109 guide display means, 110 makeup pattern adjustment means, 111 order information generation means, 112 transmission means, 500 eyebrow makeup stamp, 501 main body, 501a Knob portion, 502 stamp member, 502a stamp surface.

Claims

1. A face image acquisition means for acquiring an image including a face; a makeup pattern acquisition means for acquiring a makeup pattern from a plurality of makeup patterns prepared in advance; a simulation image generating means for generating a simulation image in which makeup based on the makeup pattern acquired by the makeup pattern acquiring means is applied to a face included in the image acquired by the face image acquiring means; a simulation image display means for displaying the simulation image generated by the simulation image generating means; a first order information generating means for generating order information for a cosmetic product based on a makeup pattern included in the simulation image displayed by the simulation image display means; a first transmission means for transmitting the order information generated by the first order information generation means to a cosmetics ordering destination; a makeup condition recognition means for recognizing a state of makeup applied to a face included in an image acquired by the face image acquisition means when the face includes actual makeup; a second order information generating means for generating order information for cosmetics having specifications corresponding to the actual makeup, based on the state of makeup recognized by the makeup state recognizing means; and a second transmission means for transmitting the order information generated by the second order information generation means to a destination of the order for the cosmetics. A makeup support device comprising:

2. a face size calculation means for calculating an actual size of a face included in the image acquired by the face image acquisition means, The order information includes information regarding the size calculated by the face size calculation means. The makeup support device according to claim 1 .

3. A makeup support device according to claim 1 or 2, an order receiving device that receives an order for cosmetics from a user who uses the makeup support device, The makeup support device and the order receiving device are configured to be connectable via a network, the order receiving device includes a receiving means for receiving the order information transmitted by the first transmitting means or the second transmitting means, The cosmetics for the user are manufactured based on the order information received by the receiving means. A custom-made cosmetics system.

4. The cosmetic product is an eyebrow makeup stamp that can be pressed against the eyebrows of the face to apply makeup to the eyebrows.

4. The system for customizing cosmetics according to claim 3.

5. A program for causing a computer to function as each of the means of the makeup support device according to claim 1 or 2.

Citation Information

Patent Citations

  • Sheet-like pack cosmetic

    JP2000086441A

  • Method for providing advice on makeup

    JP2002092132A

  • Network sales system for order-made commodity

    JP2002251532A

  • Cosmetic product presentation system, cosmetic product presentation method, and cosmetic product presentation server

    JP2018109842A

  • Systems and Methods for Identification and Virtual Application of Cosmetic Products

    US20190166980A1