Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program which make a user confirm an appropriate application method of a spray cosmetic in real time when the user applies the spray cosmetic to his / her skin or an application tool.SOLUTION: An information processing system includes a control section. The control section receives application state information which indicates a state of application of a spray cosmetic to user's skin or an application tool from a user terminal of the user using the spray cosmetic, and generates advice information for improving the state of the application on the basis of the application state information and information previously stored according to the state of the application, so as to transmit the generated advice information to the user terminal.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program for providing users with advice information on how to use spray cosmetics. [Background technology]

[0002] There have been technologies for providing users with advice on how to use cosmetics. For example, Patent Document 1 below discloses a method for extracting a facial region from an input image of a user's face, detecting skin color, facial feature points, hair color near the top of the head, and clothing color, detecting a cosmetic color that suits the user based on color data previously stored in the device, and generating an edited image in which a color that suits the user is overlaid on a predetermined position of the input image based on shape data previously stored in the device, and transmitting the edited image to the user's terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-211308 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology of Patent Document 1, although the user can check in advance what the user will look like when they have applied appropriate makeup using colors that suit them, they are unable to receive advice on how to apply the makeup, and they are unable to check whether the user's appearance after applying the makeup is in line with the advice they received in advance.

[0005] In particular, with products such as spray cosmetics, where the finish can vary from person to person, prior advice alone may not be sufficient.

[0006] In view of the above circumstances, an object of the present invention is to provide an information processing system, an information processing method, and a program that enable a user to confirm in real time the appropriate application method when applying spray cosmetics to the skin or an applicator. [Means for solving the problem]

[0007] An information processing system according to one aspect of the present invention includes a control unit that receives application status information indicating the state of application of the spray cosmetic to the user's skin or an applicator from a user terminal of the user who uses the spray cosmetic, generates advice information for improving the state of application based on the application status information and information pre-stored according to the state of application, and transmits the generated advice information to the user terminal.

[0008] An information processing method according to another aspect of the present invention includes: receiving application status information indicating the state of application of the spray cosmetic to the user's skin or application tool from a user terminal of the user who uses the spray cosmetic; generating advice information for improving the application state based on the application state information and information pre-stored according to the application state; and transmitting the generated advice information to the user terminal.

[0009] According to yet another aspect of the present invention, there is provided a program for executing the program on an information processing device, Receiving application status information from a user terminal of a user who uses the spray cosmetic, the application status information indicating the application status of the spray cosmetic to the skin or application tool of the user; generating advice information for improving the application state based on the application state information and information pre-stored according to the application state; and transmitting the generated advice information to the user terminal. [Effects of the Invention]

[0010] According to an information processing system according to one aspect of the present invention, when a user applies a spray cosmetic to the skin or an applicator, the user can check the appropriate application method in real time. However, this effect is not intended to limit the present invention. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing the configuration of a cosmetic application support system according to an embodiment of the present invention. [Figure 2] 1 is a diagram showing the hardware configuration of a cosmetic application support server according to an embodiment of the present invention. [Figure 3] 1 is a diagram showing the configuration of a database included in a cosmetic application support server according to one embodiment of the present invention. [Figure 4] 10 is a flowchart showing the flow of a process for providing spray cosmetic advice information by a cosmetic application support server according to one embodiment of the present invention. [Figure 5] 10A to 10C are diagrams illustrating a plurality of analysis methods of captured images by a cosmetic application support server according to an embodiment of the present invention. [Figure 6] 10 is a diagram showing an example of a method for analyzing a captured image by a cosmetic application support server according to one embodiment of the present invention. FIG. [Figure 7] 10 is a diagram showing an example of a method for analyzing a captured image by a cosmetic application support server according to an embodiment of the present invention. FIG. [Figure 8] 10 is a diagram showing an example of a method for analyzing a captured image by a cosmetic application support server according to an embodiment of the present invention. FIG. [Figure 9] 10 is a diagram showing an example of a method for analyzing a captured image by a cosmetic application support server according to an embodiment of the present invention. FIG. [Figure 10] 10 is a diagram showing an example of a method for analyzing a captured image by a cosmetic application support server according to an embodiment of the present invention. FIG. [Figure 11] 10 is a diagram showing an example of advice information generated by a cosmetic application support server according to one embodiment of the present invention. FIG. [Figure 12]10 is a diagram showing a flow in which a cosmetic application support server acquires the application status of spray cosmetic in another embodiment of the present invention. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, cosmetics are used as an example of a product.

[0013] [System Configuration] As shown in FIG. 1, this system includes a cosmetic application support server 100 on the Internet 50, a plurality of user terminals 200, and an advisor terminal 300.

[0014] The cosmetic application support server 100 is a server (information processing device) that can provide information about cosmetics, and in this embodiment, in particular, provides advice information about how to use spray cosmetics in response to a request from a user terminal 200. The cosmetic application support server 100 is connected to multiple user terminals 200 via the Internet 50.

[0015] User terminal 200 (200A, 200B, 200C, etc.) is a terminal used by a user, such as a smartphone, mobile phone, tablet PC (Personal Computer), notebook PC, desktop PC, etc. When a user uses a spray cosmetic product, user terminal 200 accesses cosmetic application support server 100 via an application installed on user terminal 200, sends an advice request, receives the advice information, etc., and displays it on the screen.

[0016] As shown in Figure 1, the spray cosmetic is, for example, ejected from a diffuser D and formed as a coating on the surface of an applicator (applicator) A, such as a flat puff or sponge, by electrospinning. That is, the spray cosmetic is, for example, a spinning solution (polymer solution) S drawn out from the diffuser D in the form of ultrafine fibers, which, when collected on the surface of the applicator A, instantly overlap to form a transparent or translucent coating made of fibrous deposits. The user transfers the coating formed on the applicator A to their skin.

[0017] Specifically, when a user holds diffuser D in one hand and applicator A in the other hand, facing the user toward the main surface of applicator A, and then turns on diffuser D, an electrical path (electrical circuit) is formed between diffuser D (its nozzle) and applicator A, with the voltage of diffuser D being positive or negative, and applicator A and the user's body in contact with it being grounded, and in this state spinning solution S is ejected, and electrospinning is performed.

[0018] Then, due to the electric field generated between the diffuser D and the applicator A, the spinning solution S discharged from the diffuser D into the electric field is polarized by electrostatic induction, and its tip becomes cone-shaped, and droplets of the charged spinning solution S are discharged from the tip of the cone along the electric field toward the applicator A into the air. The discharged spinning solution S is repeatedly stretched by electric attraction and self-repulsion due to the electric charge possessed by the spinning solution S itself, and is turned into ultrafine fibers, which reach the main surface of the applicator A. As a result, a coating is formed on the applicator A as a fiber deposition body. In this embodiment, the spinning solution S is white and the applicator A is black, but this is not limited to this.

[0019] Because the formation of a coating using such an electrospinning method is often unfamiliar to ordinary users, it is expected that a coating may not be formed properly on applicator A (the spray cosmetic may not be sprayed in a state suitable for transfer). Therefore, in this embodiment, the cosmetic application support server 100 receives from the user terminal 200 a captured image of the state of application (spraying) of the spray cosmetic to applicator A, analyzes it, and determines whether the application state meets predetermined standards. If it determines that the predetermined standards are not met, it generates advice information for improving the application state and transmits it to the user terminal 200. Details of the process of generating the advice information will be described later.

[0020] The advisor terminal 300 is a terminal used by an advisor, such as a person in charge of a cosmetics manufacturer, who has knowledge about the spray cosmetics, and is, for example, a smartphone, mobile phone, tablet PC, notebook PC, desktop PC, etc. For users who are unable to solve their problems using the advice information generated by the cosmetics application support server 100, the advisor terminal 300 connects to the user terminal 200 via, for example, telephone, video call, chat, etc., and provides advice directly to the user.

[0021] [Hardware configuration of the cosmetics application support server] As shown in FIG. 2, the cosmetic application support server 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an input / output interface 15, and a bus 14 connecting these to each other.

[0022] The CPU 11 accesses the RAM 13 and other memory as needed, and performs various arithmetic processing while providing overall control over each block of the cosmetic application support server 100. The ROM 12 is a non-volatile memory that permanently stores firmware such as the OS, programs, and various parameters executed by the CPU 11. The RAM 13 is used as a working area for the CPU 11, and temporarily stores the OS, various applications currently being executed, and various data currently being processed.

[0023] The input / output interface 15 is connected to a display unit 16, an operation reception unit 17, a storage unit 18, a communication unit 19, and the like.

[0024] The display unit 16 is a display device that uses, for example, an LCD (Liquid Crystal Display), an OLED (Organic ElectroLuminescence Display), a CRT (Cathode Ray Tube), or the like.

[0025] The operation reception unit 17 is, for example, a pointing device such as a mouse, a keyboard, a touch panel, or other input device. When the operation reception unit 17 is a touch panel, the touch panel can be integrated with the display unit 16.

[0026] The storage unit 18 is a non-volatile memory such as a hard disk drive (HDD), a flash memory (solid state drive (SSD)), or other solid-state memory. The storage unit 18 stores the OS, various applications, and various data.

[0027] As will be described later, particularly in this embodiment, the memory unit 18 has a user information database, a captured image information database, and an advice information database, in addition to programs such as applications necessary for the evaluation and analysis information provision process according to skin type, which will be described later.

[0028] The communication unit 19 is, for example, a NIC (Network Interface Card) for Ethernet or various modules for wireless communication such as a wireless LAN, and is responsible for communication processing between the user terminal 200 and the advisor terminal 300.

[0029] Although not shown, the basic hardware configuration of the user terminal 200 and the advisor terminal 300 is also substantially the same as the hardware configuration of the cosmetic application support server 100 described above.

[0030] [Database configuration of the cosmetics application support server]

[0031] 3, the cosmetic application support server 100 has, in the storage unit 18, a user information database 31, a captured image information database 32, and an advice information database 33. Note that each of these databases may be stored in a storage device or server externally connected to the cosmetic application support server 100, rather than in the storage unit 18.

[0032] The user information database 31 stores attribute information for each user who is a member of the support service provided by the cosmetic application support server 100. The user attribute information includes general information such as the user's name, a user ID for identifying the user, age (generation), occupation, address (area of ​​residence), gender, and email address, as well as information about the user's skin, such as the user's skin type, cosmetic preferences, and skin problems the user may have.

[0033] The captured image information database 32 stores captured images of the application state of the spray cosmetic applicator A received from the user terminal 200, as well as processed images (described later).

[0034] The advice information database 33 analyzes the captured image and, if the analysis results do not satisfy predetermined criteria (described below) related to the application state of the spray cosmetics, stores advice information generated for the user to improve the application state of the spray cosmetics, as well as information related to the predetermined criteria (threshold information). The advice information is generated and stored for each different (poor) application state of the spray cosmetics.

[0035] These databases are used by mutual reference as necessary in the process of providing cosmetic evaluation information according to skin type by the cosmetic application support server 100, which will be described later.

[0036] [Operation of the cosmetics application support server] Next, we will explain the operation of the cosmetic application support server 100 configured as described above. This operation is performed by the cooperation of hardware such as the CPU 11 and communication unit 19 of the cosmetic application support server 100 and software stored in the storage unit 18. For convenience, in the following explanation, the CPU 11 is the subject of operation.

[0037] 4, the CPU 11 of the cosmetic application support server 100 first determines whether or not an advice request has been received from the user terminal 200 (step 41). The advice request is sent when the user of the user terminal 200 is unable to successfully apply (spray) the spray cosmetic to the applicator A, and the advice request includes a captured image taken by the user terminal 200 of the state of application of the spray cosmetic to the applicator A. The captured image is stored in the captured image information database 32.

[0038] When it is determined that the advice request has been received (Yes in step 41), the CPU 11 extracts a rectangular application area image that surrounds the maximum contour of the applicator A from the captured image included in the advice request (step 42). The extracted application area image is also stored in the captured image information database 32.

[0039] Next, the CPU 11 generates a grayscale image and a binarized image of the rectangular application area image (steps 43 and 46). The grayscale conversion is performed by calculating a weighted sum of the R, G, and B components, for example, as follows. The threshold for the binarization is set to 100, for example. Y = 0.299 * R + 0.587 * G + 0.114 * B

[0040] Next, the CPU 11 determines whether the grayscale converted image satisfies a predetermined standard (described later) related to brightness values ​​(step 44). The predetermined standard is referenced from the advice information database 33.

[0041] If it is determined that the image does not satisfy the predetermined criteria regarding the brightness value (No in step 44), the CPU 11 generates advice information corresponding to the criteria (step 45).If it is determined that the image satisfies the predetermined criteria (Yes in step 44), the CPU 11 ends the process.

[0042] Meanwhile, the CPU 11 determines whether the binarized image satisfies a predetermined standard (described later) regarding the white area of ​​the image (step 47). The predetermined standard is also referenced from the advice information database 33.

[0043] If it is determined that the image does not satisfy the predetermined criteria for the white area (No in step 47), the CPU 11 generates advice information based on the information corresponding to the criteria stored in the advice information database 33 (step 48).If it is determined that the image satisfies the predetermined criteria (Yes in step 47), the CPU 11 ends the process.

[0044] Next, the CPU 11 transmits advice information generated according to the above criteria to the user terminal 200 (step 49). Note that if it is determined that both the grayscale converted image and the binarized image satisfy the above predetermined criteria (Yes in step 44 and Yes in step 47), the CPU 11 may generate advice information indicating that there is no problem with the current method and transmit it to the user terminal 200.

[0045] Next, the CPU 11 determines whether or not an advisor notification request has been received from the user terminal 200 (step 50). This advisor notification request is sent when the user's problem cannot be solved by the advice information sent above.

[0046] If it is determined that an advisor notification request has been received (Yes in step 50), the CPU 11 notifies the advisor terminal 300 that the problem of poor application status has not been resolved, along with information about the user contained in the user information database 31. The advisor terminal 300 connects to the user terminal 200 via, for example, a telephone call, a video call, or a chat, and provides advice directly to the user.

[0047] FIG. 5 is a diagram illustrating a plurality of analysis methods of the captured image by the cosmetic application support server 100.

[0048] As shown in the figure and as described above, the CPU 11 extracts a rectangular application area from the captured image received from the user terminal 200, and performs grayscale conversion and binarization on it.

[0049] For the grayscale converted image, the CPU 11 performs (A) image analysis by calculating density (average luminance value) and (B) image analysis by calculating average luminance value for each of the multiple divided regions.

[0050] On the other hand, for the binarized image, CPU 11 performs (C) image analysis by calculating the area of ​​the white area, (D) image analysis by calculating the width from the edge of the white area to the edge of the binarized image, and (E) image analysis by calculating the number of white areas.

[0051] 6 is a diagram showing an example of the analysis method (A) above. As shown in the diagram, the CPU 11 plots a histogram of the brightness values ​​of the application area image after the grayscale conversion and calculates the average brightness value of the application area image. In the diagram, the second application area image and histogram from the left show good application of the spray cosmetic to applicator A (average brightness value: 114.9), while the other four application area images and histograms show uneven application of the spray cosmetic or an insufficient amount of spray.

[0052] Possible causes of such uneven application or insufficient spray amount include fast hand movements when spraying, large hand movements when spraying, or short spray time.

[0053] When the CPU 11 determines that the average brightness value is less than the first threshold value, the CPU 11 generates advice information for increasing the amount or duration of application of the spray cosmetic. The first threshold value is set to, for example, 100, but is not limited to, this value.

[0054] Examples of such advice might include, "When you start spraying, hold the device still and aim at the center. When the sprayed cosmetic begins to turn white, slowly spread it around." and "The recommended spray time is 10 seconds."

[0055] 7 is a diagram showing an example of the analysis method (B) above. As shown in the diagram, CPU 11 divides the application area image after the grayscale conversion into n x m regions, calculates the average brightness value of each divided region, and displays it as a heat map. The top row of the diagram is the original application area image after grayscale conversion, the middle row is an image obtained by dividing it into 4 x 8 regions and generating a heat map, and the bottom row is an image obtained by dividing it into 8 x 16 regions and generating a heat map.

[0056] As a criterion for the brightness value, the CPU 11 determines whether the difference between the maximum and minimum values ​​or the standard deviation of the average brightness value of the divided area is greater than or equal to a second threshold value, and if it determines that the difference between the maximum and minimum values ​​or the standard deviation is greater than or equal to the second threshold value, it generates advice information for reducing uneven application of the spray cosmetic in the application area image.

[0057] Such advice information could be similar to that mentioned above, such as, "When you start spraying, hold the device still and aim at the center. When the sprayed cosmetic starts to turn white, slowly spread it around."

[0058] In the same figure, the application area image second from the left has the smallest difference between the maximum and minimum values ​​and the smallest standard deviation, indicating good application of the spray cosmetic to the applicator A, while the other four application area images indicate uneven application of the sprayed spray cosmetic. The second threshold value for the difference between the maximum and minimum values ​​is, for example, 35, and the second threshold value for the standard deviation is, for example, 60, but is not limited to these.

[0059] Figure 8 shows an example of the analysis method (C) above. As shown in the figure, the CPU 11 calculates the area (number of pixels) of the white area in the application area image after binarization processing. Again, the application area image second from the left has the largest area of ​​the white area (5,406,540px), indicating good application of the spray cosmetic to applicator A, while the other four application area images indicate insufficient application of the spray cosmetic sprayed onto applicator A.

[0060] Possible causes of this insufficient coverage area include the distance between the diffuser D and the applicator A being too close, or the spraying time being too short.

[0061] The CPU 11 then determines whether the area of ​​the white area included in the application area image is less than a third threshold value as a predetermined criterion for the white area, and if it determines that the area is less than the third threshold value, generates advice information for increasing the application area of ​​the spray cosmetic. The third threshold value is, for example, 5,000,000 px, but is not limited to this.

[0062] Such advice information may include, for example, "Spray from a distance of one sponge (approximately 7 cm)" and "Spray for approximately 10 seconds."

[0063] 9 is a diagram showing an example of the analysis method (D) above. As shown in the diagram, the CPU 11 removes noise from the application area image after the binarization process, for example, by opening and closing, and then identifies a rectangular area (white frame in the diagram) that surrounds the maximum contour of the white area.

[0064] Then, as a predetermined criterion for the white area, CPU 11 determines whether the minimum value of the length from the edge of the rectangular area to the edge of the application area image outside it (for example, the four directions of Top, Bottom, Left, and Right in the same figure) is less than a fourth threshold value, and if it determines that the minimum value is less than the fourth threshold value, generates advice information to prevent the spray cosmetic from being applied to a predetermined width area from the outer edge of applicator A toward the center.

[0065] This is because if the spray cosmetic adheres to the very edge of the outer periphery of applicator A, it becomes difficult to transfer that area to the user's skin, so a certain width from the outer periphery must be secured as a clearance area during transfer. One possible reason for the spray cosmetic adhering to the outer periphery of applicator A is that the user moves the hand holding diffuser D too much.

[0066] Regarding the minimum value, the second application area image from the left in the figure is close to the ideal application state, with a Top Distance of 542 pixels, a Bottom Distance of 615 pixels, a Left Distance of 340 pixels, and a Right Distance of 410 pixels, with the minimum value being the Left Distance of 340 pixels. The fourth threshold value is, for example, about 80% (270 pixels) of the minimum value in the ideal state, but is not limited to this.

[0067] Examples of such advice might include, "When you start spraying, hold the device still and aim at the center. When the sprayed cosmetic starts to turn white, slowly spread it around." or "If you spray it with your arms close to your sides, it will be easier to keep your hand steady."

[0068] 10 is a diagram showing an example of the analysis method (E) above. As shown in the diagram, the CPU 11 removes noise from the application area image after binarization processing, for example, by opening and closing, and then counts the number of continuous white areas included in the application area image.

[0069] The CPU 11 then determines whether the number of white areas is a predetermined number, and if it determines that the number is not the predetermined number, generates advice information for increasing the amount or application time of the spray cosmetic in the application area image or advice information for reducing uneven application. The predetermined number is typically one, but may be two or more depending on the applicator A and the spraying technique.

[0070] In the example shown in the figure, the second application area image from the left has one white area, indicating good application, while the leftmost application area image has zero white areas, and the other application area images have four or more white areas, indicating insufficient application or uneven application.

[0071] Possible causes of this insufficient application include a short spray time, and possible causes of uneven application include the user moving their hand holding diffuser D too quickly, the spray speed being too fast, or the distance between diffuser D and applicator A being too close.

[0072] Examples of such advice might include, "When you start spraying, hold the device still and aim at the center. When the sprayed makeup starts to turn white, slowly spread it around.", "Spray from a distance of one sponge (approximately 7 cm).", and "The recommended spray time is 10 seconds."

[0073] The CPU 11 may use the determination of the average brightness value described in (A) above in combination with this determination. That is, the CPU 11 may generate the advice information when it determines that the average brightness value of the application area image after the grayscale conversion is less than the first threshold value and the number of white areas in the application area image after the binarization process is a predetermined number (1).

[0074] FIG. 11 is a diagram showing an example of a display screen of advice information generated by the cosmetics application support server 100 and displayed on the user terminal 200 (on its application).

[0075] As shown in the figure, the advice information includes, for example, a photograph received from the user terminal 200, along with text information 61 indicating the reason why the spraying condition was judged to be poor (for example, uneven application in the figure) and how to improve it (for example, in the figure, fixing the spray at the center of applicator A at first and slowly spreading it around, maintaining a specified spraying time, etc.).

[0076] Furthermore, for example, a video playback button 62 with a hyperlink to a video showing an appropriate spraying method is provided below the text information 61. The user can refer to the text information 61 and improve the spraying condition while viewing the video using the video playback button 62.

[0077] Furthermore, for example, at the bottom of the display screen, an advisor notification button 63 is also provided for connecting to the advisor terminal 300. If the spraying condition does not improve even after the text information 61 and video are used, the user can press the advisor notification button 63 to directly communicate with the advisor via video call, chat, telephone, etc., to improve the spraying condition.

[0078] As described above, according to this embodiment, the cosmetic application support server 100 can allow the user to confirm the appropriate application method in real time when applying spray cosmetic to the applicator A.

[0079] [Variations] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and it goes without saying that various modifications can be made within the scope of the gist of the present invention.

[0080] In the above-described embodiment, the spray cosmetic is sprayed from the diffuser D onto the applicator A and then transferred to the user's skin, but it may also be sprayed directly onto the user's skin to form a film.

[0081] In this case, too, the cosmetic application support server 100 receives from the user terminal 200 an image of the state in which the spray cosmetic has been applied to the user's skin, and performs image analysis of the image to generate advice information if the various criteria described above are not met.

[0082] In this case, in the above-described embodiment, the background color of the captured image was black, which is the color of applicator A, but the difference is that in the captured image when applied directly to the skin, the background color is the color of the user's skin.

[0083] Furthermore, in the above-described embodiment, the image of the rectangular application area to which the spray cosmetics are to be applied is the area surrounding the maximum outline of applicator A, but when the spray cosmetics are to be applied directly to the skin, the cosmetics application support server 100 performs image recognition of the user's body parts (cheeks, hands, etc.) in the captured image, extracts an image of a rectangular area corresponding to the area relative to the size of the recognized body part as the image of the application area to which the spray cosmetics are to be applied, and further excludes (converts to black) the skin-colored part, which is the background color.

[0084] The cosmetic application support server 100 then performs the image analysis processes (A), (B), (C) and (E) on the application area image, excluding the analysis process (D), which is an analysis process specific to applicator A, from the image analysis processes (A) to (E) performed in the above-described embodiment, thereby determining the application status of the spray cosmetic and generating advice information, as in the above-described embodiment.

[0085] In the above-described embodiment, the spray cosmetic is an ultrafine fiber product made using an electrospinning method, but the spray cosmetic is not limited to this and can be any cosmetic product that can be sprayed.

[0086] In the above-described embodiment, only one cosmetic application support server 100 is shown, but the processing executed by the cosmetic application support server 100 may be distributed and executed by multiple servers. For example, the image analysis processing of the application area image after grayscale conversion and binarization processing in the above-described embodiment may be executed by a server other than the cosmetic application support server 100, and the cosmetic application support server 100 may receive the image analysis results from the other server.

[0087] In the above-described embodiment and modified examples, the cosmetic application support server 100 receives from the user terminal 200 a captured image of the applicator A or the user's skin with the spray cosmetic applied as information indicating the state of application of the spray cosmetic to the applicator A or the user's skin (application status information), and performs image analysis to determine the application status and generate advice information. However, the methods of receiving and determining the application status information are not limited to receiving and analyzing the captured image. For example, the cosmetic application support server 100 may display the application status information as options on the user terminal 200 and receive information on the option selected by the user terminal 200 based on the user's selection operation.

[0088] 12, for example, the cosmetic application support server 100 may acquire information indicating the application status of the spray cosmetic by displaying, as options, problems the user is having when applying the spray cosmetic using the diffuser D or applicator A from a start menu provided on an application or website on the user terminal 200, and having the user terminal 200 select one of the problems (application status) hierarchically based on a selection operation by the user. When displaying each option, the cosmetic application support server 100 may also display an image or video indicating the status of each option.

[0089] Then, when the cosmetic application support server 100 identifies the application state of the spray cosmetic based on the user's selection, it generates advice information on the cause and how to improve it based on information previously stored in the advice information database 33 corresponding to the application state, as shown in Figure 11 above, and displays this on the user terminal 200.

[0090] In this way, the cosmetics application support server 100 can identify the application state of the spray cosmetics based on the user's selection operation for options regarding the application state of the spray cosmetics, without relying on the above-mentioned captured image, and present the user with advice information corresponding to the identification.

[0091] Furthermore, even when the cosmetic application support server 100 receives, as the application state information, a captured image of the applicator A or the user's skin in a state where the spray cosmetic has been applied from the user terminal 200, the cosmetic application support server 100 may determine the application state without performing image analysis on the captured image. For example, a person in charge, such as the advisor, may visually check the application state shown in the captured image received from the user terminal 200 and accept, from the person in charge's terminal, an input operation to select an option similar to the above options, or input information identifying the application state (tags, metadata, etc.), and the cosmetic application support server 100 may determine the application state based on the input information.

[0092] Among the inventions described in the claims of this application, the invention described as an "information processing method" is one in which each step is automatically performed by at least one device such as a computer through software-based information processing, and is not performed by a human using a device such as a computer. In other words, the "information processing method" is an information processing method using computer software, and is not a method in which a human operates a computing tool called a computer. [Explanation of symbols]

[0093] 11...CPU 18...Storage section 19…Communications Department 31...User physical information database 32...Image information database 33...Advice information database 61...Text information 100...Cosmetics application support server 200...User terminal 300...Advisor terminal

Claims

1. receiving application status information indicating the state of application of the spray cosmetic to the user's skin or application tool from a user terminal of the user who uses the spray cosmetic; generating advice information for improving the application state based on the application state information and information pre-stored according to the application state; transmitting the generated advice information to the user terminal; Control unit An information processing system comprising:

2. The control unit receives, as the application state information, a captured image of the user's skin or an applicator in a state where the spray cosmetic has been applied.

2. The information processing system according to claim 1.

3. The control unit Extracting an application area image to which the spray cosmetic is to be applied from the received captured image, and performing image analysis on the application area image to determine whether the application state satisfies a predetermined standard; When it is determined that the application state does not satisfy the predetermined standard, advice information for improving the application state is generated. The information processing system according to claim 2 .

4. The control unit converts the application area image into a grayscale image and determines whether the application area image satisfies a criterion related to a luminance value of the application area image as the predetermined criterion. The information processing system according to claim 3 .

5. The control unit As the predetermined criterion, it is determined whether the average luminance value of the application area image is less than a first threshold value; When it is determined that the average brightness value is less than the first threshold, the advice information for increasing the application amount or application time of the spray cosmetic is generated. The information processing system according to claim 4 .

6. The control unit Dividing the application area image into a plurality of divided areas, calculating an average luminance value for each divided area, and determining whether the difference between the maximum value and the minimum value or the standard deviation of the average luminance value for each divided area is equal to or greater than a second threshold value as the predetermined criterion; When it is determined that the difference between the maximum value and the minimum value or the standard deviation is equal to or greater than the second threshold, the advice information for reducing uneven application of the spray cosmetic in the application area image is generated. The information processing system according to claim 4 .

7. The control unit binarizes the application area image and determines whether the application area image satisfies the predetermined criterion regarding the area or number of white areas included in the application area image. The information processing system according to claim 3 .

8. The control unit As the predetermined criterion, it is determined whether or not the area of ​​the white area included in the application area image is less than a third threshold value; When it is determined that the area is less than the third threshold, the advice information for increasing the application area of ​​the spray cosmetic is generated. The information processing system according to claim 7 .

9. The captured image is an image of the applicator, The control unit a rectangular area that encloses the largest outline of the white area included in the application area image is identified, and the predetermined criterion is whether or not the minimum value of the length from an edge of the rectangular area to an outer edge of the application area image is less than a fourth threshold value; When it is determined that the minimum value is less than the fourth threshold value, the advice information is generated so that the spray cosmetic is not applied to a predetermined width portion from the outer circumferential edge of the applicator toward the center. The information processing system according to claim 7 .

10. The control unit As the predetermined criterion, it is determined whether the number of continuous white areas included in the application area image is a predetermined number; When it is determined that the number is not a predetermined number, the advice information for reducing uneven application of the spray cosmetic in the application area image or the advice information for increasing the application amount or application time of the spray cosmetic is generated. The information processing system according to claim 7 .

11. The control unit displays, as the application state information, the state of the user's skin or applicator after the spray cosmetic has been applied as options on the user terminal, and receives information about the option selected by the user terminal based on a selection operation by the user. The information processing system according to claim 1 .

12. The control unit causes the user terminal to display an image or video showing a state of the option together with the option.

12. The information processing system according to claim 11.

13. receiving application status information indicating the state of application of the spray cosmetic to the user's skin or application tool from a user terminal of the user who uses the spray cosmetic; generating advice information for improving the application state based on the application state information and information pre-stored according to the application state; transmitting the generated advice information to the user terminal; Information processing methods.

14. In the information processing device, Receiving application status information from a user terminal of a user who uses the spray cosmetic, the application status information indicating the application status of the spray cosmetic to the skin or application tool of the user; generating advice information for improving the application state based on the application state information and information pre-stored according to the application state; transmitting the generated advice information to the user terminal; A program that executes the following.

Images