Cosmetic surgery application software

The cosmetic surgery application software addresses practitioner limitations by simulating facial changes based on genetic inheritance and downtime effects, providing expert recommendations to ensure safer cosmetic surgery outcomes.

JP2024165842AActive Publication Date: 2024-11-28CROWNSTRATEGY CO LTD

Patent Information

Application Number
JP2023082378
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-28
Estimated Expiration
2043-05-18

AI Technical Summary

Technical Problem

Cosmetic surgery practitioners often fail to reshape a patient's face as desired due to the difficulty of major procedures or lack of expertise in the treatment area, leading to potential treatment failures.

Method used

A cosmetic surgery application software that simulates facial changes using genetic inheritance from parents, predicts downtime effects, and suggests suitable specialists, displayed on a mobile device to guide safer cosmetic surgery decisions.

Benefits of technology

Enables users to visualize and plan cosmetic surgery outcomes accurately, avoiding treatment failures by considering genetic inheritance and expert recommendations, thus ensuring safer cosmetic surgery procedures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165842000001_ABST
    Figure 2024165842000001_ABST
Patent Text Reader

Abstract

To provide a useful face image and information contributing to safely performing cosmetic surgery.SOLUTION: A computer is caused to execute cosmetic surgery application software to cause a mobile terminal to combine a face image of a user and face images of the father and the mother of the user input to the mobile terminal according to a composition ratio input thereto to create a composite face image, thereby simulating the face of the user changed over time, and to display a result of the simulation, cause the mobile terminal to simulate a face when the face in the face image of the user is beautified and a face when the face in the face image of the user is subjected to cosmetic surgery by using a beautification simulation function and a cosmetic surgery simulation function, and to display results of the simulation, and cause the mobile terminal to display, according to a portion subjected to the surgery input to the mobile terminal, information including a medical specialist suitable for the surgery to be performed for the portion.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to cosmetic surgery application software for use on mobile terminals and the like. [Background technology]

[0002] In recent years, the demand for cosmetic surgery has been increasing due to the growing awareness of beauty. Accordingly, devices that create and display images of faces after cosmetic surgery are becoming widespread, allowing consumers to concretely imagine what their face will look like after cosmetic surgery. As an example of such a facial image display device, there is one that creates and displays an image of a face expected after cosmetic surgery, taking into account the factors of deformation caused by cosmetic surgery (see Patent Document 1 below). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-224802 A Summary of the Invention [Problem to be solved by the invention]

[0004] Therefore, when a consumer of facial cosmetic surgery visits a cosmetic surgery clinic for the first time, the consumer may already have an image of his or her ideal post-surgery face. In such cases, the practitioner at the cosmetic surgery clinic is first required to reshape the consumer's face to match the facial image, in accordance with the consumer's wishes.

[0005] However, in reality, the practitioner often fails to perform the treatment because he / she is unable to reshape the face according to the face image desired by the patient. The first factor of such treatment failure is that the practitioner attempts a highly difficult treatment. For example, generally, the greater the change in the face of the patient before and after reshaping, the greater the difficulty of the treatment. However, if the practitioner attempts to perform a treatment with a large change according to the face image in order to meet the unreasonable request of the patient, the greater the possibility of failure. Another factor of treatment failure is that the practitioner attempts to perform a treatment on a part that is not his / her specialty. For example, depending on the part to be treated and the method, the treatment may require skill, and if such a treatment is performed by an inexperienced person, the greater the possibility of failure.

[0006] In consideration of the above circumstances, the present invention aims to provide an app that enables a terminal to realize a function that allows a consumer to obtain useful facial images and information that will contribute to safer cosmetic surgery. [Means for solving the problem]

[0007] In the present invention, in order to reduce the probability of cosmetic surgery failure, the facial shapes of the father and mother of the cosmetic surgery client are emphasized, particularly in simulations of facial cosmetic surgery. That is, the present invention focuses on the genetic phenomenon in which traits are passed from parents to children through reproduction, and provides the user with the results of a simulation using facial images of the parents. This is because it is safer and can prevent cosmetic surgery failures such as those described above in advance. In addition, the changes in the treatment site immediately after the procedure are also simulated (downtime simulation, described later), and the changes in the face after cosmetic surgery are predicted to some extent, making it possible to propose cosmetic surgery within a reasonable range.

[0008] Therefore, the present invention is a cosmetic surgery application software that realizes a function of displaying information about cosmetic surgery on a mobile terminal that can communicate with a server, and by executing the software on a computer of a system including a server and a mobile, a facial image of a user input to the mobile terminal and facial images of the user's father and mother are synthesized according to an input synthesis ratio to generate a synthetic facial image, simulating the face as it changes over time and displaying the simulation result on the mobile terminal, a beautification simulation function and a cosmetic surgery simulation function are used to simulate the face when the user's facial image is beautified and the face when the user's facial image is cosmetically modified, and displaying the simulation results on the mobile terminal, and according to the name of the part to be treated input to the mobile terminal, information including a specialist suitable for the treatment of that part is displayed on the mobile terminal. Note that in the following description, the cosmetic surgery application software may be simply referred to as a "cosmetic surgery app".

[0009] Furthermore, a downtime simulation function may be used to simulate a face during downtime after cosmetic surgery for the composite face image to generate a face image during the downtime, and the face image during the downtime may be displayed on the mobile terminal. Furthermore, a plurality of face images of the user may be displayed on the mobile terminal, and when generating a composite face image, a face image selected by the user from the plurality of face images of the user may be used to generate the composite face image. Effect of the Invention

[0010] According to the present invention, consumers can easily obtain facial images and specialist information that will contribute to safe cosmetic surgery, thereby enabling them to avoid surgical failures in advance.

[0011] That is, according to the present invention, a more accurate face image is output by synthesizing the face images of the user's parents with the face image of the user as an image of the user's face that is expected to change over time without cosmetic surgery, and it is possible to identify areas where cosmetic surgery is recommended, and an ideal face image generated based on the face image of the user is displayed on the mobile terminal, so that a face image that can be reasonably cosmetically modified can be provided to the user. Therefore, it is possible to avoid in advance a failure in the treatment caused by forcing the practitioner to perform unreasonable cosmetic surgery. Furthermore, according to the present invention, information including a specialist suitable for the treatment is displayed on the mobile terminal, so that the user can safely receive the treatment by requesting the displayed specialist to perform the cosmetic surgery. Therefore, it is possible to avoid a situation where a person with little experience performs the treatment and fails. Furthermore, according to the present invention, the user can imagine the face after cosmetic surgery from the cosmetic surgery face image displayed on the mobile terminal. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing an example of a configuration of a system that executes a cosmetic surgery application according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing the configuration of the system shown in FIG. 1. [Diagram 3] 11 is a flowchart illustrating an example of execution steps of a cosmetic surgery application according to the embodiment. [Figure 4] 5 is a flowchart illustrating an example of execution steps of the cosmetic surgery app according to the embodiment, following FIG. 3. [Diagram 5] 5 is a flowchart illustrating an example of execution steps of the cosmetic surgery application according to the embodiment, following FIG. 4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described. However, the present invention is not limited to this. In addition, in the drawings, some parts may be enlarged or emphasized, and the scale may be appropriately changed to express the drawings. The cosmetic surgery application software according to the embodiment is used mainly by users who wish to have cosmetic surgery on their faces, and is application software that realizes the function of displaying an image of an ideal face and useful information about cosmetic surgery on a mobile terminal.

[0014] More specifically, the cosmetic surgery app causes an information processing device of the cosmetic surgery system (i.e., a computer constituting the mobile terminal 100 and / or server 200 described below) to execute specified processing, thereby enabling the cosmetic surgery system to realize an aging simulation function, and also generates an image of the user's virtual face using the beautification simulation function and cosmetic surgery simulation function of the cosmetic surgery system, and displays the generated image on the mobile terminal.

[0015] Here, the aging simulation function refers to a function that simulates a person's face due to aging (for example, a face that has changed due to sagging or dullness of the skin, an increase in wrinkles and age spots, graying or thinning of hair, etc. due to aging). By displaying a virtual face image that is the result of this simulation on a mobile terminal, the user can know the expected changes in the face that will occur with aging in the case of not undergoing cosmetic surgery. The beautification simulation function refers to a function that simulates a face when a real face is changed into a more beautiful face, and for example, an input face image is processed to look like a photo of a beautiful model using a specified algorithm, or an image of a specific part of the input face image (for example, the eyes or nose) is synthesized with a part of the face image of a person who is considered beautiful, thereby generating a face image when the real face is beautified. By displaying a virtual face image that is the result of this simulation on a mobile terminal, the user can obtain a model of an ideal face. The cosmetic surgery simulation function refers to a function that simulates a face after cosmetic surgery. By displaying the resulting virtual facial image on a mobile device, users can visualize what their face will look like after cosmetic surgery, allowing them to check, for example, whether the surgically altered parts suit them.

[0016] In addition, the cosmetic surgery app uses a downtime simulation function of the cosmetic surgery system to generate a face image during downtime and displays the generated face image on a mobile device. Downtime refers to the period from cosmetic surgery treatment to recovery. In other words, when cosmetic surgery is performed, swelling, internal bleeding, and edema may occur in the treated area immediately after the treatment, and the downtime is the time it takes for these symptoms to subside and for the patient to recover. The downtime simulation function is a function that simulates the face during such downtime. By displaying the results of this simulation on a mobile device, the user can imagine in advance what their own face will look like during downtime.

[0017] In addition, the cosmetic surgery app has a function of displaying information including the name of the treatment area and the specialist suitable for the treatment method on the mobile terminal by executing a predetermined process on the computer of the cosmetic surgery system. The treatment area means the area of ​​a person's face that is the subject of treatment related to cosmetic surgery, such as the eyes, mouth, nose, and face. The treatment method is, for example, double eyelid surgery, irradiation with a cosmetic laser, excision using a scalpel, etc.

[0018] Fig. 1 is a schematic diagram showing an example of the configuration of a cosmetic surgery system SYS that executes a cosmetic surgery application according to an embodiment. As shown in Fig. 1, the cosmetic surgery system SYS includes a mobile terminal 100 and a server 200. The mobile terminal 100 and the server 200 are connected to each other via a network 300 so as to be able to communicate with each other. The network 300 is a communication network including a wireless communication line, such as the Internet or a wireless communication line network. Note that the network 300 may be a wired communication line.

[0019] Fig. 2 is a block diagram showing the configuration of the cosmetic surgery system SYS of Fig. 1. As shown in Fig. 2, the mobile terminal 100 has an input display unit 10, a communication unit 11, a control unit 12, a storage unit 13, and an imaging unit 14. The mobile terminal 100 is a portable information processing device that has a built-in computer and can communicate with the server 200, such as a smartphone or a tablet terminal. The mobile terminal 100 includes an input display means such as a touch panel that constitutes the input display unit 10, an information storage device such as a memory, an arithmetic processing device such as a CPU, a transmission / reception device, an imaging means such as a digital camera, and the like.

[0020] The input display unit 10 has an input display screen, and for example, receives an operation by a user's touch, and inputs facial image data and instructions by the user. The input display unit 10 also displays simulation results and specialist information, which will be described later, on the input display surface. The communication unit 11 transmits and receives data to and from the server 200. The control unit 12 is a processing unit that executes various information processes according to a program such as a cosmetic surgery app. The storage unit 13 stores programs such as a cosmetic surgery app, facial image data input by the user, simulation results and specialist information, which will be described later, and the like. The imaging unit 14 photographs a person's face to obtain a facial image.

[0021] The server 200 has a server-side communication unit 20, a storage unit 21, and an information processing unit 22, which are realized by, for example, a computer. Such a computer includes an information storage device such as a hard disk or memory, an arithmetic processing device such as a CPU, a transmission / reception device, and the like. The server-side communication unit 20 transmits and receives data to and from the mobile terminal 100. The storage unit 21 stores various data such as facial images and specialist information, simulation results (for example, facial images obtained when aging simulation is performed), and programs for causing the information processing unit 22 to execute predetermined processing, which will be described later. The information processing unit 22 is a processing unit that executes various information processes based on information transmitted from the mobile terminal 100 and information stored in the storage unit 21.

[0022] The information processing unit 22 includes an aging simulation unit 23, a beautification simulation unit 24, a cosmetic surgery simulation unit 25, a downtime simulation unit 26, and a medical examination information search unit 27. The aging simulation unit 23 is a processing unit that generates a composite face image by synthesizing the facial image of the user and the facial images of the user's father and mother input to the input display unit 10 of the mobile terminal 100 according to the input synthesis ratio, and simulates the aging of the user's face. The aging simulation unit 23 operates according to a program such as a cosmetic surgery app, thereby realizing the above-mentioned aging simulation function. The beautification simulation unit 24 is a processing unit that performs beautification processing on the facial image of the user, and simulates the face after beautification. The beautification simulation unit 24 operates according to a program, thereby realizing the above-mentioned beautification simulation function. The cosmetic surgery simulation unit 25 is a processing unit that simulates the face after cosmetic surgery on the face of the image based on the facial image of the user. The downtime simulation unit 26 is a processing unit that simulates the face during downtime and displays it on the mobile terminal 100, and realizes the above-mentioned downtime simulation function by operating according to a program. The medical examination information search unit 27 is a processing unit that searches for information on specialists suitable for the cosmetic surgery desired by the user and cosmetic surgery facilities with many reviews.

[0023] Next, the operation of the cosmetic surgery system SYS when the cosmetic surgery app according to the embodiment is executed will be described. Figs. 3 to 5 are flowcharts showing an example of the operation of the cosmetic surgery system SYS when the cosmetic surgery app according to the embodiment is executed. The cosmetic surgery app is installed on the mobile terminal 100, for example, and is activated when the user taps (lightly taps with a finger) an icon displayed on the input display unit 10. Then, when the cosmetic surgery app is executed, the cosmetic surgery system SYS operates, for example, as follows.

[0024] First, the mobile terminal 100 acquires a face image (step S01). In step S01, for example, the imaging unit 14 of the mobile terminal 100 is used to acquire a plurality of different face images of the user himself / herself and a plurality of different face images of the user's parents. These plurality of face images are images taken, for example, at different times and places.

[0025] The cosmetic surgery system SYS first performs a simulation of changes over time.

[0026] First, a plurality of different face images are displayed on the input display unit 10 of the mobile terminal 100 so that the user can select from the plurality of face images a face image of the user to be used in the simulation. That is, a face image selection screen including a plurality of face images of the user himself / herself is displayed on the input display unit 10 (step S02). The user selects a face image to be used in the simulation from the plurality of face images of the user displayed on the input display unit 10, and designates the selected face image by tapping, for example.

[0027] Furthermore, a plurality of different facial images are displayed on the input display unit 10 of the mobile terminal 100 so that the user can select from the plurality of facial images the facial images of the user's father and mother to be used in the simulation. That is, a parent-and-mother facial image selection screen including the plurality of facial images of the user's father and mother is displayed on the input display unit 10 (step S03). On the parent-and-mother facial image selection screen, the user selects the facial images of the father and mother to be used in the simulation from the plurality of different facial images displayed on the input display unit 10, and designates the selected facial image by tapping, for example.

[0028] Following step S02 and step S03, a parent face image synthesis ratio input screen is displayed on the input display unit 10 (step S04). This parent face image synthesis ratio input screen is a screen for the user to input the synthesis ratio related to the parent face image specified by the user as described above. The user inputs the synthesis ratio of the parent face image into the parent face image synthesis ratio input screen. At this time, the input synthesis ratio is determined taking into consideration which of the parents the user's face resembles. For example, if the user's face resembles the father's face more than the mother's face, the synthesis ratio of the father's face image is set higher than that of the mother, and conversely, if the user's face resembles the mother's face more than the father's face, the synthesis ratio of the mother's face image is set higher than that of the father. In the parent face image synthesis ratio input screen, for example, the synthesis ratio of the father's face image and the mother's face image to the user's face image is input as "20%, 30%", "10%, 40%", etc.

[0029] Next, the mobile terminal 100 transmits data of the user's face image and the face images of the parents designated by the user, and information on the above-mentioned synthesis ratio, to the server 200. Then, the mobile terminal 100 instructs the server 200 to generate a composite face image of the face images of the user and the parents by synthesizing the face images of the user's parents with the face image of the user at the synthesis ratio (step S05). The mobile terminal 100 also instructs the server 200 to transmit the generated composite face image data to the mobile terminal 100.

[0030] When the server 200 receives such an instruction, the aging simulation unit 23 synthesizes the facial images of the parents and the user's facial image according to the synthesis ratio to generate a synthetic facial image (step S06). This synthetic facial image is an image obtained by synthesizing the facial images of the user and the user's parents at a set ratio. Therefore, if the synthesis ratio of the mother's facial image is set higher than that of the father's facial image in step S04, the face of the generated synthetic facial image will be closer to the mother's face than the father's face, and conversely, if the synthesis ratio of the father's facial image is set higher than that of the mother's facial image, the face of the generated synthetic facial image will be closer to the father's face than the mother's face. Since the shape and features of a child's face are influenced by inheritance from the parents, it is considered that the child's future face will gradually resemble the current faces of the parents, so by generating a synthetic facial image of the parent and child facial images in this way, it is possible to simulate the expected future face of the child (user). The server 200 transmits the generated synthetic facial image data, i.e., the result of the simulation, to the mobile terminal 100 (step S07).

[0031] When the mobile terminal 100 receives the composite face image data, the composite face image is displayed on the input display unit 10. That is, a simulation result of the user's predicted future face if no cosmetic surgery is performed is displayed on the input display unit 10 of the mobile terminal 100 (step S08). From this simulation result, the user can decide whether or not to undergo cosmetic surgery, and if it is decided that cosmetic surgery is recommended, can specify the area for cosmetic surgery.

[0032] Next, the cosmetic surgery system SYS executes a beautification simulation. The user's facial image to be beautified in the beautification simulation may be an image of the user's face used in the above-mentioned aging simulation (i.e., the user's current face), or it may be a facial image generated by the aging simulation.

[0033] First, a beautification range selection screen that allows the user to select the range of the user's face image to be beautified by the beautification simulation is displayed on the input display unit 10 of the mobile terminal 100 (step S09). This selection screen allows the user to select either the entire face or a part of the face as the range to be beautified by the beautification simulation. The user selects whether the entire face or a part of the face is to be beautified by tapping on one of the letters.

[0034] In step S09, if the area that the user wishes to be beautify through the beautification simulation is the entire face, the process skips step S10 (to be described later) and proceeds to step S11.

[0035] On the other hand, in step S09, if the area the user wishes to be beautify through the beautification simulation is a part of the face (e.g., eyes or nose), then following step S09, the input display unit 10 of the mobile terminal 100 displays a beautification part selection screen that allows the user to select a part to be beautified through the beautification simulation (e.g., eyes or nose) and an image of the part after beautification (e.g., an image of the eyes or nose cut out from a face image of a person who is considered beautiful) (step S10). On this beautification part selection screen, for example, the name of the face part and a plurality of images of the part after beautification are displayed, so that the user can select the face part to be beautified through the beautification simulation and the image after beautification. On the beautification part selection screen, the user selects the part to be beautified through the beautification simulation by, for example, tapping the name of the face part, and selects and specifies the image to be used in the beautification simulation by, for example, tapping the image of the part after beautification.

[0036] Next, the mobile terminal 100 requests the server 200 to execute a beautification simulation (step S11). At this time, if the area of ​​the face that the user wishes to be beautify through the beautification simulation is not the entire face but a part of the face, the mobile terminal 100 transmits to the server 200 data on the part to be beautified through the beautification simulation designated by the user and an image of the part after beautification.

[0037] When the server 200 receives the request, the beautification simulation unit 24 executes a beautification simulation (step S12). The beautification simulation is performed by executing a predetermined process on the user's face image. That is, the beautification simulation unit 24 applies a predetermined algorithm to the image of the user's entire face to process it into a facial photograph of a beautiful model and output it, or replaces or synthesizes the image of the facial part specified in step S10 with the image of the part after beautification, thereby generating a beautified face image (beautified face image). The server 200 transmits data of the generated image, i.e., the beautification simulation result, to the mobile terminal 100 (step S13).

[0038] When the mobile terminal 100 receives the data of the beautified face image, the beautified face image is displayed on the input display unit 10. That is, a simulation result of the beautification of the user's face is displayed on the input display unit 10 of the mobile terminal 100 (step S14). From this simulation result, the user can imagine an ideal face that suits him or her.

[0039] Next, the cosmetic surgery system SYS executes a cosmetic surgery simulation. The user's face image used in the cosmetic surgery simulation may be the user's face image used in the above-mentioned aging simulation, or may be a face image generated by the aging simulation.

[0040] First, a treatment site selection screen for the user to select a treatment site is displayed on the input display unit 10 of the mobile terminal 100 (step S15). For example, the names of a plurality of different treatment sites are displayed on this treatment site selection screen. The user selects the site on which the user wants to have cosmetic surgery from the displayed names of the plurality of treatment sites and specifies the name by tapping on the name, for example.

[0041] A treatment method selection screen for selecting a treatment method and a degree of change is displayed on the input display unit 10 of the mobile terminal 100 (step S16). For example, the names of a plurality of different treatment methods are displayed on this treatment method selection screen. The user selects a desired treatment method from the displayed names of the plurality of treatment methods and specifies the name by tapping on the name, for example. The user also specifies the degree of change on the treatment method selection screen. The degree of change refers to, for example, the magnitude of change before and after treatment in a specified treatment area, and the degree of change is, for example, a numerical value corresponding to the amount of change.

[0042] Next, the mobile terminal 100 transmits data indicating the treatment area, treatment method, and degree of change specified by the user to the server 200, and requests the server 200 to simulate the user's face after the cosmetic surgery based on this data (step S17).

[0043] When the server 200 receives the request, the cosmetic surgery simulation unit 25 executes a simulation of the face after the cosmetic surgery (step S18). In this step S18, the cosmetic surgery simulation unit 25 simulates the face after the cosmetic surgery using a learning model generated by machine learning using teacher data in which the name of the treatment site, the image of the treatment site before the treatment, the name of the treatment method, and the degree of change in the cosmetic surgery performed in the past are used as input variables, and the actual image of the treatment site after the treatment is used as output variables. Specifically, the cosmetic surgery simulation unit 25 inputs, for example, the name of the treatment site, the image of the treatment site, the name of the treatment method, and the degree of change specified by the user into this learning model, and simulates the face in the image output as the expected face after the cosmetic surgery. Note that the image of the treatment site after the treatment included in the teacher data is an image of the treatment site after the downtime has elapsed, that is, an image of the treatment site after the swelling, internal bleeding, and swelling of the treatment site that occurred immediately after the treatment have subsided. Therefore, the face image output from the learning model is an image of the face expected after the downtime has elapsed. The server 200 inputs the user's facial image, etc. into the learning model, and transmits the facial image output from the learning model, i.e., data of the image of the face after cosmetic surgery generated by simulation, to the mobile terminal 100 (step S19).

[0044] When the mobile terminal 100 receives the data of the image of the simulated face after cosmetic surgery, the image is displayed on the input display unit 10. That is, the result of the simulation of the user's face after cosmetic surgery is displayed on the input display unit 10 of the mobile terminal 100 (step S20). From the simulation result, the user can imagine what the face will look like after cosmetic surgery.

[0045] Next, the cosmetic surgery system SYS performs a simulation of the face during downtime after cosmetic surgery. The user's face image used in the downtime simulation may be the user's face image used in the above-mentioned aging simulation, or may be a face image generated by the aging simulation.

[0046] The mobile terminal 100 transmits data indicating the treatment area, treatment method, and degree of change specified by the user on the treatment area selection screen and treatment method selection screen described above to the server 200, and requests the server 200 to simulate the face during downtime based on this data (step S21).

[0047] When the server 200 receives the request, the cosmetic surgery simulation unit 25 executes a simulation of the face during the downtime (step S22). In this step S22, the downtime simulation unit 26 performs a simulation using a learning model generated by machine learning using teacher data in which, for example, an image of the treatment site before the treatment, the treatment method, and the degree of change in a cosmetic surgery performed in the past are used as input variables, and an actual face image during the downtime of the treatment site is used as an output variable. That is, the downtime simulation unit 26 inputs the image of the treatment site specified by the user, the treatment method, and the degree of change into this learning model, and simulates the face of the image output as a face during the downtime. The server 200 transmits the face image output from this learning model, i.e., the data of the simulated face image during the downtime, to the mobile terminal 100 (step S23).

[0048] When the mobile terminal 100 receives the simulated image data of the face during the downtime, the image is displayed on the input display unit 10 (step S24). That is, the input display unit 10 of the mobile terminal 100 displays the results of a simulation of the user's face during the downtime after cosmetic surgery. From the simulation results, the user can imagine in advance what their own face will look like during the downtime if they have cosmetic surgery.

[0049] Next, the cosmetic surgery system SYS provides information on cosmetic surgery specialists and cosmetic surgery facilities (such as cosmetic surgery clinics).

[0050] First, the mobile terminal 100 requests the server 200 to search for a suitable specialist and a cosmetic surgery facility with many reviews according to the cosmetic surgery desired by the user (step S25).

[0051] When the server 200 receives this request, the consultation information search unit 27 searches, for example, from the Internet, for information on specialists and cosmetic surgery facilities suitable for the cosmetic surgery desired by the user (step S26). Specifically, the consultation information search unit 27 searches, for example, for doctors and cosmetic surgery facilities that specialize in the treatment area and treatment method selected by the user on the treatment area selection screen and treatment method selection screen described above. In this case, with regard to cosmetic surgery facilities, the consultation information search unit 27 searches, for example, for facilities with a large number of word-of-mouth reviews. The server 200 then transmits the search results to the mobile terminal 100 (step S27).

[0052] When the mobile terminal 100 receives the search results, a list of the search results is displayed on the input / display unit 10. That is, a search result screen including information including the names of specialists suitable for the treatment and treatment method for the area to be cosmetic surgery, as well as information on cosmetic surgery facilities with many word-of-mouth reviews, is displayed on the mobile terminal 100 (step S27).

[0053] Then, when the user specifies the name of a specialist or the name of a cosmetic surgery facility displayed on the search result screen, for example by tapping, the screen of the input display unit 10 transitions from the search result screen to a consultation appointment screen (step S29). The consultation appointment screen is, for example, a reservation reception screen of the website of the cosmetic surgery facility where the specialist specified by the user works, or a reservation reception screen of the website of the cosmetic surgery facility specified by the user. The user can make a reservation for a cosmetic surgery consultation on this reservation reception screen. Once the reservation is completed, the cosmetic surgery system SYS transmits data of the above simulation results (the above composite face image, beautified face image, and simulated images of the face after cosmetic surgery and during downtime) to the cosmetic surgery facility with which the reservation was made.

[0054] Although the operation of the cosmetic surgery system SYS has been described above, the operation of the cosmetic surgery system SYS is not limited to the above, and various modifications are possible. For example, in the operation of the cosmetic surgery system SYS described above, the process of synthesizing face images in step S06 of the aging simulation is executed by the server 200 according to the cosmetic surgery application, but the process may be executed by the mobile terminal 100 instead of the server 200. That is, the cosmetic surgery application may execute the process of the aging simulation in the order of steps S01 to S04, step S06, and step S08 on the mobile terminal 100. Also, some of the steps described above may be omitted. For example, the user may use only the aging simulation function (the process of steps S01 to S08), in which case only steps S01 to S08 are executed, and the execution of steps S09 and onward is omitted. Also, the steps described above may be executed in a different order from the above. For example, step S02 and step S03 may be executed simultaneously or in the reverse order. Also, the steps described above may be executed in an appropriate combination. Furthermore, although the above-mentioned aging simulation is performed for the entire face, it is also possible to simulate specific parts of the face (e.g., eyes, nose, mouth, etc.). For example, it is also possible to focus on the eyes and simulate only the parts surrounding the eyes. In such a case, for example, the user selects the part to be simulated on the treatment part selection screen in step S15. Then, the aging simulation of the part is performed by synthesizing the images of the user's face and the parent's face images of the part selected by the user (e.g., the eyes).

[0055] In addition, in the above description of the operation of the cosmetic surgery system SYS, the cosmetic surgery app is installed on the mobile terminal 100, but this is not limited to this, and the cosmetic surgery app may be, for example, a program stored in the server 200 that causes the mobile terminal 100 and the server 200 to execute the above-mentioned processing.

[0056] According to the cosmetic surgery app according to the embodiment described above, the following effects can be achieved by having the cosmetic surgery system SYS execute the app. That is, the cosmetic surgery app causes the cosmetic surgery system SYS to simulate the expected face of the user when it changes over time without cosmetic surgery. However, the simulation is not performed using facial image data of a person who is not related to the user, as in the case where the simulation is generated using a learning model that is machine-learned using teacher data including facial images of other people, but using facial images of the user and the user's parents. Therefore, the user can obtain a more accurate simulation result, and can accurately narrow down the area that requires cosmetic surgery from the result. In addition, since a facial image that has been beautified based on the user's facial image is displayed on the mobile terminal 100, the user can obtain an ideal facial image that allows for cosmetic surgery without difficulty. In addition, since an image of the expected face after cosmetic surgery is displayed on the mobile terminal 100, the user can imagine the face after cosmetic surgery. In addition, since a search result screen including information on specialists for the treatment area and information on facilities with many reviews is displayed on the mobile terminal 100, the user can safely receive the treatment by requesting cosmetic surgery from the displayed specialist. This makes it possible to avoid situations where cosmetic surgery is performed by someone with little experience, resulting in the surgery failing.

[0057] In addition, there is also atavism, a genetic phenomenon in which an individual's genetic traits are not expressed in the parent generation, but appear to be inherited from grandparents or previous generations, skipping generations. Examples of this include male pattern baldness and ABO blood type. If atavism of the face can be scientifically proven, it will be possible to simulate not only using the facial images of the father and mother as described above, but also using the facial images of the father's parents or the mother's parents. Therefore, the technical scope of the present invention does not exclude a mechanism for simulating using the facial images of the grandmother and grandfather in addition to or instead of the facial images of the user's parents. It is believed that this will provide a safer and more natural cosmetic surgery. [Explanation of symbols]

[0058] 100 mobile devices 200 Servers

Claims

1. A cosmetic surgery application software that realizes a function of displaying information about cosmetic surgery on a mobile terminal that can communicate with a server, By causing a computer of a system including the server and the mobile device to execute the above-mentioned a face image of the user input to the mobile terminal and face images of the user's father and mother are synthesized in accordance with an input synthesis ratio to generate a synthetic face image, thereby simulating a face as it changes over time, and displaying the result of this simulation on the mobile terminal; a beautification simulation function and a cosmetic surgery simulation function are used to simulate what the face of the user's facial image would look like if it were beautified and what the face of the user's facial image would look like if it were cosmetically surgically treated, and these simulation results are displayed on the mobile terminal; Cosmetic surgery application software that displays on the mobile terminal information including specialists suitable for treating the area in accordance with the name of the area to be treated input into the mobile terminal.

2. The cosmetic surgery application software of claim 1, further comprising a downtime simulation function for simulating a face during downtime after cosmetic surgery for the synthetic face image to generate a face image during the downtime, and displaying the face image during the downtime on the mobile terminal.

3. displaying a plurality of facial images of a user on the mobile device; 3. The cosmetic surgery application software according to claim 1, wherein when generating the composite face image, a face image selected by a user from the plurality of face images of the user is used as the user's face image to generate the composite face image.

Citation Information

Patent Citations

  • Virtual face image creation device and virtual face image creation method

    JP2010224802A

  • Treatment support system

    JP2020067868A

  • JP2010‐224802A

Cited By

  • Cosmetic surgery simulation system, information processing device, program, information processing method, machine learning system, and machine learning method

    JP7825909B1