Face photo beautifying method, face photo beautifying device and computer program product

Abstract

The invention provides a face photo beautifying method, a face photo beautifying device and a computer program product. The method comprises the following steps: acquiring an original face image of a user; a generation step of generating an idealized face image at least according to the original face image and the optimized cue word, and changing the face of the original face image according to the optimized cue word and the idealized face image to generate a beautified face image; an extraction step of extracting facial feature data in the beautified face image to obtain first facial feature data; a feature comparison step: performing feature comparison on the first facial feature data and the second facial feature data to obtain a deviation degree; and a determination step of determining the beautified face image as a target face image when the deviation degree is less than or equal to a deviation threshold. According to the method and the device, the problem that the generated beautified image is distorted and unnatural due to global unified processing of the whole face in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of image beautification technology, and in particular to a face photo beautification method, a face photo beautification device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the past, beauty technology was usually based on global adjustments of filters and fixed parameters. Although this method can quickly improve facial features, it is easy to cause over-beautification, loss of naturalness, and it is difficult to fine-tune individual features. As users' requirements for beauty effects increase, the market urgently needs a more intelligent and personalized beauty solution that can achieve a more natural beautification effect while maintaining the uniqueness of the user's appearance.

[0003] At present, traditional beauty technology often processes the entire face uniformly, and adjusts according to preset parameters regardless of the user's facial features. Although this method is convenient, it is easy to cause excessive beauty, especially in the adjustment of facial features and face shape, which may distort the user's facial features and produce exaggerated or unnatural effects. The existing technology is usually unable to perform personalized beauty according to the user's specific facial features, but relies on fixed beauty templates, which makes it difficult for different users to reflect personalized needs when using the same beauty effect. In addition, the precise optimization of certain facial details (such as eyes, nose, etc.) is also inadequate, and it is difficult to achieve fine local adjustments. While improving the beauty, the existing beauty technology often ignores the naturalness of the appearance. Especially when the user wants the beauty effect to be natural and not obvious, it is difficult for the existing technology to achieve fine beautification adjustments, which easily causes a huge difference between the beauty effect and the user's original appearance, thereby reducing the user's satisfaction and user experience. Summary of the invention

[0004] The main purpose of the present application is to provide a face photo beautification method, a face photo beautification device, a computer-readable storage medium and a computer program product, so as to at least solve the problem in the prior art that the whole face is often processed globally and uniformly, resulting in distortion and unnaturalness of the generated beautified image.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for beautifying a face photo is provided, comprising: obtaining an original face image of a user; a generating step, generating an idealized face image based on at least the original face image and an optimization prompt word, and performing a face swap on the original face image based on the optimization prompt word and the idealized face image to generate a beautified face image, wherein the optimization prompt word is a prompt word for optimizing facial features to be beautified, and the optimization prompt word includes a beautification position and a beautification degree; an extracting step, extracting facial feature data from the beautified face image to obtain first facial feature data; performing a feature comparison between the first facial feature data and the second facial feature data to obtain a deviation degree, wherein the second facial feature data is facial feature data extracted from the original face image; and a determining step, determining that the beautified face image is a target face image when the deviation degree is less than or equal to a deviation threshold.

[0006] Optionally, after obtaining the original facial image of the user, the method further includes: extracting facial feature data of the original facial image to obtain the second facial feature data; analyzing the difference between the second facial feature data and the corresponding target standard facial feature data to obtain feature deviation data, wherein the target standard facial feature data is the facial feature data extracted from the standard facial image; inputting the feature deviation data into a large language model to generate the optimized prompt words, wherein the large language model is a convergent model obtained by training the large language model using machine learning with multiple groups of first training data, wherein each group of the first training data includes sample feature deviation data and sample optimization prompt words corresponding to the sample feature deviation data.

[0007] Optionally, before analyzing the difference between the second facial feature data and the target standard facial feature data to obtain feature deviation data, the method also includes: extracting the facial feature data of all the standard face images in the standard face library to obtain multiple groups of standard facial feature data, one group of the standard facial feature data corresponding to one standard face image; matching the first facial feature data with the standard facial feature data in sequence to obtain a matching degree; and determining the standard facial feature data corresponding to the highest matching degree as the target standard facial feature data.

[0008] Optionally, generating an idealized face image at least according to the original face image and the optimization prompt word, and performing a face-changing operation on the original face image according to the optimization prompt word and the idealized face image to generate a beautified face image, at least comprises: inputting the original face image and the optimization prompt word into a first image-generated image model, and using the first image-generated image model to perform full-face beautification according to the optimization prompt word to generate the idealized face image, wherein the first image-generated image model is a convergent model obtained by training a first predetermined model using machine learning through multiple sets of second training data, and each set of the second training data includes a first input data The invention relates to a method for beautifying a facial image, wherein the first input data comprises a sample original facial image and a sample optimized prompt word, and the first output data comprises a sample idealized facial image; in the absence of a specified prompt word, selecting a beautification part whose deviation degree is greater than a set deviation degree according to the feature deviation data, and the specified prompt word is a prompt word for the facial features area specified by the user to be beautify; selecting a corresponding prompt word from the optimized prompt words according to the beautification part to obtain a first prompt word; and locally fusing the beautification part in the idealized facial image into the original facial image according to the first prompt word to generate the beautification facial image.

[0009] Optionally, after the original facial image and the optimized prompt word are input into a first image-generated image model, and the first image-generated image model is used to perform full-face beautification according to the optimized prompt word to generate the idealized facial image, the method further includes: in the presence of the designated prompt word, extracting the corresponding designated beautification area in the idealized facial image according to the designated prompt word; selecting the corresponding prompt word from the optimized prompt word according to the designated beautification area to obtain a second prompt word; and locally fusing the beautification part in the idealized facial image into the original facial image according to the second prompt word to generate the beautified facial image.

[0010] Optionally, in the presence of a designated prompt word, an idealized facial image is generated at least according to the original facial image and the optimized prompt word, and the original facial image is replaced according to the optimized prompt word and the idealized facial image to generate a beautified facial image, further comprising: inputting the original facial image, the designated prompt word and the optimized prompt word into a second image-generated image model, so as to use the second image-generated image model to perform local beautification according to the designated prompt word to generate the idealized facial image, wherein the second image-generated image model is a convergent model obtained by training a second predetermined model using machine learning through multiple groups of third training data, each group of the third training data includes second input data and second output data, the second input data includes a sample original facial image, a sample designated prompt word and a sample optimized prompt word, and the second output data includes a sample idealized facial image; extracting a corresponding designated beautification area in the idealized facial image according to the designated prompt word; and locally fusing the designated beautification area in the idealized facial image into the original facial image according to the designated prompt word to generate the beautified facial image.

[0011] Optionally, after performing feature comparison between the first facial feature data and the standard facial feature data to obtain the deviation, the method further includes: when the deviation is greater than the deviation threshold, determining that the beautified facial image is over-beautified; feeding back the determination result to the large language model, so as to use the large language model to correct and adjust the optimization prompt words to obtain the adjusted optimization prompt words, wherein the large language model is a convergent model obtained by training the large language model using machine learning with multiple groups of first training data, each group of the first training data including sample feature deviation data and sample optimization prompt words corresponding to the sample feature deviation data; and using the adjusted optimization prompt words to re-execute the generation step, the extraction step, the feature comparison step and the determination step in sequence to obtain the target facial image.

[0012] According to another aspect of the present application, a facial photo beautification device is provided, the device comprising: an acquisition unit, for acquiring an original facial image of a user; a first generation unit, for executing a generation step, generating an idealized facial image at least according to the original facial image and an optimization prompt word, and performing a face swap on the original facial image according to the optimization prompt word and the idealized facial image to generate a beautified facial image, wherein the optimization prompt word is a prompt word for optimizing facial features to be beautified, and the optimization prompt word includes a beautification position and a beautification degree; a first extraction unit, for executing an extraction step, extracting facial feature data from the beautified facial image to obtain first facial feature data; a feature comparison unit, for executing a feature comparison step, performing a feature comparison between the first facial feature data and target standard facial feature data to obtain a deviation, wherein the target standard facial feature data is the facial feature data extracted from a standard facial image; and a first determination unit, for executing a determination step, determining that the beautified facial image is a target facial image when the deviation is less than or equal to a deviation threshold.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described.

[0014] According to another aspect of the present application, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, any one of the methods described above is implemented.

[0015] By applying the technical solution of the present application, in a facial photo beautification method, first, an original facial image of a user is obtained; then, a generation step is performed to generate an idealized facial image based on at least the original facial image and an optimization prompt word, and the original facial image is replaced with a beautified facial image based on the optimization prompt word and the idealized facial image to generate the beautified facial image, wherein the optimization prompt word is a prompt word for optimizing facial features to be beautified, and the optimization prompt word includes a beautification position and a beautification degree; then, an extraction step is performed to extract facial feature data from the beautified facial image to obtain first facial feature data; then, a feature comparison step is performed to compare the first facial feature data with the second facial feature data to obtain a deviation, wherein the second facial feature data is facial feature data extracted from the original facial image; finally, a determination step is performed to determine that the beautified facial image is a target facial image when the deviation is less than or equal to a deviation threshold. This application combines image generation technology with face-changing technology to generate an idealized face image that is similar to the user's appearance. It can also fuse the idealized face image with the user's original face image based on face-changing technology to achieve accurate replacement of local facial features, so that beautification is no longer limited to global adjustments, but can be refined and personalized based on the user's actual characteristics. And it can judge whether it is over-beautified to ensure that the beautification effect is both natural and meets the user's expectations. This application solves the problem that the existing technology often performs global unified processing on the entire face, resulting in distorted and unnatural beauty images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A hardware structure block diagram of a mobile terminal for executing a facial photo beautification method provided in an embodiment of the present application is shown;

[0017] Figure 2 A schematic diagram of a process of a facial photo beautification method provided according to an embodiment of the present application is shown;

[0018] Figure 3 A schematic diagram of a facial photo beautification process provided according to an embodiment of the present application is shown;

[0019] Figure 4 Another schematic diagram of a facial photo beautification process according to an embodiment of the present application is shown;

[0020] Figure 5 A structural block diagram of a facial photo beautification device provided according to an embodiment of the present application is shown.

[0021] The above drawings include the following reference numerals:

[0022] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0027] Face photo beautification: optimize the visual effects of face photos through algorithms, making facial features more delicate and the face shape more aesthetically pleasing;

[0028] Image-generated images: Generate similar or stylized new images from existing images using image generation technology;

[0029] Face-changing technology: By replacing the user's selected virtual facial features or facial features with their original face, local beautification and natural fusion effects are achieved;

[0030] As introduced in the background technology, the prior art often performs uniform processing on the entire face, and regardless of the user's facial features, it will be adjusted according to preset parameters. In order to solve the problem that the prior art often performs global uniform processing on the entire face, resulting in distorted and unnatural beautification images, the embodiments of the present application provide a face photo beautification method, a face photo beautification device, a computer-readable storage medium and a computer program product.

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0032] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal for a facial photo beautification method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0033] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the face photo beautification method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] In this embodiment, a method for beautifying facial photos that runs on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0035] Figure 2 FIG. 1 is a flow chart of a method for beautifying a face photo according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0036] Step S201, obtaining the original face image of the user.

[0037] Specifically, before image beautification, the user needs to upload a face photo of the user (i.e., the original face image mentioned above). The image needs to be clear and evenly illuminated so that facial features can be accurately recognized and analyzed. It is also necessary to verify whether the uploaded image meets the requirements, such as resolution, lighting conditions, etc. If not, the user is prompted to re-upload. Finally, the system obtains the original face image that meets the requirements uploaded by the user.

[0038] Step S202, a generation step, generates an idealized face image at least based on the original face image and the optimization prompt words, and replaces the original face image based on the optimization prompt words and the idealized face image to generate a beautified face image, wherein the optimization prompt words are prompt words for optimizing the facial features that need to be beautified, and the optimization prompt words include the beautification position and the beautification degree.

[0039] Specifically, extract facial feature data from the original face image, including the user's face shape and the position and size of the facial features. Determine the facial features that need to be beautified and the degree of beautification, and use image generation technology in combination with optimized prompt words to generate an idealized face image that is similar to the user's appearance. Align the idealized face image with the original face image to ensure that the positions of the facial features match. Apply face-changing technology to replace the facial features of the idealized face image with the original face image, while keeping other facial features unchanged, such as skin texture, to ensure the natural fusion of the facial features with other parts of the face and avoid obvious boundaries or unnatural effects.

[0040] Step S203, an extraction step, extracting facial feature data from the beautified face image to obtain first facial feature data.

[0041] Specifically, a facial feature extraction model (such as a deep learning model) is used to process the two sets of images to extract key facial feature points, including the coordinates of key parts such as eyes, nose, mouth, chin, etc. In order to analyze the effect after beautification and ensure that the beautification effect does not deviate from the user's expectations, it is necessary to extract facial feature data from the beautified image (i.e. the above-mentioned beautified face image) for subsequent analysis.

[0042] Step S204, a feature comparison step, performs feature comparison on the first facial feature data and the second facial feature data to obtain a degree of deviation, wherein the second facial feature data is facial feature data extracted from the original face image.

[0043] Specifically, two sets of facial feature data are compared, and the distance and angle difference between each corresponding feature point are calculated. This may involve mathematical methods such as Euclidean distance calculation and angle difference calculation, or the method of establishing a mathematical model. The relative position deviation of the facial features is calculated, such as the position of the eyes relative to the nose, the position of the mouth relative to the chin, etc., and the shape and size deviation of the facial features is calculated, such as the width of the eyes, the length of the nose, the width of the mouth, etc. All distance and angle differences are quantified into numerical values, and for each facial feature (such as eyes, nose, mouth, etc.), an overall deviation is calculated, which may be obtained by weighted average or other statistical methods. The difference between the user's face image after beauty enhancement and the original face image can be accurately identified, providing precise guidance for subsequent beauty processing, ensuring that the beauty effect is both natural and meets user expectations. If the deviation is too large, the system will automatically adjust the beauty intensity and beauty position before re-beautifying the operation.

[0044] Step S205, a determination step, in which, when the deviation is less than or equal to a deviation threshold, the beauty-enhanced facial image is determined to be a target facial image.

[0045] Specifically, a deviation threshold is set to determine whether the beautification effect is natural. If the deviation of the beautification face image is less than or equal to the deviation threshold, the image is determined to be a target face image, indicating that the beautification effect meets the needs and expectations of the user.

[0046] In this embodiment, first, an original face image of the user is obtained; then, a generation step is performed to generate an idealized face image at least based on the original face image and the optimization prompt words, and the original face image is replaced based on the optimization prompt words and the idealized face image to generate a beautified face image, wherein the optimization prompt words are prompt words for optimizing the facial features to be beautified, and the optimization prompt words include the beautification position and the beautification degree; then, an extraction step is performed to extract facial feature data from the beautified face image to obtain first facial feature data; then, a feature comparison step is performed to compare the first facial feature data with the target standard facial feature data to obtain a deviation, wherein the target standard facial feature data is the facial feature data extracted from the standard face image; finally, a determination step is performed to determine that the beautified face image is a target face image when the deviation is less than or equal to a deviation threshold. This application combines image generation technology with face-changing technology to generate an idealized face image that is similar to the user's appearance. It can also fuse the idealized face image with the user's original face image based on face-changing technology to achieve accurate replacement of local facial features, so that beautification is no longer limited to global adjustments, but can be refined and personalized based on the user's actual characteristics. And it can judge whether it is over-beautified to ensure that the beautification effect is both natural and meets the user's expectations. This application solves the problem that the existing technology often performs global unified processing on the entire face, resulting in distorted and unnatural beauty images.

[0047] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the face photo beautification method of the present application will be described in detail below in combination with specific embodiments.

[0048] In order to avoid unnecessary full-face adjustment, in an optional implementation, after the above step S201, the method further includes:

[0049] Step S301, extracting facial feature data of the original face image to obtain the second facial feature data;

[0050] Step S302, analyzing the difference between the second facial feature data and the corresponding target standard facial feature data to obtain feature deviation data, wherein the target standard facial feature data is the facial feature data extracted from the standard face image;

[0051] Step S303: input the feature deviation data into the large language model to generate the optimization prompt words. The large language model is a convergent model obtained by training the large language model using machine learning with multiple groups of first training data. Each group of the first training data includes sample feature deviation data and sample optimization prompt words corresponding to the sample feature deviation data.

[0052] In the above embodiment, Figure 3 A schematic diagram of a facial photo beautification process according to an embodiment of the present application is shown in FIG. Figure 3 As shown, it mainly involves a facial feature recognition module and a face-changing beauty module. First, after the user uploads the user's own face photo (i.e., the above-mentioned original face image) to the facial feature recognition module, the facial feature extraction model in the facial feature recognition module extracts the user's own facial feature data (i.e., the above-mentioned second facial feature data) from the above-mentioned original face image. At the same time, the facial feature extraction model will also extract the facial feature data in the standard face library (i.e., the above-mentioned target standard facial feature data) for subsequent feature comparison. The user's own second facial feature data and the facial feature data of the standard face (i.e., the above-mentioned target standard facial feature data) are input together into the feature comparison model in the facial feature recognition module to analyze the difference between the user's face and the standard face, calculate the feature deviations of the facial features, face shape and other parts, and obtain feature deviation data. These deviation data will serve as the basis for subsequent beauty processing. The feature deviation data finally obtained by the face feature discrimination module is input into the large language model of the face-changing beauty module. The large language model analyzes and generates the positions and degrees of beauty required for the user's facial features and face shape based on the feature deviation data, thereby further generating the prompt words required by the subsequent image generation model to guide the generation of face images, i.e., the above-mentioned optimization prompt words. Compared with the unified optimization of the whole face by traditional beauty technology, the present invention accurately identifies the parts that need to be optimized through feature deviation analysis and makes targeted adjustments. This precise local optimization avoids unnecessary full-face adjustments, which not only improves the refinement of the beauty effect, but also better meets the user's personalized needs for specific facial features.

[0053] It should also be noted that all models in the present invention, namely the facial feature extraction model, the feature comparison model, the large language model, the image-generated image model (the first image-generated image model and the second image-generated image model) and the face-changing model are convergent models trained using deep learning algorithms.

[0054] In order to provide personalized beauty to meet the personalized needs of users, in an optional implementation, before the above step S302, the method further includes:

[0055] Step S401, extracting the facial feature data of all the standard face images in the standard face library to obtain multiple groups of standard facial feature data, where one group of the standard facial feature data corresponds to one standard face image;

[0056] Step S402, matching the first facial feature data with the standard facial feature data in sequence to obtain a matching degree;

[0057] Step S403: determining the standard facial feature data corresponding to the highest matching degree as the target standard facial feature data.

[0058] In the above embodiment, if Figure 3 As shown, the facial feature extraction model will also extract facial feature data from the standard face library for subsequent feature comparison. Here, the user is allowed to upload a customized standard face image, and the facial feature discrimination module only extracts the facial feature data of the standard face uploaded by the user for subsequent feature comparison. It should be noted that the customized standard face image uploaded by the user used here is also only used for user feature comparison, and the facial features of the person in the uploaded "standard face" image will not be directly used in the face-changing model, thereby avoiding the problem of infringement of portrait rights. There may be multiple standard face images in the standard face library, so each time the user inputs an image, the facial feature data of all the above standard face images will be extracted to obtain the corresponding standard facial feature data, and the above first facial feature data will be matched with the above standard facial feature data in turn to obtain the matching degree, and the above standard facial feature data with the highest matching degree will be selected to determine as the above target standard facial feature data. The present invention provides a high degree of customization options, and users can customize the beauty standard according to their own preferences, and even upload their own standard face for feature bias.

[0059] In order to accurately identify the facial part that needs to be optimized for directional adjustment, in an optional implementation, the above step S202 includes:

[0060] Step S2021, inputting the original face image and the optimization prompt words into a first image-generated image model, so as to use the first image-generated image model to perform full-face beautification according to the optimization prompt words to generate the idealized face image, wherein the first image-generated image model is a converged model obtained by training a first predetermined model using machine learning with multiple sets of second training data, each set of the second training data includes first input data and first output data, the first input data includes a sample original face image and a sample optimization prompt word, and the first output data includes a sample idealized face image;

[0061] Step S2022, when there is no designated prompt word, selecting a beautification area having a deviation greater than a set deviation degree according to the feature deviation data, wherein the designated prompt word is a prompt word for the facial features area designated by the user to be beautify;

[0062] Step S2023, selecting a corresponding prompt word from the above-mentioned optimized prompt words according to the above-mentioned beautification part to obtain a first prompt word;

[0063] Step S2024: locally merge the beautification part in the idealized face image into the original face image according to the first prompt word to generate the beautification face image.

[0064] In the above embodiment, based on the original face image of the user as a reference image, the facial features that the user wants to beautify are used as prompt words, the first image model of the face-changing beautification module is input, and the optimized prompt words generated by the large language model are input at the same time. Based on the above input, the first image model generates an idealized face image similar to the user. In the absence of a user-specified beautification part prompt word, the system needs to select the part that needs beautification based on the feature deviation data. The feature deviation data is a measure of the difference between the user's facial features and the standard face. A deviation degree threshold can be set. Only parts with a deviation degree greater than this threshold will be selected for beautification. According to the selected beautification part, the system will select the corresponding prompt word from the optimized prompt words to form the first prompt word. These prompt words will guide the image model to focus on and optimize specific facial features when generating an idealized face image. This precise local optimization avoids unnecessary full-face adjustments and not only improves the fineness of the beautification effect. Of course, full-face beautification can also be performed, but the fineness of the beautification effect is not improved at this time. Finally, the system will partially fuse the beautified parts of the idealized face image into the original face image based on the first prompt word. This process involves image processing techniques such as image segmentation, feature alignment and fusion to ensure that the beautified parts are naturally integrated with the rest of the original image to generate the final beautified face image.

[0065] In order to ensure that the beautification effect is natural and does not deviate from the original appearance of the user, and to meet the personalized needs of the user, in an optional implementation, after the above step S2021, the method further includes:

[0066] Step S501, in the case where the designated prompt word exists, extracting the designated beauty area corresponding to the idealized face image according to the designated prompt word;

[0067] Step S502, selecting a corresponding prompt word from the above-mentioned optimized prompt words according to the above-mentioned designated beautification area to obtain a second prompt word;

[0068] Step S503 , locally integrating the beautification part in the idealized face image into the original face image according to the second prompt word, to generate the beautification face image.

[0069] In the above embodiment, based on the original face image of the user as a reference image, with the facial features that the user wants to beautify as prompt words, the first image model of the face-changing beautification module is input, and the optimized prompt words generated by the large language model are input at the same time. Based on the above input, the first image model generates an idealized face image similar to the user. Since the existing beautification technology usually cannot meet the personalized needs of users, the present invention provides a high degree of customization options. Users can customize the beautification standards according to their preferences. If the user has a specified beautification part prompt word (i.e., the specified prompt word), the system will select the beautification part according to these prompt words. These prompt words may be specified by the user according to his or her preferences. For example, the user may want to pay special attention to the beautification effect of the eyes or lips. Finally, the system will partially merge the specified beautification part in the idealized face image into the original face image according to the specified prompt word to generate the final beautification face image. While ensuring that the beautification effect is natural and does not deviate from the original appearance of the user, the personalized needs of the user are met at the same time. For example, if the user specifies that only the eye area should be beautified, the eye area needs to be extracted, and the prompt words related to the eyes (i.e., the second prompt words mentioned above) will be selected from the optimized prompt words, such as "darken eye color", "remove dark circles", etc. According to the second prompt words, it will be guided how to merge the beautification area of ​​the idealized face image into the original face image. In simple terms, firstly, the whole face is beautified according to the optimized prompt words generated by the large language model and the original face image to generate an idealized face image. After that, the specified beauty part in the idealized face image is selected and merged into the original face image according to the user's specified prompt words to generate a beauty image. The fusion operation can be achieved using image fusion technology (such as Alpha fusion, Poisson fusion, etc.). Local beauty processing of the user's original face image is achieved while maintaining the overall natural harmony. The present invention not only utilizes the image generation technology to generate idealized features, but also realizes accurate local beautification through face-changing technology, and finally achieves personalized and natural beauty effects.

[0070] In order to ensure that the beautification effect is natural and does not deviate from the original appearance of the user, and to meet the personalized needs of the user, in an optional implementation, the above step S202 further includes:

[0071] Step S2025, inputting the original face image, the specified prompt word and the optimized prompt word into a second image generation model, so as to use the second image generation model to perform local beautification according to the specified prompt word to generate the idealized face image, wherein the second image generation model is a convergent model obtained by training a second predetermined model using machine learning with multiple sets of third training data, each set of the third training data includes second input data and second output data, the second input data includes a sample original face image, a sample specified prompt word and a sample optimized prompt word, and the second output data includes a sample idealized face image;

[0072] Step S2026, extracting the designated beauty area corresponding to the idealized face image according to the designated prompt word;

[0073] Step S2027: locally fuse the designated beautification area in the idealized face image into the original face image according to the designated prompt word to generate the beautification face image.

[0074] In the above embodiment, if Figure 4 As shown, the unprocessed original face photo uploaded by the user, the facial area or feature that the user specifies to be optimized, such as "skin smoothing", "thin face", "bright eyes", etc., and the optimization prompt words generated by the large language model to guide the image model to beautify the specified area, such as "increase the brightness of the eyes" and "reduce facial shadows", are input into the second image model, and the second image model generates an idealized face image based on the input data. According to the specified prompt words, the extracted specified beauty area is aligned with the original face image to ensure that the fused part matches the corresponding part in the original face image. After completing the local fusion, the final beauty face image is generated. In simple terms, first, according to the user's specified prompt words, the corresponding prompt words and the original face image are selected from the optimization prompt words generated by the large language model for local beauty, and an idealized face image is generated (at this time, only the part of the specified prompt words is replaced). After that, according to the user's specified prompt words, the specified beauty part in the idealized face image is selected and fused to the original face image to generate a beauty image.

[0075] In order to ensure a natural image beautification effect, in an optional implementation, after the above step S203, the method further includes:

[0076] Step S601, when the deviation is greater than the deviation threshold, determining that the beautified face image is over-beautified;

[0077] Step S602, feeding back the determination result to the large language model, so as to use the large language model to modify and adjust the optimization prompt word, and obtain the adjusted optimization prompt word, wherein the large language model is a convergence model obtained by training the large language model by using machine learning with multiple sets of first training data, and each set of the first training data includes sample feature deviation data and sample optimization prompt words corresponding to the sample feature deviation data;

[0078] Step S603, using the adjusted optimized prompt words, the generation step, the extraction step, the feature comparison step and the determination step are sequentially re-executed to obtain the target face image.

[0079] In the above embodiment, when the above deviation is greater than the above deviation threshold, it means that there is a problem of excessive beautification in the current beautification of the image, that is, if the system detects that the beautification effect deviates too much from the original appearance of the user, it is determined to be excessive beautification, and the determination result is fed back to the large language model of the face-changing beautification module to re-optimize the prompt words and correct and adjust them, and then perform another round of idealized facial image generation and face-changing operations to ensure that the final beautification effect remains natural and not excessive, so as to obtain the final beautification image, that is, the above-mentioned target facial image.

[0080] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0081] The embodiment of the present application also provides a facial photo beautification device. It should be noted that the facial photo beautification device of the embodiment of the present application can be used to execute the facial photo beautification method provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0082] The following is an introduction to the face photo beautification device provided in an embodiment of the present application.

[0083] Figure 5 is a structural block diagram of a facial photo beautification device according to an embodiment of the present application. Figure 5 As shown, the device comprises:

[0084] The acquisition unit 10 is used to acquire the original face image of the user.

[0085] Specifically, before image beautification, the user needs to upload a face photo of the user (i.e., the original face image mentioned above). The image needs to be clear and evenly illuminated so that facial features can be accurately recognized and analyzed. It is also necessary to verify whether the uploaded image meets the requirements, such as resolution, lighting conditions, etc. If not, the user is prompted to re-upload. Finally, the system obtains the original face image that meets the requirements uploaded by the user.

[0086] The first generation unit 20 is used to execute the generation step, at least generate an idealized face image based on the above-mentioned original face image and the optimization prompt words, and replace the above-mentioned original face image according to the above-mentioned optimization prompt words and the above-mentioned idealized face image to generate a beautified face image, wherein the above-mentioned optimization prompt words are prompt words for optimizing the facial features that need to be beautified, and the above-mentioned optimization prompt words include beautification position and beautification degree.

[0087] Specifically, extract facial feature data from the original face image, including the user's face shape and the position and size of the facial features. Determine the facial features that need to be beautified and the degree of beautification, and use image generation technology in combination with optimized prompt words to generate an idealized face image that is similar to the user's appearance. Align the idealized face image with the original face image to ensure that the positions of the facial features match. Apply face-changing technology to replace the facial features of the idealized face image with the original face image, while keeping other facial features unchanged, such as skin texture, to ensure the natural fusion of the facial features with other parts of the face and avoid obvious boundaries or unnatural effects.

[0088] The first extraction unit 30 is used to execute the extraction step to extract the facial feature data in the beauty face image to obtain the first facial feature data.

[0089] Specifically, a facial feature extraction model (such as a deep learning model) is used to process the two sets of images to extract key facial feature points, including the coordinates of key parts such as eyes, nose, mouth, chin, etc. In order to analyze the effect after beautification and ensure that the beautification effect does not deviate from the user's expectations, it is necessary to extract facial feature data from the beautified image (i.e. the above-mentioned beautified face image) for subsequent analysis.

[0090] The feature comparison unit 40 is used to perform a feature comparison step, compare the first facial feature data with the second facial feature data to obtain a deviation, and the second facial feature data is the facial feature data extracted from the original face image.

[0091] Specifically, two sets of facial feature data are compared, and the distance and angle difference between each corresponding feature point are calculated. This may involve mathematical methods such as Euclidean distance calculation and angle difference calculation, or the method of establishing a mathematical model. The relative position deviation of the facial features is calculated, such as the position of the eyes relative to the nose, the position of the mouth relative to the chin, etc., and the shape and size deviation of the facial features is calculated, such as the width of the eyes, the length of the nose, the width of the mouth, etc. All distance and angle differences are quantified into numerical values, and for each facial feature (such as eyes, nose, mouth, etc.), an overall deviation is calculated, which may be obtained by weighted average or other statistical methods. The difference between the user's face image after beauty enhancement and the original face image can be accurately identified, providing precise guidance for subsequent beauty processing, ensuring that the beauty effect is both natural and meets user expectations. If the deviation is too large, the system will automatically adjust the beauty intensity and beauty position before re-beautifying the operation.

[0092] The first determining unit 50 is used to execute the determining step, and determine that the beauty-enhanced facial image is a target facial image when the deviation is less than or equal to a deviation threshold.

[0093] Specifically, a deviation threshold is set to determine whether the beautification effect is natural. If the deviation of the beautification face image is less than or equal to the deviation threshold, the image is determined to be a target face image, indicating that the beautification effect meets the needs and expectations of the user.

[0094] In this embodiment, the acquisition unit is used to acquire the original face image of the user; the first generation unit is used to execute the generation step, generate an idealized face image at least according to the original face image and the optimization prompt word, and perform face replacement on the original face image according to the optimization prompt word and the idealized face image to generate a beautified face image, wherein the optimization prompt word is a prompt word for optimizing the facial features to be beautified, and the optimization prompt word includes the beautification position and the beautification degree; the first extraction unit is used to execute the extraction step, extract facial feature data from the beautified face image, and obtain first facial feature data; the feature comparison unit is used to execute the feature comparison step, perform feature comparison between the first facial feature data and the second facial feature data, and obtain a deviation degree, wherein the second facial feature data is the facial feature data extracted from the original face image; the first determination unit is used to execute the determination step, and determine that the beautified face image is a target face image when the deviation degree is less than or equal to a deviation threshold. This application combines image generation technology with face-changing technology to generate an idealized face image that is similar to the user's appearance. It can also fuse the idealized face image with the user's original face image based on face-changing technology to achieve accurate replacement of local facial features, so that beautification is no longer limited to global adjustments, but can be refined and personalized based on the user's actual characteristics. And it can judge whether it is over-beautified to ensure that the beautification effect is both natural and meets the user's expectations. This application solves the problem that the existing technology often performs global unified processing on the entire face, resulting in distorted and unnatural beauty images.

[0095] In order to avoid unnecessary full-face adjustment, in an optional embodiment, the device further includes:

[0096] A second extraction unit, after acquiring an original face image of the user, extracts facial feature data of the original face image to obtain the second facial feature data;

[0097] An analyzing unit, analyzing a difference between the second facial feature data and corresponding target standard facial feature data to obtain feature deviation data, wherein the target standard facial feature data is the facial feature data extracted from a standard human face image;

[0098] The second generating unit is used to input the above-mentioned feature deviation data into the large language model to generate the above-mentioned optimization prompt words. The above-mentioned large language model is a convergent model obtained by training the large language model with machine learning through multiple groups of first training data. Each group of the above-mentioned first training data includes sample feature deviation data and sample optimization prompt words corresponding to the above-mentioned sample feature deviation data.

[0099] In the above embodiment, Figure 3A schematic diagram of a facial photo beautification process according to an embodiment of the present application is shown in FIG. Figure 3 As shown, it mainly involves a facial feature recognition module and a face-changing beauty module. First, after the user uploads the user's own face photo (i.e., the above-mentioned original face image) to the facial feature recognition module, the facial feature extraction model in the facial feature recognition module extracts the user's own facial feature data (i.e., the above-mentioned second facial feature data) from the above-mentioned original face image. At the same time, the facial feature extraction model will also extract the facial feature data in the standard face library (i.e., the above-mentioned target standard facial feature data) for subsequent feature comparison. The user's own second facial feature data and the facial feature data of the standard face (i.e., the above-mentioned target standard facial feature data) are input together into the feature comparison model in the facial feature recognition module to analyze the difference between the user's face and the standard face, calculate the feature deviations of the facial features, face shape and other parts, and obtain feature deviation data. These deviation data will serve as the basis for subsequent beauty processing. The feature deviation data finally obtained by the face feature discrimination module is input into the large language model of the face-changing beauty module. The large language model analyzes and generates the positions and degrees of beauty required for the user's facial features and face shape based on the feature deviation data, thereby further generating the prompt words required by the subsequent image generation model to guide the generation of face images, i.e., the above-mentioned optimization prompt words. Compared with the unified optimization of the whole face by traditional beauty technology, the present invention accurately identifies the parts that need to be optimized through feature deviation analysis and makes targeted adjustments. This precise local optimization avoids unnecessary full-face adjustments, which not only improves the refinement of the beauty effect, but also better meets the user's personalized needs for specific facial features.

[0100] It should also be noted that all models in the present invention, namely the facial feature extraction model, the feature comparison model, the large language model, the image-generated image model (the first image-generated image model and the second image-generated image model) and the face-changing model are convergent models trained using deep learning algorithms.

[0101] In order to provide personalized beauty to meet the personalized needs of users, in an optional implementation, the device also includes:

[0102] A third extraction unit is used to extract the facial feature data of all the standard face images in the standard face library to obtain multiple groups of standard facial feature data, where one group of the standard facial feature data corresponds to one standard face image;

[0103] A matching unit, used for matching the first facial feature data with the standard facial feature data in sequence to obtain a matching degree;

[0104] The second determining unit is used to determine the standard facial feature data corresponding to the highest matching degree as the target standard facial feature data.

[0105] In the above embodiment, if Figure 3 As shown, the facial feature extraction model will also extract facial feature data from the standard face library for subsequent feature comparison. Here, the user is allowed to upload a customized standard face image, and the facial feature discrimination module only extracts the facial feature data of the standard face uploaded by the user for subsequent feature comparison. It should be noted that the customized standard face image uploaded by the user used here is also only used for user feature comparison, and the facial features of the person in the uploaded "standard face" image will not be directly used in the face-changing model, thereby avoiding the problem of infringement of portrait rights. There may be multiple standard face images in the standard face library, so each time the user inputs an image, the facial feature data of all the above standard face images will be extracted to obtain the corresponding standard facial feature data, and the above first facial feature data will be matched with the above standard facial feature data in turn to obtain the matching degree, and the above standard facial feature data with the highest matching degree will be selected to determine as the above target standard facial feature data. The present invention provides a high degree of customization options, and users can customize the beauty standard according to their own preferences, and even upload their own standard face for feature bias.

[0106] In order to accurately identify the facial part that needs to be optimized for directional adjustment, in an optional implementation manner, the first generating unit includes:

[0107] A first generation module inputs the original face image and the optimization prompt words into a first image generation model, so as to use the first image generation model to perform full face beautification according to the optimization prompt words to generate the idealized face image, wherein the first image generation model is a convergent model obtained by training a first predetermined model using machine learning with multiple sets of second training data, each set of the second training data includes first input data and first output data, the first input data includes a sample original face image and a sample optimization prompt word, and the first output data includes a sample idealized face image;

[0108] A first selection module, in the absence of a designated prompt word, selects a beautification area with a deviation greater than a set deviation degree according to the feature deviation data, wherein the designated prompt word is a prompt word for the facial features area designated by the user to be beautify;

[0109] A second selection module selects a corresponding prompt word from the above-mentioned optimized prompt words according to the above-mentioned beautification part to obtain a first prompt word;

[0110] The second generation module partially merges the beautification part in the idealized face image into the original face image according to the first prompt word to generate the beautification face image.

[0111] In the above embodiment, based on the original face image of the user as a reference image, the facial features that the user wants to beautify are used as prompt words, the first image model of the face-changing beautification module is input, and the optimized prompt words generated by the large language model are input at the same time. Based on the above input, the first image model generates an idealized face image similar to the user. In the absence of a user-specified beautification part prompt word, the system needs to select the part that needs beautification based on the feature deviation data. The feature deviation data is a measure of the difference between the user's facial features and the standard face. A deviation degree threshold can be set. Only parts with a deviation degree greater than this threshold will be selected for beautification. According to the selected beautification part, the system will select the corresponding prompt word from the optimized prompt words to form the first prompt word. These prompt words will guide the image model to focus on and optimize specific facial features when generating an idealized face image. This precise local optimization avoids unnecessary full-face adjustments and not only improves the fineness of the beautification effect. Of course, full-face beautification can also be performed, but the fineness of the beautification effect is not improved at this time. Finally, the system will partially fuse the beautified parts of the idealized face image into the original face image based on the first prompt word. This process involves image processing techniques such as image segmentation, feature alignment and fusion to ensure that the beautified parts are naturally integrated with the rest of the original image to generate the final beautified face image.

[0112] In order to ensure that the beautification effect is natural and does not deviate from the original appearance of the user, while meeting the personalized needs of the user, in an optional implementation, the device further includes:

[0113] A fourth extraction unit, configured to extract, in the presence of the designated prompt word, the designated beauty area corresponding to the idealized face image according to the designated prompt word;

[0114] A selection unit, configured to select a corresponding prompt word from the optimized prompt words according to the designated beautification area to obtain a second prompt word;

[0115] A fusion unit is used to partially fuse the beautification part in the idealized face image into the original face image according to the second prompt word to generate the beautification face image.

[0116] In the above embodiment, based on the original face image of the user as a reference image, with the facial features that the user wants to beautify as prompt words, the first image model of the face-changing beautification module is input, and the optimized prompt words generated by the large language model are input at the same time. Based on the above input, the first image model generates an idealized face image similar to the user. Since the existing beautification technology usually cannot meet the personalized needs of users, the present invention provides a high degree of customization options. Users can customize the beautification standards according to their preferences. If the user has a specified beautification part prompt word (i.e., the specified prompt word), the system will select the beautification part according to these prompt words. These prompt words may be specified by the user according to his or her preferences. For example, the user may want to pay special attention to the beautification effect of the eyes or lips. Finally, the system will partially merge the specified beautification part in the idealized face image into the original face image according to the specified prompt word to generate the final beautification face image. While ensuring that the beautification effect is natural and does not deviate from the original appearance of the user, the personalized needs of the user are met at the same time. For example, if the user specifies that only the eye area should be beautified, the eye area needs to be extracted, and the prompt words related to the eyes (i.e., the second prompt words mentioned above) will be selected from the optimized prompt words, such as "darken eye color", "remove dark circles", etc. According to the second prompt words, it will be guided how to merge the beautification area of ​​the idealized face image into the original face image. In simple terms, firstly, the whole face is beautified according to the optimized prompt words generated by the large language model and the original face image to generate an idealized face image. After that, the specified beauty part in the idealized face image is selected and merged into the original face image according to the user's specified prompt words to generate a beauty image. The fusion operation can be achieved using image fusion technology (such as Alpha fusion, Poisson fusion, etc.). Local beauty processing of the user's original face image is achieved while maintaining the overall natural harmony. The present invention not only utilizes the image generation technology to generate idealized features, but also realizes accurate local beautification through face-changing technology, and finally achieves personalized and natural beauty effects.

[0117] In order to ensure that the beautification effect is natural and does not deviate from the original appearance of the user, while meeting the personalized needs of the user, in an optional implementation manner, the first generating unit further includes:

[0118] a third generation module, inputting the original face image, the specified prompt word and the optimized prompt word into a second image generation model, so as to use the second image generation model to perform local beautification according to the specified prompt word to generate the idealized face image, wherein the second image generation model is a convergent model obtained by training a second predetermined model using machine learning with multiple sets of third training data, each set of the third training data includes second input data and second output data, the second input data includes a sample original face image, a sample specified prompt word and a sample optimized prompt word, and the second output data includes a sample idealized face image;

[0119] An extraction module extracts the designated beauty area corresponding to the idealized face image according to the designated prompt words;

[0120] The fourth generation module partially merges the designated beautification area in the idealized face image into the original face image according to the designated prompt word to generate the beautification face image.

[0121] In the above embodiment, if Figure 4 As shown, the unprocessed original face photo uploaded by the user, the facial area or feature that the user specifies to be optimized, such as "skin smoothing", "thin face", "bright eyes", etc., and the optimization prompt words generated by the large language model to guide the image model to beautify the specified area, such as "increase the brightness of the eyes" and "reduce facial shadows", are input into the second image model, and the second image model generates an idealized face image based on the input data. According to the specified prompt words, the extracted specified beauty area is aligned with the original face image to ensure that the fused part matches the corresponding part in the original face image. After completing the local fusion, the final beauty face image is generated. In simple terms, first, according to the user's specified prompt words, the corresponding prompt words and the original face image are selected from the optimization prompt words generated by the large language model for local beauty, and an idealized face image is generated (at this time, only the part of the specified prompt words is replaced). After that, according to the user's specified prompt words, the specified beauty part in the idealized face image is selected and fused to the original face image to generate a beauty image.

[0122] In order to ensure a natural image beautification effect, in an optional implementation manner, the device further includes:

[0123] A determination unit, configured to perform feature comparison between the first facial feature data and the standard facial feature data to obtain a deviation, and determine that the beautified face image is over-beautified if the deviation is greater than the deviation threshold;

[0124] a correction unit, configured to feed back the determination result to the large language model, so as to use the large language model to correct and adjust the optimized prompt word to obtain the adjusted optimized prompt word, wherein the large language model is a convergent model obtained by training the large language model using machine learning with multiple sets of first training data, and each set of the first training data includes sample feature deviation data and sample optimized prompt words corresponding to the sample feature deviation data;

[0125] The repeating unit is used to sequentially re-execute the generation step, the extraction step, the feature comparison step and the determination step using the adjusted optimization prompt words to obtain the target face image.

[0126] In the above embodiment, when the above deviation is greater than the above deviation threshold, it means that there is a problem of excessive beautification in the current beautification of the image, that is, if the system detects that the beautification effect deviates too much from the original appearance of the user, it is determined to be excessive beautification, and the determination result is fed back to the large language model of the face-changing beautification module to re-optimize the prompt words and correct and adjust them, and then perform another round of idealized facial image generation and face-changing operations to ensure that the final beautification effect remains natural and not excessive, so as to obtain the final beautification image, that is, the above-mentioned target facial image.

[0127] The above-mentioned face photo beautification device includes a processor and a memory, and the above-mentioned acquisition unit, the first generation unit and the first extraction unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0128] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem that the whole face is often processed globally and uniformly in the prior art, resulting in distortion and unnaturalness of the generated beauty image.

[0129] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0130] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is executed, the device where the computer-readable storage medium is located is controlled to execute the facial photo beautification method.

[0131] Specifically, the facial photo beautification method includes:

[0132] Step S201, obtaining the original face image of the user;

[0133] Step S202, a generating step, generating an idealized face image based on at least the original face image and the optimization prompt words, and performing a face-changing operation on the original face image based on the optimization prompt words and the idealized face image to generate a beautified face image, wherein the optimization prompt words are prompt words for optimizing the facial features to be beautified, and the optimization prompt words include beautification positions and beautification degrees;

[0134] Step S203, an extraction step, extracting facial feature data from the beautified face image to obtain first facial feature data;

[0135] Step S204, a feature comparison step, performing feature comparison on the first facial feature data and the second facial feature data to obtain a degree of deviation, wherein the second facial feature data is facial feature data extracted from the original face image;

[0136] Step S205, a determination step, in which, when the deviation is less than or equal to a deviation threshold, the beauty-enhanced facial image is determined to be a target facial image.

[0137] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the facial photo beautification method is executed when the program is run.

[0138] Specifically, the facial photo beautification method includes:

[0139] Step S201, obtaining the original face image of the user;

[0140] Step S202, a generating step, generating an idealized face image based on at least the original face image and the optimization prompt words, and performing a face-changing operation on the original face image based on the optimization prompt words and the idealized face image to generate a beautified face image, wherein the optimization prompt words are prompt words for optimizing the facial features to be beautified, and the optimization prompt words include beautification positions and beautification degrees;

[0141] Step S203, an extraction step, extracting facial feature data from the beautified face image to obtain first facial feature data;

[0142] Step S204, a feature comparison step, performing feature comparison on the first facial feature data and the second facial feature data to obtain a degree of deviation, wherein the second facial feature data is facial feature data extracted from the original face image;

[0143] Step S205, a determination step, in which, when the deviation is less than or equal to a deviation threshold, the beauty-enhanced facial image is determined to be a target facial image.

[0144] An embodiment of the present invention provides a face photo beautification system, the face photo beautification system comprising a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0145] Step S201, obtaining the original face image of the user;

[0146] Step S202, a generating step, generating an idealized face image based on at least the original face image and the optimization prompt words, and performing a face-changing operation on the original face image based on the optimization prompt words and the idealized face image to generate a beautified face image, wherein the optimization prompt words are prompt words for optimizing the facial features to be beautified, and the optimization prompt words include beautification positions and beautification degrees;

[0147] Step S203, an extraction step, extracting facial feature data from the beautified face image to obtain first facial feature data;

[0148] Step S204, a feature comparison step, performing feature comparison on the first facial feature data and the second facial feature data to obtain a degree of deviation, wherein the second facial feature data is facial feature data extracted from the original face image;

[0149] Step S205, a determination step, in which, when the deviation is less than or equal to a deviation threshold, the beauty-enhanced facial image is determined to be a target facial image.

[0150] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0151] Step S201, obtaining the original face image of the user;

[0152] Step S202, a generating step, generating an idealized face image based on at least the original face image and the optimization prompt words, and performing a face-changing operation on the original face image based on the optimization prompt words and the idealized face image to generate a beautified face image, wherein the optimization prompt words are prompt words for optimizing the facial features to be beautified, and the optimization prompt words include beautification positions and beautification degrees;

[0153] Step S203, an extraction step, extracting facial feature data from the beautified face image to obtain first facial feature data;

[0154] Step S204, a feature comparison step, performing feature comparison on the first facial feature data and the second facial feature data to obtain a degree of deviation, wherein the second facial feature data is facial feature data extracted from the original face image;

[0155] Step S205, a determination step, in which, when the deviation is less than or equal to a deviation threshold, the beauty-enhanced facial image is determined to be a target facial image.

[0156] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0158] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0161] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0162] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0163] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0164] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0165] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0166] 1) The face photo beautification method of the present application first obtains the original face image of the user; then, a generation step is performed to generate an idealized face image at least based on the original face image and the optimization prompt words, and the original face image is replaced according to the optimization prompt words and the idealized face image to generate a beautified face image, wherein the optimization prompt words are prompt words for optimizing the facial features to be beautified, and the optimization prompt words include the beautification position and the beautification degree; then, an extraction step is performed to extract facial feature data from the beautified face image to obtain first facial feature data; then, a feature comparison step is performed to compare the first facial feature data with the target standard facial feature data to obtain a deviation, wherein the target standard facial feature data is the facial feature data extracted from the standard face image; finally, a determination step is performed to determine that the beautified face image is a target face image when the deviation is less than or equal to a deviation threshold. This application combines image generation technology with face-changing technology to generate an idealized face image that is similar to the user's appearance. It can also fuse the idealized face image with the user's original face image based on face-changing technology to achieve accurate replacement of local facial features, so that beautification is no longer limited to global adjustments, but can be refined and personalized based on the user's actual characteristics. And it can judge whether it is over-beautified to ensure that the beautification effect is both natural and meets the user's expectations. This application solves the problem that the existing technology often performs global unified processing on the entire face, resulting in distorted and unnatural beauty images.

[0167] 2) The facial photo beautification device of the present application comprises an acquisition unit for acquiring an original facial image of a user; a first generation unit for executing a generation step, generating an idealized facial image at least according to the original facial image and the optimization prompt word, and performing a face swap on the original facial image according to the optimization prompt word and the idealized facial image to generate a beautified facial image, wherein the optimization prompt word is a prompt word for optimizing the facial features to be beautified, and the optimization prompt word includes a beautification position and a beautification degree; a first extraction unit for executing an extraction step, extracting facial feature data from the beautified facial image to obtain first facial feature data; a feature comparison unit for executing a feature comparison step, performing a feature comparison between the first facial feature data and the second facial feature data to obtain a deviation, wherein the second facial feature data is the facial feature data extracted from the original facial image; and a first determination unit for executing a determination step, determining that the beautified facial image is a target facial image when the deviation is less than or equal to a deviation threshold. This application combines image generation technology with face-changing technology to generate an idealized face image that is similar to the user's appearance. It can also fuse the idealized face image with the user's original face image based on face-changing technology to achieve accurate replacement of local facial features, so that beautification is no longer limited to global adjustments, but can be refined and personalized based on the user's actual characteristics. And it can judge whether it is over-beautified to ensure that the beautification effect is both natural and meets the user's expectations. This application solves the problem that the existing technology often performs global unified processing on the entire face, resulting in distorted and unnatural beauty images.

[0168] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A facial photo beautification method, characterized in that: include: Get the user's original face image; a generating step, generating an idealized face image at least according to the original face image and the optimization prompt words, and performing a face-changing operation on the original face image according to the optimization prompt words and the idealized face image to generate a beautified face image, wherein the optimization prompt words are prompt words for optimizing the facial features to be beautified, and the optimization prompt words include the beautification position and the beautification degree; An extraction step of extracting facial feature data from the beauty face image to obtain first facial feature data; A feature comparison step, performing feature comparison on the first facial feature data and the second facial feature data to obtain a deviation degree. The second facial feature data is the facial feature data extracted from the original face image; The determination step is to determine that the beauty facial image is a target facial image when the deviation is less than or equal to a deviation threshold.

2. The method according to claim 1, characterized in that: After obtaining the original face image of the user, the method further includes: Extracting facial feature data of the original facial image to obtain the second facial feature data; Analyzing the difference between the second facial feature data and the corresponding target standard facial feature data to obtain feature deviation data, wherein the target standard facial feature data is the facial feature data extracted from the standard human face image; The feature deviation data is input into a large language model to generate the optimization prompt words, wherein the large language model is a convergent model obtained by training the large language model using machine learning through multiple groups of first training data, and each group of the first training data includes sample feature deviation data and sample optimization prompt words corresponding to the sample feature deviation data.

3. The method according to claim 2, characterized in that Before analyzing the difference between the second facial feature data and the target standard facial feature data to obtain feature deviation data, the method further includes: Extracting the facial feature data of all the standard face images in the standard face library to obtain multiple groups of standard facial feature data, wherein one group of the standard facial feature data corresponds to one standard face image; Matching the first facial feature data with the standard facial feature data in sequence to obtain a matching degree; The standard facial feature data corresponding to the highest matching degree is determined as the target standard facial feature data.

4. The method according to claim 2, characterized in that: Generating an idealized face image at least according to the original face image and the optimization prompt word, and performing face replacement on the original face image according to the optimization prompt word and the idealized face image to generate a beautified face image, at least comprising: Inputting the original face image and the optimized prompt words into a first image-generated image model, so as to use the first image-generated image model to perform full-face beautification according to the optimized prompt words to generate the idealized face image, wherein the first image-generated image model is a convergent model obtained by training a first predetermined model using machine learning through multiple groups of second training data, each group of the second training data includes first input data and first output data, the first input data includes a sample original face image and a sample optimized prompt word, and the first output data includes a sample idealized face image; In the absence of a designated prompt word, selecting a beautification area having a deviation greater than a set deviation degree according to the feature deviation data, wherein the designated prompt word is a prompt word for the facial feature area designated by the user to be beautify; Selecting a corresponding prompt word from the optimized prompt words according to the beautification part to obtain a first prompt word; The beautification part in the idealized facial image is partially merged into the original facial image according to the first prompt word to generate the beautification facial image.

5. The method according to claim 4, characterized in that After inputting the original face image and the optimization prompt word into the first image-generated image model, and performing full-face beautification according to the optimization prompt word using the first image-generated image model to generate the idealized face image, the method further includes: In the case where the designated prompt word exists, extracting the designated beauty area corresponding to the idealized face image according to the designated prompt word; Selecting a corresponding prompt word from the optimized prompt words according to the designated beautification area to obtain a second prompt word; The beautification part in the idealized facial image is partially merged into the original facial image according to the second prompt word to generate the beautification facial image.

6. The method according to claim 4, characterized in that In the case where there is a designated prompt word, an idealized face image is generated at least according to the original face image and the optimized prompt word, and the original face image is replaced according to the optimized prompt word and the idealized face image to generate a beautified face image, further comprising: Inputting the original face image, the designated prompt word and the optimized prompt word into a second image-generated image model, so as to use the second image-generated image model to perform local beautification according to the designated prompt word to generate the idealized face image, wherein the second image-generated image model is a convergent model obtained by training a second predetermined model using machine learning with multiple sets of third training data, each set of the third training data includes second input data and second output data, the second input data includes a sample original face image, a sample designated prompt word and a sample optimized prompt word, and the second output data includes a sample idealized face image; Extracting the designated beauty area corresponding to the idealized face image according to the designated prompt word; The designated beautification area in the idealized face image is partially fused into the original face image according to the designated prompt word to generate the beautification face image.

7. The method according to claim 1, characterized in that After comparing the first facial feature data with the second facial feature data to obtain the degree of deviation, the method further includes: When the degree of deviation is greater than the deviation threshold, determining that the beautified facial image is over-beautified; Feeding back the determination result to the large language model, so as to modify and adjust the optimization prompt word by using the large language model to obtain the adjusted optimization prompt word, wherein the large language model is a convergent model obtained by training the large language model by using machine learning with multiple groups of first training data, and each group of the first training data includes sample feature deviation data and sample optimization prompt words corresponding to the sample feature deviation data; The generating step, the extracting step, the feature comparing step and the determining step are sequentially re-executed using the adjusted optimized prompt words to obtain the target face image.

8. A facial photo beautification device, characterized in that: The device comprises: An acquisition unit, used to acquire an original face image of a user; a first generating unit, configured to execute a generating step, generating an idealized face image at least according to the original face image and the optimization prompt word, and performing a face-changing operation on the original face image according to the optimization prompt word and the idealized face image to generate a beautified face image, wherein the optimization prompt word is a prompt word for optimizing the facial features to be beautified, and the optimization prompt word includes a beautification position and a beautification degree; A first extraction unit, configured to execute an extraction step to extract facial feature data from the beauty face image to obtain first facial feature data; a feature comparison unit, configured to perform a feature comparison step, compare the first facial feature data with the second facial feature data to obtain a deviation, wherein the second facial feature data is facial feature data extracted from the original facial image; The first determining unit is used to execute a determining step, and determine that the beauty facial image is a target facial image when the deviation is less than or equal to a deviation threshold.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images