Medical beauty content AI generation method and system based on multi-source heterogeneous data

Abstract

The invention provides a medical beauty content AI generation method and system based on multi-source heterogeneous data, and relates to the field of image processing, and the method comprises the steps: obtaining a plurality of face-lifting template images, and extracting face features corresponding to the face-lifting template images; acquiring a face image and medical beauty demand information of the user; extracting face features corresponding to the face image of the user, determining at least one target face-lifting template image from the multiple face-lifting template images, and then determining at least one candidate medical beauty item; obtaining feedback information of the user on the at least one candidate medical beauty item, and determining at least one target medical beauty item; and according to the at least one target medical beauty item, the face feature corresponding to the face image of the user and the face feature corresponding to the at least one target plastic face template image, generating a medical beauty effect prediction image of the user, and having the advantage of providing medical beauty reference information for the user.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for generating medical aesthetic content AI based on multi-source heterogeneous data. Background Art

[0002] Medical aesthetics aims to improve people's appearance through various medical technologies and cosmetic methods. Medical aesthetics gives people more choices, including botulinum toxin injections, laser treatments, and bone cutting. Correspondingly, there is a great deal of uncertainty in medical aesthetics. Different medical aesthetics methods have different corresponding medical aesthetic effects. In addition, even the same medical aesthetics method will produce different effects on different people. Existing technology cannot provide users with a preliminary reference to the effects that medical aesthetics projects can achieve.

[0003] Therefore, it is necessary to provide a medical beauty content AI generation method and system based on multi-source heterogeneous data to solve the above technical problems. Summary of the invention

[0004] The present invention provides a method for generating medical beauty content AI based on multi-source heterogeneous data, comprising: obtaining a plurality of cosmetic surgery template images; for each cosmetic surgery template image, extracting facial features corresponding to the cosmetic surgery template image; obtaining a facial image of a user; obtaining medical beauty demand information of the user; extracting facial features corresponding to the facial image of the user; determining at least one candidate cosmetic surgery template image from the plurality of cosmetic surgery template images according to the facial features of the facial image of the user and the facial features corresponding to each of the cosmetic surgery template images; determining at least one recommended cosmetic surgery template image from the at least one candidate cosmetic surgery template according to the medical beauty demand information of the user; and The method comprises the steps of: obtaining a face image of a cosmetic surgery template and generating a target cosmetic surgery template image; determining at least one target cosmetic surgery template image according to the user's feedback information on the at least one recommended cosmetic surgery template; determining at least one candidate cosmetic surgery project according to the user's medical cosmetic demand information, the facial features corresponding to the user's facial image, and the facial features corresponding to the at least one target cosmetic surgery template image; obtaining the user's feedback information on at least one candidate cosmetic surgery project, and determining at least one target cosmetic surgery project; generating a cosmetic surgery effect prediction image for the user according to the at least one target cosmetic surgery project, the facial features corresponding to the user's facial image, and the facial features corresponding to the at least one target cosmetic surgery template image.

[0005] Furthermore, extracting facial features corresponding to the plastic surgery template image includes: acquiring multiple sample facial images; determining multiple facial feature factors; for each sample facial image, extracting sample facial features of the sample facial image according to the multiple facial feature factors; for each facial feature factor, calculating a feature difference value corresponding to the facial feature factor based on the sample facial features of each sample facial image; for any two facial feature factors, calculating a difference association value corresponding to any two facial feature factors based on the sample facial features of each sample facial image; determining multiple target facial feature factors from the multiple facial feature factors according to the feature difference value corresponding to each facial feature factor and the difference association values ​​corresponding to any two facial feature factors; and extracting facial features corresponding to the plastic surgery template image according to the multiple target facial feature factors.

[0006] Furthermore, according to the feature difference value corresponding to each facial feature factor and the difference association value corresponding to any two facial feature factors, multiple target facial feature factors are determined from the multiple facial feature factors, including: according to the feature difference value corresponding to each facial feature factor, multiple first key facial feature factors are determined from the multiple facial feature factors; for each facial feature factor, according to the difference association value between the facial feature factor and each key facial feature factor, the association key value of the facial feature factor is calculated; according to the multiple first key facial feature factors and the association key value of each facial feature factor, multiple target facial feature factors are determined from the multiple facial feature factors.

[0007] Furthermore, at least one candidate plastic surgery template image is determined from the multiple plastic surgery template images based on the facial features of the user's facial image and the facial features corresponding to each of the plastic surgery template images, including: classifying the multiple plastic surgery template images according to the facial features corresponding to each of the plastic surgery template images to determine multiple plastic surgery template image classes; for each plastic surgery template image class, determining the class-center facial features corresponding to the plastic surgery template image class according to the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class; determining a target plastic surgery template image class from the multiple plastic surgery template image classes according to the facial features of the user's facial image and the class-center facial features corresponding to the plastic surgery template image class; and determining a candidate plastic surgery template image from the target plastic surgery template image class according to the facial features of the user's facial image and the facial features corresponding to each plastic surgery template image included in the target plastic surgery template image class.

[0008] Furthermore, based on the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class, the class-central facial features corresponding to the plastic surgery template image class are determined, including: for each plastic surgery template image class, based on the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class, the class-mean facial features corresponding to the plastic surgery template image class are determined; for each plastic surgery template image class, based on the class-mean facial features corresponding to each plastic surgery template image class, the total difference value of each facial feature factor of the plastic surgery template image class is determined; based on the total difference value of each facial feature factor of the plastic surgery template image class, the central facial feature factor corresponding to the plastic surgery template image class is determined; based on the central facial feature factor corresponding to the plastic surgery template image class and the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class, the class-central facial features corresponding to the plastic surgery template image class are determined.

[0009] Furthermore, based on the user's medical beauty demand information, at least one recommended plastic surgery template image is determined from the at least one candidate plastic surgery template image, including: based on the user's medical beauty demand information, determining the required target facial feature factors and the non-required target facial feature factors from the multiple target facial feature factors; for each candidate plastic surgery template image, calculating the key feature difference value corresponding to the candidate plastic surgery template image based on the required target facial feature factors, the facial features of the user's facial image and the facial features corresponding to each candidate plastic surgery template image; for each candidate plastic surgery template image, calculating the basic similarity value corresponding to the candidate plastic surgery template image based on the non-required target facial feature factors, the facial features of the user's facial image and the facial features corresponding to each candidate plastic surgery template image; based on the key feature difference value and the basic similarity value corresponding to each candidate plastic surgery template image, determining at least one recommended plastic surgery template image from the at least one candidate plastic surgery template image.

[0010] Furthermore, at least one candidate medical beauty project is determined based on the user's medical beauty demand information, facial features corresponding to the user's facial image, and facial features corresponding to the at least one target plastic surgery template image, including: using a project prediction model to determine at least one candidate medical beauty project based on the user's medical beauty demand information, facial features corresponding to the user's facial image, and facial features corresponding to the at least one target plastic surgery template image.

[0011] Furthermore, a predicted image of the user's medical beauty effect is generated based on at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to at least one target plastic surgery template, including: using an image generation model to generate a predicted image of the user's medical beauty effect by performing image reasoning through a reverse process based on at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to at least one target plastic surgery template.

[0012] Furthermore, the reverse process p of the image generation model θ (x t-1 |x t ) through the neural network q(x t-1 |x t ,x0) fitting results: p θ (x t-1 |x t )=N(x t-1 |μ θ (x t ,t),∑ θ (x t ,t)); where x t-1 is the image at time step t-1, x t is the image at time step t, x0 is the original image, At time step t, given x t and x0, the reverse process x t-1 The predicted mean of is the reverse process x at time step t t-1 The prediction variance, μ θ (x t ,t) is the true mean function of the reverse process controlled by parameter θ, θ is the parameter, ∑ θ (x t ,t) is the true variance function of the reverse process controlled by parameter θ.

[0013] The present invention provides a medical beauty content AI generation system based on multi-source heterogeneous data, which applies the above-mentioned medical beauty content AI generation method based on multi-source heterogeneous data, including: an image acquisition module, used to acquire multiple plastic surgery template images; a feature extraction module, used to extract facial features corresponding to the plastic surgery template image for each plastic surgery template image; the image acquisition module is also used to acquire the user's facial image; the feature extraction module is also used to extract facial features corresponding to the user's facial image; an information acquisition module is used to acquire the user's medical beauty demand information; a template determination module is used to determine at least one candidate plastic surgery template image from the multiple plastic surgery template images according to the facial features of the user's facial image and the facial features corresponding to each of the plastic surgery template images; the template determination module is also used to determine at least one candidate plastic surgery template image according to the user's medical beauty demand information , determining at least one recommended plastic surgery template from the at least one candidate plastic surgery template image; the template determination module is also used to determine at least one target plastic surgery template image according to the user's feedback information on the at least one recommended plastic surgery template image; the project determination module is used to determine at least one candidate medical beauty project according to the user's medical beauty demand information, the facial features corresponding to the user's facial image, and the facial features corresponding to the at least one target plastic surgery template image; the project determination module is also used to obtain the user's feedback information on at least one candidate medical beauty project and determine at least one target medical beauty project; the effect prediction module is used to generate a user's medical beauty effect prediction image according to at least one target medical beauty project, the facial features corresponding to the user's facial image, and the facial features corresponding to the at least one target plastic surgery template image.

[0014] Compared with the prior art, the method and system for generating medical aesthetic content AI based on multi-source heterogeneous data provided by the present invention have at least the following beneficial effects:

[0015] 1. By extracting the user's facial features and combining the user's medical beauty needs information, the most suitable candidate cosmetic surgery template can be selected from multiple cosmetic surgery template images. This personalized customization method ensures that the medical beauty plan is more in line with the user's actual needs and expectations. The use of AI technology to automatically extract facial features, match cosmetic surgery templates, and determine the target medical beauty project based on user feedback greatly shortens the time of traditional medical beauty consultation and improves the accuracy and scientificity of the recommended plan. By generating medical beauty effect prediction images, users can intuitively see the possible change effects before implementing medical beauty projects, which helps users make more informed decisions and reduces the decision-making risks caused by information asymmetry or misunderstanding. Throughout the process, users are highly involved. From selecting cosmetic surgery templates to feedback on medical beauty projects, users can make adjustments according to their wishes. This interactivity not only enhances the user's sense of participation, but also improves the overall user experience. It provides an innovative, technology-driven solution for the medical beauty industry, which helps to promote the industry to develop in a more intelligent and personalized direction. At the same time, through big data analysis and AI prediction, medical beauty institutions can better understand user needs, optimize service processes, and improve customer satisfaction. By providing images of predicted medical aesthetic effects, this method increases the transparency of the medical aesthetic process, allowing users to have a clearer understanding of the services they are about to receive. This transparency helps to establish a trusting relationship between users and medical aesthetic institutions and promote the healthy development of the industry.

[0016] 2. By determining multiple facial feature factors and calculating feature difference values ​​and difference association values ​​based on sample facial images, this method can more accurately identify the feature factors that are most critical to the description of facial features of the cosmetic template image. This refined feature extraction method is helpful for the personalized recommendation and effect prediction of subsequent medical beauty projects. Using a large number of sample facial images for feature extraction and association analysis can enable the model to better learn the commonalities and differences of facial features of different individuals, thereby improving the model's ability to predict the facial features of unknown users. This improvement in generalization ability helps to expand the scope of application of AI generation methods. By calculating the associated key values ​​of facial feature factors and determining the target facial feature factors from multiple facial feature factors, this method can screen out the most critical features for predicting cosmetic effects and avoid the interference of redundant features on the prediction results. This feature selection method helps to improve the accuracy and reliability of prediction results.

[0017] 3. By calculating the class mean facial features and combining the total difference value of each facial feature factor to determine the class center facial features, this method can more comprehensively consider the differences and commonalities within the cosmetic template image class, thereby more accurately describing the center features of the class. This helps to improve the accuracy of subsequent candidate cosmetic template screening. According to the user's medical beauty demand information, this method can distinguish between the target facial feature factors and the non-target facial feature factors, and calculate the key feature difference values ​​and basic similarity values ​​of the candidate cosmetic templates accordingly. This personalized recommendation method can better meet the user's cosmetic needs and improve user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0019] Figure 1 It is a flowchart of a method for generating medical beauty content AI based on multi-source heterogeneous data according to some embodiments of this specification;

[0020] Figure 2 is a schematic diagram of a process of extracting facial features corresponding to a cosmetic surgery template image according to some embodiments of this specification;

[0021] Figure 3 It is a module diagram of a medical beauty content AI generation system based on multi-source heterogeneous data as shown in some embodiments of this specification. DETAILED DESCRIPTION

[0022] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0023] Figure 1 is a flowchart of a method for generating medical beauty content AI based on multi-source heterogeneous data according to some embodiments of this specification, such as Figure 1 As shown, the medical beauty content AI generation method based on multi-source heterogeneous data may include the following steps.

[0024] S111, obtaining a plurality of plastic surgery template images.

[0025] Specifically, a cosmetic surgery template is a facial feature or facial style that is widely recognized and used as a reference standard for plastic surgery in the field of cosmetic surgery. A cosmetic surgery template provides a clear goal or expected effect for the beauty seeker, allowing them to express their needs more clearly before undergoing plastic surgery.

[0026] S112: For each plastic surgery template image, extract facial features corresponding to the plastic surgery template image.

[0027] Figure 2 is a schematic diagram of a process for extracting facial features corresponding to a cosmetic surgery template image according to some embodiments of this specification, such as Figure 2 As shown, in some embodiments, S112 specifically includes:

[0028] Obtain multiple sample face images;

[0029] Determine multiple facial feature factors, such as face shape factors, facial features factors (such as the shape, size and relative position of eyes, nose, mouth, ears, etc.), facial wrinkle factors, facial muscle factors, etc.;

[0030] For each sample facial image, extracting sample facial features of the sample facial image according to a plurality of facial feature factors;

[0031] For each facial feature factor, based on the sample facial features of each sample facial image, a feature difference value corresponding to the facial feature factor is calculated;

[0032] For any two facial feature factors, based on the sample facial features of each sample facial image, calculate the difference correlation values ​​corresponding to the any two facial feature factors;

[0033] Determining a plurality of target facial feature factors from a plurality of facial feature factors according to a feature difference value corresponding to each facial feature factor and a difference correlation value corresponding to any two facial feature factors;

[0034] According to a plurality of target facial feature factors, facial features corresponding to the plastic surgery template image are extracted.

[0035] Specifically, the multiple sample facial images need to cover different face shapes, facial features, facial wrinkles and muscle shapes, etc., with moderate resolution, and the sample facial images should not have complex backgrounds and other irrelevant characters or facial organs. Facial organ images should have as many angles and expressions as possible, and the number of images that highlight the face should be slightly larger. The multiple sample facial images collected need to be cropped into a size of 512x512 pixels and unified in size.

[0036] A first feature extraction model can be established based on multiple facial feature factors, and the sample facial features of the sample facial image can be extracted by the first feature extraction model, wherein the first feature extraction model can be a convolutional neural network model. The sample facial features of the sample facial image can include the feature values ​​of the sample facial image in each facial feature factor. Taking the aspect ratio factor of the face as an example, the feature value of the aspect ratio factor of the sample image A corresponding to the facial aspect ratio factor is 1.33, and the feature value of the aspect ratio factor of the sample image B corresponding to the facial aspect ratio factor is 1.13.

[0037] The feature difference value corresponding to the facial feature factor can be calculated according to the following formula:

[0038]

[0039] Among them, σ i is the feature difference value corresponding to the i-th facial feature factor, V (i,n) is the eigenvalue of the nth sample facial image at the ith facial feature factor, and N1 is the total number of sample facial images.

[0040] The difference correlation value corresponding to two facial feature factors can be calculated according to the following formula:

[0041]

[0042] Among them, α (i,j) is the difference correlation value between the ith facial feature factor and the jth facial feature factor, V (j,n) is the eigenvalue of the jth facial feature factor of the nth sample facial image.

[0043] In some embodiments, determining a plurality of target facial feature factors from a plurality of facial feature factors according to a feature difference value corresponding to each facial feature factor and a difference association value corresponding to any two facial feature factors includes:

[0044] Determining a plurality of first key facial feature factors from the plurality of facial feature factors according to the feature difference value corresponding to each facial feature factor;

[0045] For each facial feature factor, calculating the associated key value of the facial feature factor according to the difference associated value between the facial feature factor and each key facial feature factor;

[0046] A plurality of target facial feature factors are determined from the plurality of facial feature factors according to the plurality of first key facial feature factors and the key value associated with each facial feature factor.

[0047] Specifically, a facial feature factor whose feature difference value is greater than a feature difference value threshold may be used as the first key facial feature factor.

[0048] For each facial feature factor that is not the first key facial feature factor, the associated key value of the facial feature factor may be calculated according to the difference association value between the facial feature factor and each key facial feature factor. For example, the associated key value of the facial feature factor may be calculated according to the following formula:

[0049]

[0050] Among them, K i is the associated key value of the i-th facial feature factor that is not the first key facial feature factor, α (i,h) is the difference correlation value between the i-th non-first key facial feature factor and the j-th first key facial feature factor, α (g,h) is the difference correlation value corresponding to the gth non-first key facial feature factor and the hth first key facial feature factor, H is the total number of first key facial feature factors, and G is the total number of facial feature factors that are not first key facial feature factors.

[0051] The plurality of target facial feature factors may include a plurality of first key facial feature factors and facial feature factors whose associated key values ​​are greater than an associated key value threshold and are not first key facial feature factors.

[0052] A second feature extraction model can be established based on multiple target facial feature factors, and the facial features corresponding to the plastic surgery template image can be extracted through the second feature extraction model, wherein the second feature extraction model can be a convolutional neural network model, and the facial features corresponding to the plastic surgery template image can include feature values ​​corresponding to each target facial feature factor.

[0053] S113: Acquire a facial image of the user.

[0054] S114. Obtain the user's medical beauty demand information.

[0055] Specifically, users can upload voice, text and other information through the user end, and extract medical beauty demand information from the voice, text and other information uploaded by the user end. Just as an example, the information uploaded by the user end is "I hope to improve the shape of my nose and make my nose look more three-dimensional and upright, but I don't want to have surgery, and I prefer to inject fillers. My planned budget for medical beauty is 5,000 yuan, and I hope to find the most suitable treatment plan for me within this range. I am sensitive to pain. I have mild hypertension and am taking antihypertensive drugs. I want to know if this will affect my acceptance of medical beauty treatment." The medical beauty demand information extracted from the information uploaded by the user end through the demand extraction model includes: Improvement site: nose; Expected effect: three-dimensional and upright nose; Treatment method preference: injection fillers.

[0056] S115: Extract facial features corresponding to the user's facial image.

[0057] Specifically, the facial features corresponding to the user's facial image may be extracted by the second feature extraction model.

[0058] S116. Determine at least one candidate plastic surgery template image from the multiple plastic surgery template images according to the facial features of the user's facial image and the facial features corresponding to each plastic surgery template image.

[0059] In some embodiments, S116 specifically includes:

[0060] Classifying the multiple plastic surgery template images according to the facial features corresponding to each plastic surgery template image to determine multiple plastic surgery template image classes;

[0061] For each cosmetic surgery template image class, determining a class center facial feature corresponding to the cosmetic surgery template image class according to the facial features corresponding to each cosmetic surgery template image included in the cosmetic surgery template image class;

[0062] Determining a target plastic surgery template image class from a plurality of plastic surgery template image classes according to facial features of the user's facial image and class center facial features corresponding to the plastic surgery template image class;

[0063] According to the facial features of the user's facial image and the facial features corresponding to each plastic surgery template image included in the target plastic surgery template image class, a candidate plastic surgery template image is determined from the target plastic surgery template image class.

[0064] Specifically, for any two plastic surgery template images, the Euclidean distance of the facial features corresponding to the two plastic surgery template images can be calculated. The multiple plastic surgery template images are classified according to the Euclidean distance of the facial features of any two plastic surgery template images through a clustering algorithm to determine multiple plastic surgery template image classes.

[0065] In some embodiments, determining the class center facial features corresponding to the cosmetic surgery template image class according to the facial features corresponding to each cosmetic surgery template image included in the cosmetic surgery template image class includes:

[0066] For each cosmetic surgery template image class, determining a class mean facial feature corresponding to the cosmetic surgery template image class according to the facial features corresponding to each cosmetic surgery template image included in the cosmetic surgery template image class;

[0067] For each plastic surgery template image class, the total difference value of each facial feature factor of the plastic surgery template image class is determined based on the class mean facial features corresponding to each plastic surgery template image class. The central facial feature factor corresponding to the plastic surgery template image class is determined based on the total difference value of each facial feature factor of the plastic surgery template image class. The class central facial features corresponding to the plastic surgery template image class are determined based on the central facial feature factors corresponding to the plastic surgery template image class and the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class.

[0068] Specifically, the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class may be averaged to obtain the class mean facial features corresponding to the plastic surgery template image class.

[0069] The total difference value of the facial feature factors of the cosmetic surgery template image class can be calculated according to the following formula:

[0070]

[0071] Among them, D (i,j) is the total difference value of the i-th cosmetic surgery template image class in the j-th facial feature factor, V ((i,j),average) is the eigenvalue of the jth facial feature factor in the class mean facial feature corresponding to the i-th cosmetic surgery template image class, V ((k,j),average) is the eigenvalue of the jth facial feature factor in the class mean facial feature corresponding to the kth plastic surgery template image class, and K is the total number of plastic surgery template image classes.

[0072] For each plastic surgery template image class, the mean of the eigenvalues ​​of each central facial feature factor can be calculated based on the eigenvalues ​​of the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class, and used as the class central facial feature corresponding to the plastic surgery template image class.

[0073] For each cosmetic surgery template image class, facial features of the user corresponding to the cosmetic surgery template image class can be extracted from the facial features of the user's facial image according to the central facial feature factor corresponding to the cosmetic surgery template image class, wherein the facial features of the user corresponding to the cosmetic surgery template image class may include the feature value of each central facial feature factor corresponding to the cosmetic surgery template image class. The Euclidean distance between the facial features of the user corresponding to the cosmetic surgery template image class and the class central facial features corresponding to the cosmetic surgery template image class is calculated, and the cosmetic surgery template image class whose Euclidean distance is less than the first Euclidean distance threshold is used as the target cosmetic surgery template image class.

[0074] The Euclidean distance between the facial features of the user's facial image and the facial features corresponding to each plastic surgery template image included in the target plastic surgery template image class is calculated, and the plastic surgery template image whose Euclidean distance is less than a second Euclidean distance threshold is used as a candidate plastic surgery template image.

[0075] S117. Determine at least one recommended plastic surgery template image from at least one candidate plastic surgery template according to the user's medical beauty demand information.

[0076] In some embodiments, S117 specifically includes:

[0077] According to the user's medical beauty demand information, determine the required target facial feature factors and non-required target facial feature factors from multiple target facial feature factors;

[0078] For each candidate plastic surgery template image, according to the required target facial feature factor, the facial features of the user's facial image and the facial features corresponding to each candidate plastic surgery template image, the key feature difference value corresponding to the candidate plastic surgery template image is calculated;

[0079] For each candidate plastic surgery template image, a basic similarity value corresponding to the candidate plastic surgery template image is calculated according to the non-demand target facial feature factor, the facial features of the user's facial image, and the facial features corresponding to each candidate plastic surgery template image;

[0080] At least one recommended plastic surgery template image is determined from at least one candidate plastic surgery template image according to the key feature difference value and the basic similarity value corresponding to each candidate plastic surgery template image.

[0081] Specifically, the target facial feature factors may be features of the user's face that need to be adjusted. For example, the medical beauty demand information includes: improvement part: nose; expected effect: three-dimensional and straight nose; treatment method preference: injection filler, then the target facial feature factors may include nose height factor, nose width factor, nose tilt angle factor, etc. Other target facial feature factors may be non-target facial feature factors.

[0082] The key feature difference value corresponding to the candidate cosmetic surgery template image can be calculated according to the following formula:

[0083]

[0084] Among them, D i is the key feature difference value corresponding to the i-th candidate plastic surgery template image, is the eigenvalue of the facial feature factor of the m1th target required by the i-th candidate plastic surgery template image, is the feature value of the m1th target facial feature factor of the user's facial image, and M1 is the total number of target facial feature factors.

[0085] The basic similarity value corresponding to the candidate plastic surgery template image can be calculated according to the following formula:

[0086]

[0087] Among them, S i is the basic similarity value corresponding to the i-th candidate plastic surgery template image, P1 is a preset parameter, P1 is greater than 0, is the eigenvalue of the m2th non-required target facial feature factor of the i-th candidate plastic surgery template image, is the feature value of the m2th non-required target facial feature factor of the user's facial image, and M2 is the total number of non-required target facial feature factors.

[0088] The candidate plastic surgery template image whose key feature difference value is greater than the key feature difference value threshold and whose basic similarity value is greater than the basic similarity value threshold can be used as the recommended plastic surgery template image.

[0089] S118. Determine at least one target plastic surgery template image according to the user's feedback information on at least one recommended plastic surgery template.

[0090] Specifically, the user's feedback information on at least one recommended plastic surgery template may include a target plastic surgery template image selected by the user from the at least one recommended plastic surgery template.

[0091] S119. Determine at least one candidate medical beauty project based on the user's medical beauty demand information, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template image.

[0092] In some embodiments, S119 specifically includes:

[0093] Through the project prediction model, at least one candidate medical beauty project is determined according to the user's medical beauty demand information, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template image, wherein the project prediction model can be a convolutional neural network model.

[0094] S120. Obtain user feedback information on at least one candidate medical beauty project, and determine at least one target medical beauty project.

[0095] Specifically, the user's feedback information on at least one candidate medical beauty project may include at least one target medical beauty project selected by the user from the at least one candidate medical beauty project.

[0096] S121. Generate a predicted image of the user's medical beauty effect based on at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template image.

[0097] In some embodiments, step S121 specifically includes:

[0098] The image generation model generates a predicted image of the user's medical beauty effect through reverse image reasoning based on at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template.

[0099] Specifically, the trained fine-tuning model LoRA model is used to fine-tune the stable-diffusion-v1-5 large model, and an image generation model is established. According to the facial features corresponding to at least one target medical beauty project and the user's facial image and the facial features corresponding to at least one target cosmetic surgery template, the user's medical beauty effect prediction image is generated through image reasoning through the reverse process.

[0100] The loRA fine-tuning model is trained through the forward process. In the forward process, the image x t Only the x of the previous moment t-1 The process can be regarded as a Markov process, satisfying the following formula

[0101]

[0102] Among them, q(x 1:T |x0) is the sequence x from time 1 to time T given the initial state x0 1:T The probability distribution of represents x at each time step t from time 1 to time T t Given the previous time step x t-1 The product of the conditional probability distributions under t |x t-1 ) represents x t At a given x t-1 The conditional distribution under is a Gaussian distribution with mean The variance is β t I.

[0103] Among them, the reverse process p of the image generation model θ (x t-1 |x t ) through the neural network q(x t-1 |x t ,x0) fitting results:

[0104]

[0105] p θ (x t-1 |x t )=N(x t-1 |μ θ (x t ,t),∑ θ (x t ,t))

[0106] Among them, x t-1 is the image at time step t-1, x t is the image at time step t, x0 is the original image, At time step t, given x t and x0, the reverse process x t-1 The predicted mean of is the reverse process x at time step t t-1 The prediction variance, μ θ (x t ,t) is the true mean function of the reverse process controlled by parameter θ, θ is the parameter, ∑ θ (x t ,t) is the true variance function of the reverse process controlled by parameter θ.

[0107] Figure 3 is a module diagram of a medical beauty content AI generation system based on multi-source heterogeneous data according to some embodiments of this specification, such as Figure 3 As shown, the medical beauty content AI generation system based on multi-source heterogeneous data can include an image acquisition module, a feature extraction module, an information acquisition module, a template determination module, a project determination module and an effect prediction module.

[0108] An image acquisition module, used to acquire multiple cosmetic surgery template images;

[0109] A feature extraction module, for extracting facial features corresponding to each plastic surgery template image;

[0110] The image acquisition module is also used to acquire the user's facial image;

[0111] The feature extraction module is also used to extract facial features corresponding to the user's facial image;

[0112] Information acquisition module, used to obtain users' medical beauty demand information;

[0113] A template determination module, used to determine at least one candidate plastic surgery template image from a plurality of plastic surgery template images according to facial features of the user's facial image and facial features corresponding to each plastic surgery template image;

[0114] The template determination module is further used to determine at least one recommended cosmetic surgery template from at least one candidate cosmetic surgery template image according to the user's medical beauty demand information;

[0115] The template determination module is further used to determine at least one target plastic surgery template image according to the user's feedback information on at least one recommended plastic surgery template image;

[0116] A project determination module, used to determine at least one candidate medical beauty project based on the medical beauty demand information of the user, the facial features corresponding to the user's facial image, and the facial features corresponding to at least one target cosmetic surgery template image;

[0117] The project determination module is further used to obtain user feedback information on at least one candidate medical beauty project and determine at least one target medical beauty project;

[0118] The effect prediction module is used to generate a user's medical beauty effect prediction image based on at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template image.

[0119] The medical beauty content AI generation system based on multi-source heterogeneous data can be used to execute the medical beauty content AI generation method based on multi-source heterogeneous data, and the relevant description will not be repeated here.

[0120] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A method for generating medical aesthetic content AI based on multi-source heterogeneous data, characterized in that: include: Obtain multiple plastic surgery template images; For each plastic surgery template image, extracting facial features corresponding to the plastic surgery template image; Get the user's facial image; Obtain users’ medical beauty demand information; Extracting facial features corresponding to the user's facial image; Determining at least one candidate plastic surgery template image from the plurality of plastic surgery template images according to facial features of the user's facial image and facial features corresponding to each of the plastic surgery template images; Determining at least one recommended plastic surgery template image from the at least one candidate plastic surgery template according to the user's medical beauty demand information; Determining at least one target plastic surgery template image according to user feedback information on the at least one recommended plastic surgery template; Determining at least one candidate medical beauty project according to the medical beauty demand information of the user, the facial features corresponding to the user's facial image, and the facial features corresponding to the at least one target cosmetic surgery template image; Obtain user feedback on at least one candidate medical beauty project, and determine at least one target medical beauty project; A predicted image of the user's medical beauty effect is generated based on at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template image.

2. The method for generating medical aesthetic content AI based on multi-source heterogeneous data according to claim 1, characterized in that: Extracting facial features corresponding to the cosmetic surgery template image includes: Obtain multiple sample face images; determining a plurality of facial feature factors; For each sample facial image, extracting sample facial features of the sample facial image according to the multiple facial feature factors; For each facial feature factor, based on the sample facial features of each of the sample facial images, calculating a feature difference value corresponding to the facial feature factor; For any two facial feature factors, based on the sample facial features of each of the sample facial images, calculate the difference correlation values ​​corresponding to the any two facial feature factors; Determining a plurality of target facial feature factors from the plurality of facial feature factors according to the feature difference value corresponding to each facial feature factor and the difference association value corresponding to any two facial feature factors; The facial features corresponding to the plastic surgery template image are extracted according to the multiple target facial feature factors.

3. The method for generating medical aesthetic content AI based on multi-source heterogeneous data according to claim 2, characterized in that: Determining a plurality of target facial feature factors from the plurality of facial feature factors according to the feature difference value corresponding to each facial feature factor and the difference association value corresponding to any two facial feature factors, comprising: Determining a plurality of first key facial feature factors from the plurality of facial feature factors according to the feature difference value corresponding to each facial feature factor; For each facial feature factor, calculating the associated key value of the facial feature factor according to the difference associated value between the facial feature factor and each key facial feature factor; A plurality of target facial feature factors are determined from the plurality of facial feature factors according to the plurality of first key facial feature factors and the key value associated with each facial feature factor.

4. The method for generating medical aesthetic content AI based on multi-source heterogeneous data according to claim 2, characterized in that: Determining at least one candidate plastic surgery template image from the plurality of plastic surgery template images according to the facial features of the user's facial image and the facial features corresponding to each of the plastic surgery template images, comprising: Classifying the plurality of plastic surgery template images according to the facial features corresponding to each of the plastic surgery template images to determine a plurality of plastic surgery template image classes; For each cosmetic surgery template image class, determining a class center facial feature corresponding to the cosmetic surgery template image class according to the facial features corresponding to each cosmetic surgery template image included in the cosmetic surgery template image class; Determining a target plastic surgery template image class from the plurality of plastic surgery template image classes according to facial features of the user's facial image and class center facial features corresponding to the plastic surgery template image class; According to the facial features of the user's facial image and the facial features corresponding to each plastic surgery template image included in the target plastic surgery template image class, a candidate plastic surgery template image is determined from the target plastic surgery template image class.

5. The method for generating medical aesthetic content AI based on multi-source heterogeneous data according to claim 4, characterized in that: According to the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class, the class center facial features corresponding to the plastic surgery template image class are determined, including: For each cosmetic surgery template image class, determining a class mean facial feature corresponding to the cosmetic surgery template image class according to the facial features corresponding to each cosmetic surgery template image included in the cosmetic surgery template image class; For each plastic surgery template image class, the total difference value of each facial feature factor of the plastic surgery template image class is determined based on the class mean facial features corresponding to each plastic surgery template image class. The central facial feature factor corresponding to the plastic surgery template image class is determined based on the total difference value of each facial feature factor of the plastic surgery template image class. The class central facial features corresponding to the plastic surgery template image class are determined based on the central facial feature factors corresponding to the plastic surgery template image class and the facial features corresponding to each plastic surgery template image included in the plastic surgery template image class.

6. The method for generating medical aesthetic content AI based on multi-source heterogeneous data according to claim 2, characterized in that: Determining at least one recommended plastic surgery template image from the at least one candidate plastic surgery template image according to the user's medical beauty demand information includes: According to the user's medical beauty demand information, determining a required target facial feature factor and a non-required target facial feature factor from the multiple target facial feature factors; For each candidate plastic surgery template image, according to the required target facial feature factor, the facial features of the user's facial image and the facial features corresponding to each candidate plastic surgery template image, the key feature difference value corresponding to the candidate plastic surgery template image is calculated; For each candidate plastic surgery template image, a basic similarity value corresponding to the candidate plastic surgery template image is calculated according to the non-demand target facial feature factor, the facial features of the user's facial image, and the facial features corresponding to each candidate plastic surgery template image; At least one recommended plastic surgery template image is determined from the at least one candidate plastic surgery template image according to the key feature difference value and the basic similarity value corresponding to each candidate plastic surgery template image.

7. The method for generating medical aesthetic content AI based on multi-source heterogeneous data according to claim 6, characterized in that: Determining at least one candidate medical beauty project according to the medical beauty demand information of the user, the facial features corresponding to the user's facial image, and the facial features corresponding to the at least one target cosmetic surgery template image, including: At least one candidate medical beauty project is determined through a project prediction model according to the user's medical beauty demand information, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template image.

8. The method for generating medical aesthetic content AI based on multi-source heterogeneous data according to any one of claims 1 to 7, characterized in that: Generate a user's medical beauty effect prediction image according to at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to the at least one target cosmetic surgery template, including: The image generation model generates a predicted image of the user's medical beauty effect through reverse image reasoning based on at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template.

9. The method for generating medical aesthetic content AI based on multi-source heterogeneous data according to claim 8, characterized in that: The reverse process of the image generation model θ (x t-1 |x t ) through the neural network q(x t-1 |x t ,x0) fitting results: p θ (x t-1 |x t )=N(x t-1 |μ θ (x t ,t),Σ θ (x t ,t)) Among them, x t-1 is the image at time step t-1, x t is the image at time step t, x0 is the original image, At time step t, given x t and x0, the reverse process x t-1 The predicted mean of is the reverse process x at time step t t-1 The prediction variance, μ θ (x t ,t) is the true mean function of the reverse process controlled by parameter θ, θ is the parameter, ∑ θ (x t ,t) is the true variance function of the reverse process controlled by parameter θ.

10. The AI ​​generation system for medical aesthetic content based on multi-source heterogeneous data is characterized by: The method for generating medical aesthetic content AI based on multi-source heterogeneous data as described in any one of claims 1 to 9 comprises: An image acquisition module, used to acquire multiple cosmetic surgery template images; A feature extraction module, for extracting facial features corresponding to each plastic surgery template image; The image acquisition module is also used to acquire the user's facial image; The feature extraction module is also used to extract facial features corresponding to the user's facial image; Information acquisition module, used to obtain users' medical beauty demand information; A template determination module, configured to determine at least one candidate plastic surgery template image from the plurality of plastic surgery template images according to facial features of the user's facial image and facial features corresponding to each of the plastic surgery template images; The template determination module is further used to determine at least one recommended cosmetic surgery template from the at least one candidate cosmetic surgery template image according to the user's medical beauty demand information; The template determination module is further used to determine at least one target plastic surgery template image according to user feedback information on the at least one recommended plastic surgery template image; A project determination module, configured to determine at least one candidate medical beauty project according to the medical beauty demand information of the user, the facial features corresponding to the user's facial image, and the facial features corresponding to the at least one target cosmetic surgery template image; The project determination module is further used to obtain user feedback information on at least one candidate medical beauty project and determine at least one target medical beauty project; The effect prediction module is used to generate a user's medical beauty effect prediction image based on at least one target medical beauty project, facial features corresponding to the user's facial image, and facial features corresponding to at least one target cosmetic surgery template image.

Images