Model generation method and device, equipment and medium

Abstract

The embodiment of the invention provides a model generation method and device, equipment and a medium. The method comprises the steps of determining a face picture used for generating a display model, generating a basic model based on the face picture, then extracting an organ model in the basic model, generating a model sequence of the organ model, and then adjusting the model sequence of the face model and the organ model based on a set face adjustment index to obtain the display model. And finally, generating a display model on the basis of the model sequences of the adjusted face model and the adjusted organ model. The method is used for solving the problem that the operation is not ideal due to the fact that the display intuition of the postoperative effect is insufficient in the prior art.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a model generation method, device, equipment and medium. Background Art

[0002] In the beauty and plastic surgery industry, due to the high precision and high risk of facial plastic surgery itself, doctors need to conduct an objective and quantitative evaluation of the patient's face in advance and demonstrate the postoperative effect, so as to determine the surgical plan based on the patient's preference for different postoperative effects, and avoid unsatisfactory or even failed surgery due to the large gap between the treatment plan and the patient's ideal. To this end, it is necessary to provide visual reference information so that patients can have an intuitive understanding of the postoperative effect.

[0003] In related technologies, the display of postoperative effects is mainly provided through two-dimensional images. For example, an ideal profile image is transformed to obtain a series of facial profile images, and then participants are asked to rank their attractiveness, or use a visual analog scale to score the transformed images to determine the surgical plan based on the scores. However, two-dimensional images still lack sufficient intuitiveness, which causes patients to often have a wrong perception of the postoperative effect, resulting in unsatisfactory surgery. Summary of the invention

[0004] The embodiments of the present application provide a model generation method, device, equipment and medium to solve the problem in the related art that the postoperative effect display is not intuitive enough, resulting in unsatisfactory surgery.

[0005] In a first aspect, an embodiment of the present application provides a model generation method, comprising:

[0006] Determine a face picture for generating a display model, and generate a basic model based on the face picture, wherein the basic model includes a face model and an organ model, there are at least two organ models, and the face model, the organ model and the display model are all three-dimensional models;

[0007] Extracting organ models from the basic model and generating a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model;

[0008] Based on the set facial adjustment index, the model sequence of the facial model and the organ model is adjusted to obtain the model sequence of the adjusted facial model and the adjusted organ model;

[0009] A presentation model is generated based on the model sequence of the adjusted face model and the adjusted organ model.

[0010] In one possible implementation, a facial image for generating a display model is determined, and a basic model is generated based on the facial image, including: modeling the facial images in a preset facial image library to obtain a preliminary facial model; determining a model with a frontal view in the preliminary facial model as a candidate model; determining a deviation value between a candidate facial structure index value in each candidate model and a predetermined facial structure reference value; and determining the candidate model with the smallest deviation value as the basic model.

[0011] In one possible implementation, the candidate model with the smallest deviation value is determined as the basic model, including: if there are at least two candidate models with the smallest deviation values, then the candidate model with the highest facial symmetry among the candidate models with the smallest deviation values ​​is determined as the basic model, wherein the facial symmetry is determined based on the difference in candidate facial structure indicators on both sides of the face.

[0012] In a possible implementation, the organ model includes an eye model, a mouth model, and a nose model.

[0013] In a possible implementation, based on the set facial adjustment index, the model sequence of the facial model and the organ model is adjusted to obtain the adjusted facial model and the model sequence of the adjusted organ model, including: adjusting the facial model based on the difference between the facial structure index value of the facial model and the predetermined facial structure reference value to obtain the adjusted facial model; determining the adjustment value of each organ model in the corresponding model sequence based on the adjustment parameter range corresponding to the organ model; and adjusting the organ models respectively based on the adjustment values ​​to obtain the model sequence of the adjusted organ models.

[0014] In a possible implementation, the facial structure reference value includes an average value and a standard deviation corresponding to each facial structure indicator; adjusting the facial model based on a difference between the facial structure indicator value of the facial model and a predetermined facial structure reference value to obtain an adjusted facial model includes: rotating the facial model to a point where the eye-ear plane is perpendicular to the coronal axis; scaling the facial model based on a head size reference value in the facial structure reference value; determining a difference between the facial structure indicator value of the facial model and an average value of a corresponding category in the facial structure reference value; determining a ratio of the difference to the standard deviation of the corresponding category in the facial structure reference value as an offset; and adjusting the corresponding facial structure indicator value in the facial model based on the offset.

[0015] In a possible implementation, after adjusting the organ models respectively based on the adjustment values ​​to obtain the model sequence of the adjusted organ models, the method further includes: replacing the double eyelid map in the eye model with a single eyelid map.

[0016] In a possible implementation, a display model is generated based on a model sequence of an adjusted facial model and an adjusted organ model, including: selecting an adjusted organ model from each model sequence for placement in the facial model; and moving the adjusted organ model to the adjusted facial model to generate a display model.

[0017] In a possible implementation, the adjusted organ model is moved to the adjusted facial model to generate a display model, including: determining the spatial range of the adjusted organ model; translating the adjusted organ model based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model; performing gradient processing on the boundary area within a set distance adjacent to the vertex coordinates; adjusting the model vertex correspondences of the adjusted organ model to the average value of the face model surface normals to obtain the moved organ model; and generating the display model based on the moved organ model and the adjusted facial model.

[0018] In a possible implementation, a display model is generated based on the moved organ model and the adjusted facial model, including: based on the sagittal plane, removing the right half of the face of the moved organ model and the adjusted facial model; performing symmetry processing on the left half of the face of the moved organ model and the adjusted facial model; importing a hair model, and combining the hair model with the facial model to obtain a display model.

[0019] In a possible implementation, after generating the display model based on the model sequence of the adjusted facial model and the adjusted organ model, it also includes at least one of the following: acquiring a facial photo of a target angle based on the display model; generating a corresponding display animation based on the display model and set facial movements.

[0020] In a second aspect, an embodiment of the present application provides a model generation device, including:

[0021] An acquisition module, used to determine a face picture for generating a display model, and generate a basic model based on the face picture, wherein the basic model includes a face model and an organ model, the organ model includes at least two types, and the face model, the organ model and the display model are all three-dimensional models;

[0022] An extraction module, used to extract the organ models in the basic model and generate a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model;

[0023] An adjustment module, used for adjusting the model sequence of the facial model and the organ model based on a set facial adjustment index to obtain an adjusted facial model and an adjusted model sequence of the organ model;

[0024] A generation module is used to generate the display model based on the model sequence of the adjusted facial model and the adjusted organ model.

[0025] In one possible implementation, the acquisition module is specifically used to model the face images in a preset face image library to obtain a preliminary face model; determine the model with a frontal face orientation in the preliminary face model as a candidate model; determine the deviation value between the candidate face structure index value in each candidate model and the predetermined face structure reference value; and determine the candidate model with the smallest deviation value as the basic model.

[0026] In one possible implementation, the acquisition module is specifically used to, if there are at least two candidate models with the smallest deviation values, determine the candidate model with the highest facial symmetry among the candidate models with the smallest deviation values ​​as the basic model, wherein the facial symmetry is determined based on the difference in candidate facial structure indicators on both sides of the face.

[0027] In a possible implementation, the acquisition module specifically includes that the organ model includes an eye model, a mouth model and a nose model.

[0028] In one possible implementation, the adjustment module is specifically used to adjust the facial model based on the difference between the facial structure index value of the facial model and a predetermined facial structure reference value to obtain an adjusted facial model; determine the adjustment value of each organ model in the corresponding model sequence based on the adjustment parameter range corresponding to the organ model; and adjust the organ models separately based on the adjustment value to obtain a model sequence of adjusted organ models.

[0029] In a possible implementation, the adjustment module is specifically used to, if the facial structure reference value includes the average value and standard deviation corresponding to each facial structure indicator, rotate the facial model to the eye-ear plane and the coronal axis; scale the facial model based on the head size reference value in the facial structure reference value; determine the difference between the facial structure indicator value of the facial model and the average value of the corresponding category in the facial structure reference value; determine the ratio of the difference to the standard deviation of the corresponding category in the facial structure reference value as an offset; and adjust the corresponding facial structure indicator value in the facial model based on the offset.

[0030] In a possible implementation, the adjustment module is further used to adjust the organ models respectively based on the adjustment value, and after obtaining the model sequence of the adjusted organ models, replace the double eyelid map in the eye model with the single eyelid map.

[0031] In a possible implementation, the generation module is specifically used to select, from each model sequence, respectively an adjusted organ model for placement into the facial model; and move the adjusted organ model into the adjusted facial model to generate a display model.

[0032] In one possible implementation, the generation module is specifically used to determine the spatial range of the adjusted organ model; translate the adjusted organ model based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model; perform gradient processing on the boundary area within a set distance adjacent to the vertex coordinates; adjust the model vertex correspondences of the adjusted organ model to the average value of the surface normals of the facial model to obtain the moved organ model; and generate a display model based on the moved organ model and the adjusted facial model.

[0033] In a possible implementation, the generation module is specifically used to, based on the sagittal plane, remove the right half of the face of the moved organ model and the adjusted facial model; perform symmetry processing on the left half of the face of the moved organ model and the adjusted facial model; import the hair model, and combine the hair model with the facial model to obtain a display model.

[0034] In a possible implementation, the generation module is also used to, after generating the display model based on the model sequence of the adjusted facial model and the adjusted organ model, include at least one of the following: obtaining a facial photo of a target angle based on the display model; generating a corresponding display animation based on the display model and set facial movements.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0036] The memory stores computer-executable instructions;

[0037] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0040] The model generation method, device, equipment and medium provided in the embodiment of the present application, by determining the face picture used to generate the display model, and generating the basic model based on the face picture, then extracting the organ model in the basic model, and generating the model sequence of the organ model, and then adjusting the model sequence of the face model and the organ model based on the set face adjustment index, to obtain the model sequence of the adjusted face model and the adjusted organ model, and finally generating the display model based on the adjusted face model and the adjusted organ model. Thus, by generating a three-dimensional display model, the intuitiveness of the postoperative effect is significantly improved, and the problem of insufficient display of two-dimensional pictures is solved. By generating a basic model from a face picture and extracting and adjusting the model sequence of the organ model, the doctor can make precise adjustments to the face and organs in three-dimensional space, thereby allowing the doctor to generate a postoperative effect display model that is more in line with the patient's expectations according to the set face adjustment index, reduce the patient's misunderstanding of the postoperative effect, reduce the risk of unsatisfactory surgery, and improve the success rate of surgery and patient satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] Figure 1 An application scenario diagram of the model generation method provided in the embodiment of the present disclosure;

[0043] Figure 2 A flowchart of a model generation method provided by one embodiment of the present disclosure;

[0044] Figure 3a A flowchart of a model generation method provided by yet another embodiment of the present disclosure;

[0045] Figure 3b for Figure 3a A schematic diagram of the structure of the nose model provided by the illustrated embodiment;

[0046] Figure 3c for Figure 3a A flow chart of a method for adjusting a facial model provided by the illustrated embodiment;

[0047] Figure 3d for Figure 3a A schematic diagram of states corresponding to different adjustment values ​​of the lip protrusion parameter index in the mouth model provided by the illustrated embodiment;

[0048] Figure 4 A flowchart of a model generation method provided by yet another embodiment of the present disclosure;

[0049] Figure 5 A schematic diagram of the structure of a model generation device provided by another embodiment of the present disclosure;

[0050] Figure 6 A schematic diagram of the structure of an electronic device provided by one embodiment of the present disclosure.

[0051] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0053] In the beauty and plastic surgery industry, the high precision and high risk of facial plastic surgery require doctors to conduct objective quantitative evaluation of the patient's face before surgery and demonstrate the postoperative effect. This process is crucial to developing a surgical plan that meets the patient's expectations. However, existing technologies mainly rely on two-dimensional images to show the postoperative effect, which has significant limitations. Two-dimensional images lack sufficient depth and realism, making it difficult for patients to accurately understand the postoperative effect, which may lead to surgical results that do not meet the patient's expectations or even lead to surgical failure. The principle is that the flat characteristics of two-dimensional images cannot fully display the three-dimensional structure and subtle changes of the face, limiting effective communication between doctors and patients.

[0054] However, in the actual beauty industry, the traditional method of displaying postoperative effects has been used for many years and can provide basic visual references to a certain extent. In many cases, the results of surgery do not meet the patient's expectations, which may also be caused by other reasons, such as the patient's high expectations for plastic surgery results or the unsatisfactory postoperative recovery process. Therefore, the relevant field lacks in-depth research on the defects of two-dimensional images in depth and realism, and their potential impact on surgical results.

[0055] At the same time, it is also difficult to display the images in a way other than two-dimensional images. For example, the generation of three-dimensional display models requires advanced image processing and modeling technology to ensure the accuracy and authenticity of the models. In addition, the generation and adjustment of three-dimensional models requires the processing of large amounts of data and complex algorithms in order to achieve real-time or near-real-time effect display without increasing the burden on patients and doctors.

[0056] Figure 1A schematic diagram of an application scenario of the model generation method provided in this application, such as Figure 1 As shown, the specific application scenario of the present application is: in cosmetic plastic surgery, the doctor will use the computer 100 to generate a facial display model 110 corresponding to different specific surgical methods and surgical adjustment amounts, so that the patient can understand the corresponding postoperative effect according to the facial display model 110, which is convenient for the patient to better determine the surgical method and adjustment amount, and improve the patient's postoperative satisfaction.

[0057] It should be noted that Figure 1 The scenes shown include only one or a specific number of computer and face display models for illustration, but the present disclosure is not limited to this, that is, the number of computer and face display models can be arbitrary.

[0058] The model generation method provided in this application generates a basic model including face and multiple organ models from a face picture, then extracts and generates a model sequence of organ models, and then adjusts them through the set face adjustment index; finally, the final display model is generated based on the adjusted face and organ models. This method enhances the three-dimensional sense and realism of the postoperative effect, enables patients to understand the expected effect more accurately, and reduces the risk of unsatisfactory surgery.

[0059] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0060] Figure 2 Schematic diagram of the model generation method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:

[0061] S201, determining a face picture for generating a display model, and generating a basic model based on the face picture.

[0062] Among them, the basic model includes a facial model and an organ model. There are at least two types of organ models. The facial model, the organ model and the display model are all three-dimensional models.

[0063] Specifically, this embodiment is used to summarize the main steps of generating a presentation model.

[0064] The execution subject in the embodiment of the present disclosure is a computer system for generating a display model, and may also be a program specifically used to generate and adjust a display model. For convenience, it is collectively referred to as a system hereinafter.

[0065] The display model can intuitively reflect the state of the human face at all angles, so that it can be combined with the patient's desired surgical site to show in detail the corresponding facial state of different surgical methods and surgical contents (such as different adjustment amounts of zygomatic bone height), allowing patients to understand the surgical effects more intuitively.

[0066] In the process of obtaining the display model, the first task is to determine the face image used to generate the display model and generate the basic model based on this image. The basic model here is used to represent the three-dimensional structure model extracted from the original face image, including the face model (i.e. the model of the main facial bones and the corresponding skin) and at least two organ models (such as the nose and eyes). These models are all three-dimensional models to provide a more realistic and three-dimensional visual effect.

[0067] The specific basic model generation method can start from high-resolution face pictures, and use image processing technology, such as computer vision and deep learning algorithms, to identify and extract facial features. Then, use 3D modeling software to convert these features into a 3D model to form a basic model.

[0068] It is crucial to obtain high-quality facial images because they directly affect the accuracy of subsequent models. Since it takes a long time to generate a display model, the display model is usually not based on the patient's face, but a more general face model. In order to better fit the patient's characteristics, the general face model can be modified according to the average value of the patient's facial feature parameters (such as the statistical average value of facial feature parameters in the region) to obtain a display model. To this end, it is necessary to select facial images for generating the basic model based on the above-mentioned average value of facial feature parameters.

[0069] Next, computer vision technology is used to extract facial feature points from the selected face images. These feature points usually include key parts such as eyes, nose, mouth, chin, etc., through which the basic geometric structure of the face can be constructed. After extracting the feature points, these points are converted into a 3D mesh model using 3D modeling software.

[0070] In some embodiments, in order to improve the realism of the model, texture mapping is usually performed to map the color information of the original image onto the three-dimensional model to make it look more realistic.

[0071] In some embodiments, a deep learning algorithm may be combined to automatically identify facial features and perform preliminary three-dimensional modeling to improve the accuracy and detail of the model.

[0072] S202: extracting organ models from the basic model and generating a model sequence of the organ models.

[0073] The model sequence includes at least two models corresponding to each organ model.

[0074] Specifically, in order to obtain the final display model, adjustments need to be made based on the basic model. The facial organs are the most commonly adjusted parts (the eyes are mainly adjusted on the eyelids). Therefore, in order to facilitate the display of the corresponding model changes, the organ model needs to be extracted. At the same time, in order to facilitate the display of the surgical effects corresponding to different adjustment methods of the same part, a model sequence corresponding to each organ model needs to be established, so that the different parameter changes of the same organ in the model sequence can reflect the corresponding surgical effects, providing patients with a better user experience.

[0075] Therefore, it is necessary to extract the organ model in the basic model and generate a model sequence of the organ model. The model sequence here refers to at least two models of different states or forms corresponding to each organ model, which are used to show different adjustment effects (when extracting, just copy the same organ model multiple times, and then make targeted adjustments in subsequent steps).

[0076] The specific implementation process involves separating each organ model from the basic model, which can be achieved through a feature segmentation algorithm (or through corresponding model modification software); then, these separated organ models are copied multiple times to generate a corresponding model sequence, so that possible changes or adjustments can be displayed by adjusting different models in the model sequence.

[0077] The specific feature segmentation algorithm can be based on geometric features, color information or depth data to ensure the accuracy of segmentation and the integrity of the model.

[0078] S203. Based on the set face adjustment index, the model sequence of the face model and the organ model is adjusted to obtain the model sequence of the adjusted face model and the adjusted organ model.

[0079] Specifically, facial adjustment indicators refer to parameters or standards used to guide model adjustment, such as proportion, symmetry, and specific aesthetic standards. There are different facial adjustment indicators corresponding to different positions in the facial model and organ model.

[0080] These facial adjustment indicators are pre-configured by doctors or system developers, such as nose bridge height, nose wing width, eye corner opening and eyelid curvature, etc. The specific values ​​can be determined by the doctor's professional judgment. Corresponding to the organ model, the same indicator will have multiple different optional values. Based on each value, a trachea model in the model sequence is adjusted to obtain different adjusted organ models, that is, a model sequence of adjusted organ models.

[0081] The specific use of facial adjustment indicators to adjust the facial model and organ model may involve complex algorithms, such as shape optimization, nonlinear deformation and constraint optimization, to ensure that the adjusted model meets the set indicator values.

[0082] In some embodiments, an automated algorithm may be applied to automatically adjust the corresponding model according to preset facial adjustment indicators and values, thereby reducing human errors and time consumption, improving efficiency, and ensuring the consistency and reliability of the adjustment results.

[0083] S204: Generate a display model based on the model sequence of the adjusted facial model and the adjusted organ model.

[0084] Specifically, based on the model sequence of the adjusted facial model and organ model, a final display model can be generated. The display model is used to intuitively display the postoperative effect.

[0085] By selecting an organ model from the model sequence of each organ model and integrating it with the adjusted facial model, a complete three-dimensional display model can be obtained. By selecting different models from the model sequence and integrating them with the adjusted facial model, different three-dimensional display models can be obtained to reflect the display effects of different facial adjustments.

[0086] By generating multiple display models simultaneously based on the model sequence, the corresponding display model can be directly called for display according to the patient's surgical selection, so that the patient can quickly and intuitively understand the surgical effect, improve the display efficiency, and enhance the user experience of patients and doctors.

[0087] The specific process of integrating the adjusted models needs to ensure seamless connection between the various models. Therefore, it is necessary to combine the model merging technology to form a complete three-dimensional structure by combining different organ models with the facial model.

[0088] In some embodiments, during the model merging process, attention needs to be paid to the proportion and symmetry of the models to ensure that the visual effect of the final displayed model is natural and realistic.

[0089] In some embodiments, after the models are merged, texture mapping and light and shadow processing can be performed to improve the realism of the models. Texture mapping can apply the color information of the original image to the three-dimensional model to make it look more realistic, and light and shadow processing can enhance the three-dimensional sense and detail performance of the model by simulating real lighting conditions. The combination of these technologies makes the displayed model not only accurate in shape, but also highly realistic in visual effect.

[0090] In some embodiments, virtual reality (VR) or augmented reality (AR) technology can also be used to provide an immersive display experience, allowing patients to view the postoperative effects in a virtual environment, enhancing understanding and decision-making accuracy. This not only improves patient participation, but also enhances the efficiency and effectiveness of communication between doctors and patients.

[0091] The model generation method provided in the embodiment of the present application determines a face picture for generating a display model, generates a basic model based on the face picture, extracts the organ model in the basic model, generates a model sequence of the organ model, and then adjusts the model sequence of the face model and the organ model based on the set face adjustment index to obtain the model sequence of the adjusted face model and the adjusted organ model, and finally generates the display model based on the adjusted face model and the adjusted organ model. Thus, by generating a three-dimensional display model, the intuitiveness of the postoperative effect is significantly improved, and the problem of insufficient two-dimensional picture display is solved. By generating a basic model from a face picture and extracting and adjusting the model sequence of the organ model, the doctor can make precise adjustments to the face and organs in three-dimensional space, thereby allowing the doctor to generate a postoperative effect display model that is more in line with the patient's expectations based on the set face adjustment index, reduce the patient's misunderstanding of the postoperative effect, reduce the risk of unsatisfactory surgery, and improve the success rate of surgery and patient satisfaction.

[0092] Figure 3a Schematic diagram of the process of generating the model provided for this application Figure 2 ,like Figure 3a As shown, in this embodiment Figure 2 Based on the embodiment, the specific implementation process of the model generation method is described in detail, and the method includes:

[0093] S301, modeling the face images in a preset face image library to obtain a preliminary face model.

[0094] Specifically, this embodiment is used to explain in detail the actions related to basic model determination, facial model and organ model adjustment in the model generation process.

[0095] When determining the facial images used to generate the basic model, since cosmetic surgery needs to reflect the patient's aesthetic preferences, facial images can be determined based on public or open source databases of photos of beautiful people in China to reflect the general aesthetic tendencies of patients and ensure that the display models screened and generated therefrom can better conform to the general subjective aesthetic evaluation of patients.

[0096] For example, in order to obtain a three-dimensional model that is closest to the average Chinese face, a public face photo database consisting entirely of facial photos of Chinese celebrities is used to obtain face images for generating the basic model.

[0097] After obtaining the face pictures, these two-dimensional pictures can be converted into three-dimensional models through computer vision technology and three-dimensional reconstruction algorithms. For example, multi-view stereo vision technology (MVS) or structured light scanning technology can be used to capture the depth information of the face in the face picture to generate a three-dimensional model.

[0098] In some embodiments, existing facial modeling software may be combined to complete the generation of a face image into a prepared face model, such as modeling through FaceGen Modeller software.

[0099] The database based on photos of beautiful people in China can reflect the aesthetic tendencies of the Chinese public in the current era, and the models selected from it can better conform to the public's subjective aesthetic evaluation.

[0100] S302: Determine the model with the frontal view in the prepared face model as the candidate model.

[0101] Specifically, in the aforementioned steps, multiple preliminary face models can be generated in batches based on the selected multiple face images, and then the basic model for generating the display model can be determined according to the facial indicator parameters of the preliminary face models.

[0102] In this process, the model with the frontal view in the prepared face model needs to be determined as the candidate model. The candidate model here is used to represent the model that meets the specific orientation requirements selected from the prepared face model, and is used for subsequent indicator comparison and adjustment.

[0103] To this end, it is necessary to identify the facial orientation of the prepared face model to filter out the model with the frontal facial orientation, which can be achieved by analyzing the facial feature points of the model. By calculating the relative position and angle of the facial feature points, it can be determined whether the model is facing the front.

[0104] The frontal view model usually provides more symmetrical and complete facial information, which helps to improve the accuracy of structural indicator comparison in subsequent steps.

[0105] S303: Determine the deviation value between the candidate facial structure index value in each candidate model and the predetermined facial structure reference value.

[0106] Specifically, after determining the candidate model, the deviations of the values ​​of various facial structure indicators of the candidate face can be further compared with the predetermined facial structure reference values. The predetermined facial structure reference values ​​can be directly determined by referring to the corresponding standard documents, such as "GB / T2428-2024 Adult Head and Facial Dimensions" (statistical analysis results of a large amount of facial data in the region or similar statistical files can also be used) to ensure that the values ​​of various facial structure feature indicators fit the faces of different patients.

[0107] As shown in Table 1 below, by measuring the measured values ​​of the candidate facial structure indicators and the predetermined facial structure reference values, the deviation values ​​of the facial structure indicators of the corresponding types can be obtained.

[0108] Table 1 Examples of corresponding measurement values ​​and corresponding offset values ​​of facial structure indicators in candidate models

[0109]

[0110] Based on the offset values ​​of the respective candidate models, the candidate models may be further determined.

[0111] S304: Determine the candidate model with the smallest deviation value as the basic model.

[0112] Specifically, only one basic model is needed, so the candidate model with the smallest deviation value (sum or average) obtained in the above steps can be directly determined as the basic model.

[0113] The implementation process includes first screening out the model with the smallest deviation value, and then the optional implementation method includes automatically evaluating the symmetry of the model using a symmetry detection algorithm, or through expert review combined with automated tools to ensure that the selection of the basic model is accurate and meets aesthetic standards.

[0114] In some embodiments, if there are at least two candidate models with the smallest deviation values, the candidate model with the highest facial symmetry among the candidate models with the smallest deviation values ​​is determined as the basic model, wherein the facial symmetry is determined based on the difference in candidate facial structure indicators on both sides of the face.

[0115] Specifically, on the basis of comparing the deviation values, the facial symmetry can be further compared. The facial symmetry can be determined by calculating the difference in structural indicators on both sides of the face. Through this double screening mechanism, it is ensured that the selected basic model not only meets the structural reference value, but also has good symmetry.

[0116] In some embodiments, the basic model can also be determined by first using an image recognition algorithm dedicated to face recognition to respectively identify the facial orientation and facial structure index values ​​of each face image in a preset face image library, and then selecting a face image with a frontal view, a face structure index value with the smallest deviation from a predetermined face structure reference value, and the highest facial symmetry, and then modeling based on the face image to obtain the basic model. The face image selection process in this implementation method can also be implemented by a pre-trained neural network model.

[0117] Those skilled in the art may select this implementation method, or the implementation method in steps S301 to S304 according to actual needs, and no limitation is made here.

[0118] S305: extracting the organ model from the basic model and generating a model sequence of the organ model.

[0119] The model sequence includes at least two models corresponding to each organ model.

[0120] Specifically, after determining the basic model, the facial model and the organ model can be extracted therefrom, and the model sequence of the organ model can be obtained by changing the facial structure indicators corresponding to the organ model (the organ models in the model sequence can be preliminarily adjusted in this step and further adjusted in subsequent steps, or the organ models can be directly copied in this step and adjusted once in subsequent steps).

[0121] In some embodiments, the organ model includes an eye model, a mouth model, and a nose model.

[0122] Specifically, the organ models separated from the basic model are mainly eye model, mouth model and nose model, which are the three most commonly involved wonders in cosmetic surgery. These organ models can be extracted from the basic model through feature segmentation algorithm.

[0123] In some embodiments, Figure 3b As shown, it is a structural schematic diagram of the nose model, and the part not including the grid lines in the figure is the range corresponding to the nose model.

[0124] The separated organ models are not separated from the facial model, but remain on the facial model. However, during the numerical adjustment process, only the organ models will be adjusted specifically. For example, the eyes and mouth remain unchanged, while the nose is adjusted to generate multiple models containing different nose models (the model at this time also contains various organ models and facial models), that is, the organ sequences corresponding to these different nose models.

[0125] S306: Adjust the facial model based on the difference between the facial structure index value of the facial model and a predetermined facial structure reference value to obtain an adjusted facial model.

[0126] Specifically, the obtained basic model may still be different from the predetermined human face structure reference value, so the facial model may be adjusted directly according to the difference.

[0127] With reference to the national standard GB / T 2428-2024 Adult head and face dimensions, the model parameters were sculpted and modified to meet the average value of Chinese women.

[0128] Furthermore, if Figure 3c As shown, it is a flow chart of the facial model adjustment method. The facial structure reference value includes the average value and standard deviation corresponding to each facial structure index. Therefore, the specific facial model adjustment method may include the following steps:

[0129] S3061. Rotate the facial model until the eye-ear plane is perpendicular to the coronal axis.

[0130] Specifically, the oculoauricular plane and the coronal axis are both special terms for describing positions in the field of beauty.

[0131] Since the generated basic model and facial model are usually not in a completely frontal state (usually in a slightly tilted state, because completely frontal face images are generally only used in ID photos, the images in the preset face image library are usually somewhat tilted relative to the completely frontal state.

[0132] Therefore, before making specific adjustments to the facial grindability, it is necessary to standardize the posture of the facial model in order to make subsequent precise adjustments.

[0133] By identifying key points on the facial model, such as the corners of the eyes, earlobes and chin tip, the position of the eye-ear plane is calculated, and then through mathematical transformation, the facial model is rotated to a position where the plane is perpendicular to the coronal axis (an automated posture adjustment algorithm can also be preset to automatically complete the rotation process according to preset standards).

[0134] S3062: Scale the facial model based on the head size reference value in the facial structure reference value.

[0135] Specifically, since the facial model size obtained at this time is different from the facial size (head size reference value) in the facial structure reference value, it is necessary to adjust the size of the facial model by scaling the facial model for further facial model adjustment.

[0136] At this point, the current size of the facial model needs to be calculated and compared with the head size reference value. Based on the comparison results, the scaling ratio is determined (such as reducing by 0.84 times or other multiples) to adjust the facial model to a size that meets the reference standard. The scaling process needs to maintain the proportions and symmetry of the model to ensure that the adjusted model is visually natural and meets aesthetic standards.

[0137] S3063. Determine the difference between the face structure index value of the facial model and the average value of the corresponding category in the face structure reference value.

[0138] Specifically, after the scaling is completed, the difference between the face structure index value of the face model and the average value of the corresponding category in the face structure reference value can be calculated. The face structure index value refers to the geometric feature parameters of the face model, such as the height of the bridge of the nose, the distance between the eyes and the width of the chin.

[0139] S3064. Determine the ratio of the difference value to the standard deviation of the corresponding category in the facial structure reference value as the offset.

[0140] Specifically, the ratio of the difference value to the standard deviation of the corresponding category in the face structure reference value is determined as the offset to ensure that the face structure index value of the facial model is within 1 standard deviation from the face structure reference value.

[0141] In some embodiments, steps S3063 to S3064 may be performed continuously without being divided into multiple steps.

[0142] S3065. Based on the offset, adjust the corresponding facial structure index value in the facial model.

[0143] Specifically, based on the offset, various structural indicators of the facial model are adjusted accordingly. The specific adjustment may involve operations such as shape deformation, size scaling, and position movement to ensure that the adjusted model is visually natural and meets the expected effect.

[0144] In some embodiments, an automated adjustment tool may also be used to automatically complete the adjustment process in steps S3061 to S3065.

[0145] Through these steps, it is ensured that the adjusted facial model not only conforms to the reference value of the human facial structure, but also has good visual effects and patient acceptance.

[0146] S307 . Determine the adjustment value of each organ model in the corresponding model sequence based on the adjustment parameter range corresponding to the organ model.

[0147] Specifically, each organ model corresponds to different facial structure parameters and has a certain adjustment range, that is, the acceptable adjustment range of each organ model, which is used to ensure that the adjusted model is visually natural and meets aesthetic standards. For example, the ideal range of nose bridge height is 10 to 14 mm, then nose models with different nose bridge heights can be generated at intervals of 0.5 mm.

[0148] The specific range of adjustment parameters can be determined based on clinical experience or statistical data (such as the numerical range in the aforementioned facial structure reference value).

[0149] Based on these parameter ranges, the adjustment values ​​of each organ model in the model sequence can be calculated to guide the subsequent model adjustment process and ensure that the adjusted model meets the expected effect.

[0150] S308. Based on the adjustment value, the organ models are adjusted respectively to obtain a model sequence of the adjusted organ models.

[0151] Specifically, the model sequence includes a plurality of adjusted organ models with different adjustment values ​​corresponding to the same organ model, so as to demonstrate the corresponding postoperative effects after different adjustments.

[0152] Specific adjustments can be operations such as shape deformation, size scaling, and position movement to ensure that the adjusted model is visually natural and meets the expected effects.

[0153] like Figure 3dAs shown in FIG. 1 , it is a schematic diagram of the corresponding states of different adjustment values ​​of the lip protrusion parameter index in the mouth model. By adjusting the facial structure parameter index in the organ model, organ models in various states can be obtained.

[0154] S309, replacing the double eyelid map in the eye model with the single eyelid map.

[0155] Specifically, single eyelids are used because from a statistical point of view, the proportion of patients with single eyelids is higher than that with double eyelids. According to actual conditions, all single eyelid maps can be replaced with double eyelid maps.

[0156] In specific implementation, for the eye model, the texture map needs to be replaced to change the appearance characteristics of the model, and the double eyelid map is replaced with a single eyelid map. The system can identify the double eyelid map in the eye model and then select a pre-drawn single eyelid map for replacement. The system can manually replace the map using texture editing software, or use automation tools to automatically complete the map replacement process according to preset standards to improve efficiency and consistency.

[0157] Thus, the adjustment of the facial model and the organ model can be completed.

[0158] S310: Select the adjusted organ models from each model sequence for placement into the facial model.

[0159] Specifically, when a model is actually provided to the patient to demonstrate the effect of the surgery, usually only one model (such as directly demonstrating the state after the surgery through the model) or two models (such as demonstrating the state before the surgery through one model and the state after the surgery through another model) will be demonstrated. At this time, the system can directly select the facial model and combine it with the organ model that best fits the reference value of the facial structure index to obtain a display model. It can also determine the corresponding display model based on the most common type of surgery and the adjustment amount of the facial structure index corresponding to the surgery.

[0160] In some embodiments, due to the adjustment of the organ model in the aforementioned steps, usually only one model is kept fixed, but when the model is actually fused, it is usually the fusion of multiple different organ models, and can also include the fusion of the organ model in the basic model and the adjusted facial model, and the selection of the most common model from the organ sequence (such as selecting the most common modified state for each parameter, and selecting an organ model for each parameter type) and combining it with the facial model. Therefore, when the model is fused, the adjustment of the model position and connection parts is also involved.

[0161] Ten to twenty model combinations are obtained through the aforementioned method (more models can also be combined, and then screened according to the frequency of occurrence of various combinations to extract the most common combinations, such as canthoplasty, double eyelid surgery, and nose bridge width reduction, which are common combinations). These combinations can generate display models for subsequent display according to user needs.

[0162] In some embodiments, the adjusted model of each organ model in the model sequence can also be evaluated according to a preset selection standard (such as a deviation value based on the organ model and the reference value of the facial structure index) to select the model that best meets the aesthetic standard. In addition to the above standards, evaluation indicators such as proportion and visual effect can also be combined to achieve automated selection.

[0163] In some embodiments, unselected organ models are also stored in the system, and based on the type of surgery and amount of adjustment required by the patient, a corresponding organ model is selected (labels corresponding to facial structure indicators can be added to the organ model) to replace the organ model in the current display model, thereby quickly generating a new display model to showcase the post-operative effect.

[0164] S311, moving the adjusted organ model to the adjusted facial model to generate a display model.

[0165] Specifically, the selected adjusted organ model is integrated with the adjusted facial model to ensure seamless connection between the models, so as to obtain the final display model.

[0166] Therefore, by generating display models, it can not only play a role in aesthetic evaluation research, making it convenient for researchers to compare different display models, score different model combinations, and understand people's preferences for aesthetics; it can also play a clinical role in cosmetic surgery, quickly displaying the corresponding model according to the type of surgery the patient expects. Even if there is no similarity, the corresponding display model can be quickly obtained by selecting from the organ sequence, and display animations and pictures can be generated so that users can select the results. This ensures that the display model fits the patient's characteristics, can conveniently display the effect of cosmetic surgery, and enhances the communication efficiency and effectiveness between doctors and patients.

[0167] The model generation method provided in the embodiment of the present application ensures the accuracy of the model by generating a basic model from a high-resolution face image and extracting and adjusting the organ model. Then, by setting the facial adjustment index and spatial range, the model is translated and gradient processed so that the organ model and the facial model are seamlessly integrated. Finally, through the combination of symmetry processing and hair model, the generated display model is more visually natural and complete. This solution not only improves the intuitiveness of the postoperative effect, but also enhances the communication efficiency and decision-making accuracy between doctors and patients, reduces the risk of unsatisfactory surgery, and improves patient satisfaction. Through automated and intelligent technical means, the solution effectively reduces human errors and improves the efficiency and consistency of model generation.

[0168] Figure 4 The flow chart 3 of the model generation provided for this application is as follows: Figure 4 As shown, in this embodiment Figure 2 and Figure 3a Based on the embodiment, the part of generating the display model after completing the model adjustment in the model generation method is further described in detail. The method includes:

[0169] S401, determining a face picture for generating a display model, and generating a basic model based on the face picture.

[0170] Among them, the basic model includes a facial model and an organ model. There are at least two types of organ models. The facial model, the organ model and the display model are all three-dimensional models.

[0171] Specifically, this embodiment is used to illustrate the process of generating a display model after the facial model and the organ model are adjusted.

[0172] S402: extracting organ models from the basic model and generating a model sequence of the organ models.

[0173] The model sequence includes at least two models corresponding to each organ model.

[0174] S403: Based on the set face adjustment index, the model sequence of the face model and the organ model is adjusted to obtain the model sequence of the adjusted face model and the adjusted organ model.

[0175] Specifically, steps S401 to S403 are the same as the corresponding steps in the aforementioned embodiment and will not be described in detail here.

[0176] S404: Determine the spatial range of the adjusted organ model.

[0177] Specifically, after the adjustment of the facial model and the organ model is completed, it is necessary to determine the combination of the facial model and the organ model to obtain a display model. At this time, it is first necessary to determine the spatial range of the adjusted organ model. The spatial range refers to the position and range of the adjusted organ model in three-dimensional space, which is used to guide subsequent translation and integration operations.

[0178] The system needs to calculate the bounding box or bounding volume of each adjusted organ model to determine its position and size in 3D space. This can be achieved by analyzing the vertex coordinates of the model to ensure accurate translation and integration of the model in subsequent steps.

[0179] S405: Based on the model vertices in the spatial range and the vertex coordinates of the corresponding vertex positions in the facial model, the adjusted organ model is translated.

[0180] Specifically, the system needs to move the organ model in three-dimensional space so that it is correctly aligned with the facial model. The specific implementation of this process includes determining the vertex coordinates of the adjusted organ model, then calculating the difference between these vertices and the corresponding vertices in the facial model, and then calculating the translation vector based on these differences, and translating the organ model accordingly in three-dimensional space.

[0181] This ensures that the organ model is accurately aligned with the facial model in terms of position, laying the foundation for subsequent integration and display.

[0182] S406: Perform gradient processing on the boundary area within a set distance adjacent to the vertex coordinates.

[0183] Specifically, during fusion, since the organ models in different models need to be fused together, gradient processing needs to be performed based on the contact parts.

[0184] In a specific implementation, after the translation is completed, the vertex coordinates within a certain range (such as an area with a width of 8 mm) around the boundary where the organ model and the facial model meet need to be gradient processed to make the model look smooth.

[0185] In some embodiments, this can be achieved by adjusting the normal direction and color value of the vertices in the boundary area to ensure that the final display model is visually seamless and natural. The specific adjustment formula may be:

[0186] ;

[0187] Among them, x is the moving distance along the normal direction starting from the boundary area of ​​the organ model. The larger the moving distance, the lighter the display color.

[0188] In some embodiments, an automated gradient processing tool may also be used to automatically complete the gradient process according to a preset standard.

[0189] S407, adjusting the model vertex correspondence of the adjusted organ model to the average value of the surface normal of the facial model to obtain the moved organ model.

[0190] Specifically, the normals corresponding to the model vertices of the adjusted organ model are adjusted to the average value of the surface normals of the facial model to obtain the moved organ model, so as to ensure that the model rendering result is normal.

[0191] The system needs to calculate the normals of the facial model at each vertex, and then apply the average of these normals to the corresponding organ model vertices. By adjusting the normal direction, it can ensure that the organ model is consistent with the facial model in terms of lighting and shadow effects, thereby improving the visual consistency and realism of the displayed model.

[0192] S408: Generate a display model based on the moved organ model and the adjusted facial model.

[0193] Specifically, after the previous processing, the system will integrate the moved organ model with the adjusted facial model to ensure seamless connection between the models to form a complete three-dimensional structure.

[0194] Furthermore, in the process of generating the display model, the specific integration actions also include the following steps:

[0195] Step A1: based on the sagittal plane, remove the right half of the face of the moved organ model and the adjusted facial model.

[0196] Specifically, the sagittal plane is a vertical plane passing through the midline of the human body, dividing the body into left and right parts. By removing the right half of the face, it provides a basis for subsequent symmetry processing and ensures that the left half of the face of the model can be accurately mirrored.

[0197] Step A2: Symmetric processing is performed on the moved organ model and the left half of the face of the adjusted facial model.

[0198] Specifically, through the mirroring operation, one side of the model is copied to the other side to ensure that the model has good visual symmetry, improve the aesthetic effect of the display model and the patient's satisfaction.

[0199] Step A3: import the hair model, and combine the hair model with the facial model to obtain a display model.

[0200] Specifically, based on the aforementioned operations, a hair model is imported and combined with a facial model to obtain a display model, so as to improve the integrity and realism of the display model.

[0201] As a result, the display model is not only structurally complete, but also more realistic and natural in visual effects, with a high sense of reality and aesthetic appeal.

[0202] In some embodiments, after the presentation model is generated, a facial photo at a target angle may be acquired based on the presentation model, or a corresponding presentation animation may be generated based on the presentation model and set facial movements.

[0203] Specifically, in order to quickly and conveniently display the demonstration model to the patient, after the demonstration model is generated, screenshots can be directly taken to obtain frontal and side profile photos, so as to quickly provide the patient with a two-dimensional display of the surgical effect.

[0204] In addition, the system can also automatically record the playback animation of the display model. By following the set facial movements (such as turning the head, closing the eyes, and opening the mouth, which can be achieved through the 3D model debugging software), adjusting the recording frame rate and the number of recording frames (such as recording an avi file of 15fps and 135 frames and converting it to gif format), and then generating the display animation, a better playback animation can be obtained.

[0205] When showing the system to patients, the combination of two-dimensional photos and three-dimensional animated images can enable users to obtain the best display effect of the system and enhance patients' understanding of the surgical results.

[0206] The model generation method provided in the embodiment of the present application generates a basic model for a face image, and extracts and adjusts the organ model to ensure the basic accuracy of the model. Then, by setting the spatial range and performing vertex translation and gradient processing, the seamless integration of the organ model and the facial model is achieved, enhancing the visual naturalness of the model. Further symmetry processing and the combination of the hair model make the final display model more complete and realistic in structure. This solution not only improves the visualization accuracy of the postoperative effect, but also promotes effective communication between doctors and patients, helping patients better understand and anticipate the results of the operation. At the same time, the automated processing steps reduce the errors caused by human intervention, improve the efficiency and consistency of the overall operation, and ultimately improve patient satisfaction and the success rate of the operation.

[0207] Figure 5 A schematic diagram of the structure of the model generation device provided in this application, such as Figure 5 As shown, the model generation device 500 provided in this embodiment includes:

[0208] An acquisition module 510 is used to determine a face picture for generating a display model, and generate a basic model based on the face picture, wherein the basic model includes a face model and an organ model, the organ model includes at least two types, and the face model, the organ model and the display model are all three-dimensional models;

[0209] An extraction module 520, configured to extract the organ models in the basic model and generate a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model;

[0210] An adjustment module 530, configured to adjust the model sequence of the facial model and the organ model based on a set facial adjustment index to obtain an adjusted facial model and an adjusted model sequence of the organ model;

[0211] The generation module 540 is used to generate the display model based on the model sequence of the adjusted facial model and the adjusted organ model.

[0212] In one possible implementation, the acquisition module 510 is specifically used to model the face images in a preset face image library to obtain a preliminary face model; determine the model with a frontal face orientation in the preliminary face model as a candidate model; determine the deviation value between the candidate face structure index value in each candidate model and the predetermined face structure reference value; and determine the candidate model with the smallest deviation value as the basic model.

[0213] In one possible implementation, the acquisition module 510 is specifically used to, if there are at least two candidate models with the smallest deviation values, determine the candidate model with the highest facial symmetry among the candidate models with the smallest deviation values ​​as the basic model, wherein the facial symmetry is determined based on the difference in candidate facial structure indicators on both sides of the face.

[0214] In a possible implementation, the acquisition module 510 specifically includes that the organ model includes an eye model, a mouth model, and a nose model.

[0215] In a possible implementation, the adjustment module 530 is specifically used to adjust the facial model based on the difference between the facial structure index value of the facial model and a predetermined facial structure reference value to obtain an adjusted facial model; determine the adjustment value of each organ model in the corresponding model sequence based on the adjustment parameter range corresponding to the organ model; and adjust the organ models separately based on the adjustment value to obtain a model sequence of adjusted organ models.

[0216] In a possible implementation, the adjustment module 530 is specifically used to, if the facial structure reference value includes the mean value and standard deviation corresponding to each facial structure indicator, rotate the facial model to the eye-ear plane and the coronal axis; scale the facial model based on the head size reference value in the facial structure reference value; determine the difference between the facial structure indicator value of the facial model and the mean value of the corresponding category in the facial structure reference value; determine the ratio of the difference to the standard deviation of the corresponding category in the facial structure reference value as an offset; and adjust the corresponding facial structure indicator value in the facial model based on the offset.

[0217] In a possible implementation, the adjustment module 530 is further configured to adjust the organ models respectively based on the adjustment value, and after obtaining the model sequence of the adjusted organ models, replace the double eyelid map in the eye model with the single eyelid map.

[0218] In a possible implementation, the generation module 540 is specifically used to select, from each model sequence, respectively an adjusted organ model for placement into the facial model; and move the adjusted organ model into the adjusted facial model to generate a display model.

[0219] In one possible implementation, the generation module is specifically used to determine the spatial range of the adjusted organ model; translate the adjusted organ model based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model; perform gradient processing on the boundary area within a set distance adjacent to the vertex coordinates; adjust the model vertex correspondences of the adjusted organ model to the average value of the surface normals of the facial model to obtain the moved organ model; and generate a display model based on the moved organ model and the adjusted facial model.

[0220] In a possible implementation, the generation module 540 is specifically used to, based on the sagittal plane, remove the right half of the face of the moved organ model and the adjusted facial model; perform symmetry processing on the left half of the face of the moved organ model and the adjusted facial model; import the hair model, and combine the hair model with the facial model to obtain a display model.

[0221] In a possible implementation, the generation module 540 is also used to, after generating the display model based on the model sequence of the adjusted facial model and the adjusted organ model, include at least one of the following: obtaining a facial photo of a target angle based on the display model; generating a corresponding display animation based on the display model and set facial movements.

[0222] The model generation device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0223] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device 60 also includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus 604.

[0224] In a specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0225] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0226] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0227] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0228] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0229] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0230] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0231] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0232] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0233] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0234] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0235] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0236] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0237] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0238] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A model generation method, characterized in that: include: Determine a face picture for generating a display model, and generate a basic model based on the face picture, wherein the basic model includes a face model and an organ model, the organ models include at least two types, and the face model, the organ model and the display model are all three-dimensional models; Extracting the organ models in the basic model and generating a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model; Based on the set face adjustment index, the model sequence of the face model and the organ model is adjusted to obtain an adjusted face model and an adjusted model sequence of the organ model; The presentation model is generated based on the model sequence of the adjusted facial model and the adjusted organ model.

2. The method according to claim 1, characterized in that The step of determining a face picture for generating a display model and generating a basic model based on the face picture includes: Modeling the face images in the preset face image library to obtain a preliminary face model; Determine the model with the face orientation of the front view in the prepared face model as a candidate model; Determine the deviation value between the candidate face structure index value in each candidate model and the predetermined face structure reference value; The candidate model with the smallest deviation value is determined as the basic model.

3. The method according to claim 2, characterized in that Determining the candidate model with the smallest deviation value as the basic model includes: If there are at least two candidate models with the smallest deviation values, the candidate model with the highest facial symmetry among the candidate models with the smallest deviation values ​​is determined as the basic model, wherein the facial symmetry is determined based on the difference in candidate facial structure indicators on both sides of the face.

4. The method according to claim 1, characterized in that The organ model includes an eye model, a mouth model and a nose model.

5. The method according to claim 4, characterized in that The step of adjusting the model sequence of the facial model and the organ model based on the set facial adjustment index to obtain the adjusted facial model and the adjusted model sequence of the organ model comprises: Based on the difference between the face structure index value of the face model and a predetermined face structure reference value, adjusting the face model to obtain an adjusted face model; Determining an adjustment value for each organ model in a corresponding model sequence based on a range of adjustment parameters corresponding to the organ model; Based on the adjustment values, the organ models are adjusted respectively to obtain a model sequence of the adjusted organ models.

6. The method according to claim 5, characterized in that The facial structure reference value includes the mean value and standard deviation corresponding to each facial structure index; The step of adjusting the facial model based on the difference between the facial structure index value of the facial model and a predetermined facial structure reference value to obtain an adjusted facial model comprises: Rotating the facial model until the eye-ear plane is perpendicular to the coronal axis; Scaling the facial model based on a head size reference value in the facial structure reference value; Determine the difference between the face structure index value of the face model and the average value of the corresponding category in the face structure reference value; Determine the ratio of the difference to the standard deviation of the corresponding category in the face structure reference value as an offset; Based on the offset, the corresponding face structure index value in the face model is adjusted.

7. The method according to claim 5, characterized in that After adjusting the organ models based on the adjustment values ​​to obtain the model sequence of the adjusted organ models, the method further includes: The double eyelid map in the eye model is replaced with a single eyelid map.

8. The method according to claim 1, characterized in that The step of generating the display model based on the model sequence of the adjusted facial model and the adjusted organ model comprises: Selecting adjusted organ models from each model sequence for placement into the face model; The adjusted organ model is moved to the adjusted facial model to generate the display model.

9. The method according to claim 8, characterized in that The step of moving the adjusted organ model to the adjusted facial model to generate the display model comprises: determining the spatial extent of the adjusted organ model; Based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model, translating the adjusted organ model; Performing gradient processing on the boundary area within a set distance adjacent to the vertex coordinates; The model vertex correspondence of the adjusted organ model is adjusted to the average value of the surface normal of the facial model to obtain the moved organ model; The presentation model is generated based on the moved organ model and the adjusted facial model.

10. The method according to claim 8, characterized in that The step of generating the display model based on the moved organ model and the adjusted facial model comprises: Based on the sagittal plane, removing the moved organ model and the right half face portion of the adjusted facial model; Symmetric processing is performed on the moved organ model and the left half face part of the adjusted facial model; A hair model is imported, and the hair model is combined with the facial model to obtain the display model.

11. The method according to any one of claims 1 to 10, characterized in that After generating the display model based on the model sequence of the adjusted facial model and the adjusted organ model, the method further includes at least one of the following: Acquire a facial photo at a target angle based on the display model; Based on the presentation model and the set facial movements, a corresponding presentation animation is generated.

12. A model generation device, characterized in that: include: An acquisition module, used to determine a face picture for generating a display model, and generate a basic model based on the face picture, wherein the basic model includes a face model and an organ model, the organ model includes at least two types, and the face model, the organ model and the display model are all three-dimensional models; An extraction module, used to extract the organ models in the basic model and generate a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model; An adjustment module, used for adjusting the model sequence of the facial model and the organ model based on a set facial adjustment index to obtain an adjusted facial model and an adjusted model sequence of the organ model; A generation module is used to generate the display model based on the model sequence of the adjusted facial model and the adjusted organ model.

13. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 11 when executed by a processor.

15. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 11 when being executed by a processor.

Images