Model generation method, apparatus, device, and medium

Abstract

Embodiments of the present application provide a model generation method, device, equipment and medium. The method comprises: determining a face picture used to generate a display model, generating a basic model based on the face picture, extracting organ models in the basic model, generating a model sequence of the organ models, adjusting the model sequence of the face model and the organ models based on a set face adjustment index to obtain an adjusted model sequence of the adjusted face model and the adjusted organ models, and finally generating the display model based on the adjusted model sequence of the adjusted face model and the adjusted organ models. The method is used to solve the problem of insufficient postoperative effect display intuitiveness in related technologies, which leads to unsatisfactory surgery.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a model generation method, apparatus, device and medium. Background Technology

[0002] In the cosmetic and plastic surgery industry, due to the inherently high precision and risk of facial plastic surgery, doctors need to conduct an objective and quantitative evaluation of the patient's face beforehand and demonstrate the post-operative effects. This is to determine the surgical plan based on the patient's preferences for different post-operative results, avoiding unsatisfactory or even failed surgeries due to a significant discrepancy between the treatment plan and the patient's ideal. Therefore, it is necessary to provide visual reference information so that patients can have a direct understanding of the post-operative results.

[0003] In related technologies, the postoperative results are mainly presented through two-dimensional images. For example, a series of facial profile images are obtained by transforming an ideal profile image, and participants are then asked to rank their attractiveness, or a visual analog scale is used to rate the transformed images in order to determine the surgical plan. However, two-dimensional images still lack sufficient intuitiveness, which often leads to patients having misunderstandings about the postoperative results, resulting in unsatisfactory surgery. Summary of the Invention

[0004] This application provides a model generation method, apparatus, device, and medium to address the problem of insufficient intuitiveness in displaying postoperative effects in related technologies, leading to unsatisfactory surgical outcomes.

[0005] In a first aspect, embodiments of this application provide a model generation method, including:

[0006] The face image used to generate the display model is determined, and a base model is generated based on the face image. The base model includes a facial model and an organ model. There are at least two types of organ models. The facial model, organ model, and display model are all three-dimensional models.

[0007] Extract organ models from the base model and generate a model sequence of organ models, wherein the model sequence includes at least two models corresponding to each organ model;

[0008] Based on the set facial adjustment indicators, the model sequences of the facial model and organ model are adjusted to obtain the adjusted facial model and the adjusted organ model sequence.

[0009] A display model is generated based on a sequence of adjusted facial and organ models.

[0010] In one possible implementation, determining the face images used to generate the display model and generating a base model based on the face images includes: modeling face images in a preset face image library to obtain preliminary face models; identifying models with a frontal view among the preliminary face models as candidate models; determining the deviation values ​​of the candidate face structure index values ​​in each candidate model from the pre-determined face structure reference values; and identifying the candidate model with the smallest deviation value as the base model.

[0011] In one possible implementation, the candidate model with the smallest deviation value is determined as the base model, including: if there are at least two candidate models with the smallest deviation value, the candidate model with the highest facial symmetry among the candidate models with the smallest deviation value is determined as the base model, wherein the facial symmetry is determined based on the difference in facial structure index values ​​between the two sides of the candidate face.

[0012] In one possible implementation, the organ models include an eye model, a mouth model, and a nose model.

[0013] In one possible implementation, the model sequences of the facial model and organ model are adjusted based on a set facial adjustment index to obtain an adjusted facial model and an adjusted organ model sequence. This includes: adjusting the facial model based on the difference between the facial structure index value of the facial model and a pre-determined facial structure reference value to obtain an adjusted facial model; determining the adjustment value of each organ model in the corresponding model sequence based on the adjustment parameter range corresponding to the organ model; and adjusting the organ models separately based on the adjustment values ​​to obtain an adjusted organ model sequence.

[0014] In one possible implementation, the facial structure reference values ​​include the average and standard deviation of each facial structure index. Based on the difference between the facial structure index values ​​of the facial model and the predetermined facial structure reference values, the facial model is adjusted to obtain an adjusted facial model, including: rotating the facial model so that the eye-ear plane is perpendicular to the coronal axis; scaling the facial model based on the head size reference values ​​in the facial structure reference values; determining the difference between the facial structure index values ​​of the facial model and the average values ​​of the corresponding categories in the facial structure reference values; determining the ratio of the difference to the standard deviation of the corresponding categories in the facial structure reference values ​​as an offset; and adjusting the corresponding facial structure index values ​​in the facial model based on the offset.

[0015] In one possible implementation, after adjusting the organ models based on the adjustment values ​​to obtain the adjusted organ model sequence, the method further includes: replacing the double eyelid texture in the eye model with a single eyelid texture.

[0016] In one possible implementation, a display model is generated based on a model sequence of adjusted facial models and adjusted organ models, including: selecting an adjusted organ model from each model sequence for placement in the facial model; moving the adjusted organ model into the adjusted facial model to generate the display model.

[0017] In one possible implementation, the adjusted organ model is moved into the adjusted facial model to generate a display model, including: determining the spatial range of the adjusted organ model; translating the adjusted organ model based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model; applying a gradient to the boundary region within a set distance of the vertex coordinates; adjusting the corresponding vertex coordinates of the adjusted organ model to the average value of the face normals of the facial model to obtain the moved organ model; and generating a display model based on the moved organ model and the adjusted facial model.

[0018] In one possible implementation, a display model is generated based on the moved organ model and the adjusted facial model, including: removing the right half of the face from the moved organ model and the adjusted facial model based on the sagittal plane; performing symmetry processing on the left half of the face from the moved organ model and the adjusted facial model; importing a hair model and combining the hair model with the facial model to obtain the display model.

[0019] In one possible implementation, after generating a display model based on a model sequence of adjusted facial and organ models, the method further includes at least one of the following: obtaining a facial photograph at a target angle based on the display model; and generating a corresponding display animation based on the display model and set facial movements.

[0020] Secondly, embodiments of this application provide a model generation apparatus, comprising:

[0021] The acquisition module is used to determine the face image used to generate the display model, and generate a basic model based on the face image. The basic model includes a facial model and an organ model. There are at least two types of organ models. The facial model, organ model and the display model are all three-dimensional models.

[0022] An extraction module is used to extract organ models from the base model and generate a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model;

[0023] The adjustment module is used to adjust the model sequences of the facial model and the organ model based on the set facial adjustment index, so as to obtain the adjusted facial model and the adjusted organ model sequence.

[0024] The generation module is used to generate the display model based on the model sequence of the adjusted facial model and the adjusted organ model.

[0025] In one possible implementation, the acquisition module is specifically used to: model face images in a preset face image library to obtain a preliminary face model; identify models with a frontal view among the preliminary face models as candidate models; determine the deviation values ​​of the candidate face structure index values ​​in each candidate model from the pre-determined face structure reference values; and identify the candidate model with the smallest deviation value as the base model.

[0026] In one possible implementation, the acquisition module is specifically used to determine the candidate model with the highest facial symmetry among the candidate models with the smallest deviation value if there are at least two candidate models with the smallest deviation value. The facial symmetry is determined based on the difference in facial structure index values ​​between the two sides of the candidate face.

[0027] In one possible implementation, the acquisition module specifically includes organ models, including an eye model, a mouth model, and a nose model.

[0028] In one possible implementation, the adjustment module is specifically used to: adjust the facial model based on the difference between the facial structure index value of the facial model and the predetermined facial structure reference value to obtain the adjusted facial model; determine the adjustment value of each organ model in the corresponding model sequence based on the adjustment parameter range corresponding to the organ model; and adjust the organ models respectively based on the adjustment value to obtain the model sequence of the adjusted organ models.

[0029] In one possible implementation, the adjustment module is specifically used to: if the facial structure reference values ​​include the average and standard deviation corresponding to each facial structure index, rotate the facial model so that the eye-ear plane is perpendicular to the coronal axis; scale the facial model based on the head size reference value in the facial structure reference values; determine the difference between the facial structure index value of the facial model and the average value of the corresponding type in the facial structure reference values; determine the ratio of the difference to the standard deviation of the corresponding type in the facial structure reference values ​​as the offset; and adjust the corresponding facial structure index value in the facial model based on the offset.

[0030] In one possible implementation, the adjustment module is further configured to, based on the adjustment value, adjust the organ models respectively, and after obtaining the model sequence of the adjusted organ models, replace the double eyelid texture in the eye model with the single eyelid texture.

[0031] In one possible implementation, the generation module is specifically used to select, from each model sequence, an adjusted organ model for placement in the facial model; move the adjusted organ model into the adjusted facial model to generate a display model.

[0032] In one possible implementation, the generation module is specifically used to: determine the spatial range of the adjusted organ model; translate the adjusted organ model based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model; perform gradient processing on the boundary regions within a set distance of the vertex coordinates; adjust the model vertices of the adjusted organ model to the average value of the surface normals of the facial model to obtain the moved organ model; and generate a display model based on the moved organ model and the adjusted facial model.

[0033] In one possible implementation, the generation module is specifically used to: remove the right half of the face from the moved organ model and the adjusted facial model based on the sagittal plane; perform symmetry processing on the left half of the face from the moved organ model and the adjusted facial model; import the hair model and combine the hair model with the facial model to obtain the display model.

[0034] In one possible implementation, the generation module is further configured to, after generating a display model based on the model sequence of the adjusted facial model and the adjusted organ model, include at least one of the following: obtaining a facial photograph at a target angle based on the display model; and generating a corresponding display animation based on the display model and set facial movements.

[0035] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0036] The memory stores computer-executed instructions;

[0037] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0038] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0039] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0040] The model generation method, apparatus, device, and medium provided in this application embodiment determine a facial image for generating a display model, generate a basic model based on the facial image, extract organ models from the basic model, generate a model sequence of organ models, and then adjust the model sequences of the facial model and organ models based on set facial adjustment indicators to obtain adjusted facial model and adjusted organ model sequences. Finally, a display model is generated based on the adjusted facial model and adjusted organ model sequences. Thus, by generating a three-dimensional display model, the intuitiveness of postoperative effects is significantly improved, solving the problem of insufficient two-dimensional image display. By generating a basic model from a facial image and extracting and adjusting the model sequences of organ models, doctors can precisely adjust the face and organs in three-dimensional space. This allows doctors to generate a postoperative effect display model that better meets the patient's expectations based on set facial adjustment indicators, reducing patient misunderstandings about postoperative effects, lowering the risk of unsatisfactory surgery, and improving surgical success rate and patient satisfaction. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0042] Figure 1 An application scenario diagram of the model generation method provided in this disclosure embodiment;

[0043] Figure 2 A flowchart illustrating a model generation method provided in one embodiment of this disclosure;

[0044] Figure 3a A flowchart of a model generation method provided in yet another embodiment of this disclosure;

[0045] Figure 3b for Figure 3a A schematic diagram of the nose model provided in the embodiment shown;

[0046] Figure 3c for Figure 3a The flowchart of the facial model adjustment method provided in the illustrated embodiment;

[0047] Figure 3d for Figure 3a The illustrated embodiment provides a schematic diagram showing the different states corresponding to the adjusted values ​​of the lip protrusion parameter index in the mouth model.

[0048] Figure 4 A flowchart of a model generation method provided in yet another embodiment of this disclosure;

[0049] Figure 5 A schematic diagram of the structure of a model generation apparatus provided in yet another embodiment of this disclosure;

[0050] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure.

[0051] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] In the cosmetic and plastic surgery industry, the high precision and high risk of facial plastic surgery necessitate that doctors conduct an objective quantitative evaluation of the patient's face before surgery and demonstrate the postoperative results. This process is crucial for developing a surgical plan that meets the patient's expectations. However, current technology primarily relies on two-dimensional images to demonstrate postoperative effects, a method with significant limitations. Two-dimensional images lack sufficient depth and realism, making it difficult for patients to accurately understand the postoperative results, potentially leading to outcomes that do not meet patient expectations, or even surgical failure. The reason is that the planar nature of two-dimensional images cannot fully demonstrate the three-dimensional structure and subtle changes of the face, limiting effective communication between doctors and patients.

[0054] However, in the actual cosmetic surgery industry, traditional methods of displaying post-operative results have been used for many years and, to some extent, provide basic visual references. Often, discrepancies between surgical results and patient expectations may be due to other reasons, such as excessively high expectations from the patient or an unsatisfactory post-operative recovery process. Therefore, the field lacks in-depth research into the limitations of two-dimensional images in terms of depth and realism, and their potential impact on surgical outcomes.

[0055] Meanwhile, there are also difficulties in displaying these images using methods other than two-dimensional images. For example, generating a three-dimensional display model requires advanced image processing and modeling techniques to ensure the model's accuracy and realism. Furthermore, the generation and adjustment of the three-dimensional model requires processing large amounts of data and complex algorithms to achieve real-time or near-real-time display effects without increasing the burden on patients and doctors. Therefore,

[0056] Figure 1A schematic diagram illustrating the application scenario of the model generation method provided in this application, such as... Figure 1 As shown, the specific application scenario of this application is as follows: In cosmetic surgery, the doctor will use computer 100 to generate a face display model 110 corresponding to different specific surgical methods and surgical adjustments, so that the patient can understand the corresponding postoperative effect based on the face display model 110, which will help the patient better determine the surgical method and adjustment amount and improve the patient's postoperative satisfaction.

[0057] It should be noted that, Figure 1 The scenario shown includes a computer and a face display model, which are only used as examples of one or a specific number of such models. However, this disclosure is not limited to this. In other words, the number of computers and face display models can be arbitrary.

[0058] The model generation method provided in this application generates a basic model containing facial and multiple organ models from a face image. Then, it extracts and generates a sequence of organ models, which are then adjusted using predefined facial adjustment metrics. Finally, a final display model is generated based on the adjusted facial and organ models. This method enhances the three-dimensionality and realism of the postoperative results, allowing patients to more accurately understand the expected outcome and reducing the risk of unsatisfactory surgery.

[0059] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0060] Figure 2 Flowchart of the model generation method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:

[0061] S201. Determine the face image used to generate the display model, and generate the base model based on the face image.

[0062] The basic model includes a facial model and organ models. There are at least two types of organ models. The facial model, organ models, and display model are all three-dimensional models.

[0063] Specifically, this embodiment provides a general description of the main steps involved in generating the display model.

[0064] In this embodiment of the disclosure, the executing entity is a computer system used to generate the display model, or it can be a program specifically used to generate and adjust the display model. For convenience, it will be referred to as the system below.

[0065] The display model can intuitively reflect the state of a person's face from various angles. It can then be combined with the patient's desired surgical site to show in detail the corresponding facial state for different surgical methods and contents (such as different adjustments to the height of the cheekbone), making it easier for the patient to understand the surgical effect more intuitively.

[0066] The primary task in obtaining the display model is to determine the facial image used to generate it and to generate a base model based on this image. This base model represents a 3D structural model extracted from the original facial image, including a facial model (i.e., a model of the main facial bones and corresponding skin) and at least two organ models (such as the nose and eyes). These models are all 3D to provide a more realistic and three-dimensional visual effect.

[0067] The specific method for generating the basic model can begin with high-resolution facial images. Image processing techniques, such as computer vision and deep learning algorithms, are used to identify and extract facial features. Then, 3D modeling software is used to transform these features into a 3D model, forming the basic model.

[0068] Obtaining high-quality facial images is crucial, as it directly impacts the accuracy of subsequent models. Since generating the display model is time-consuming, it's typically not based on the patient's face but rather on a more general, generic facial model. To better reflect the patient's features, the generic facial model can be refined based on the average facial feature parameters of the patient (e.g., the statistical average of facial feature parameters in the local area), resulting in the display model. Therefore, it's necessary to select facial images for generating the base model based on these average facial feature parameters.

[0069] Next, computer vision technology is used to extract facial feature points from the selected face images. These feature points typically include key areas such as the eyes, nose, mouth, and chin, and the basic geometric structure of the face can be constructed using these points. After extracting the feature points, 3D modeling software is used to convert these points into a 3D mesh model.

[0070] In some embodiments, texture mapping is often performed to enhance the realism of the model, mapping the color information of the original image onto the 3D model to make it look more realistic.

[0071] In some embodiments, deep learning algorithms can also be combined to automatically identify facial features and perform preliminary 3D modeling, thereby improving the accuracy and detail of the model.

[0072] S202. Extract organ models from the basic model and generate organ model sequences.

[0073] The model sequence includes at least two models corresponding to each organ model.

[0074] Specifically, to obtain the final display model, adjustments need to be made to the basic model. The facial organs are the most frequently adjusted parts (the eyes mainly involve adjustments to the eyelids). Therefore, to facilitate the display of the corresponding model changes, the organ models need to be extracted. At the same time, to facilitate the display of the surgical effects corresponding to different adjustment methods for the same part, a model sequence corresponding to each organ model needs to be established. This allows the different parameter changes of the same organ in the model sequence to reflect the corresponding surgical effects, providing patients with a better user experience.

[0075] Therefore, it is necessary to extract organ models from the base model and generate a model sequence of organ models. Here, the model sequence refers to at least two models of different states or forms corresponding to each organ model, used to demonstrate different adjustment effects (during extraction, the same organ model is simply copied multiple times, and then adjusted separately in subsequent steps).

[0076] The specific implementation process involves separating individual organ models from the base model, which can be achieved through feature segmentation algorithms (or through corresponding model modification software); then, these separated organ models are copied multiple times to generate corresponding model sequences, so that possible changes or adjustments can be shown by adjusting different models in the model sequences later.

[0077] Specific feature segmentation algorithms can be based on geometric features, color information, or depth data to ensure the accuracy of segmentation and the integrity of the model.

[0078] S203. Based on the set facial adjustment index, adjust the model sequences of the facial model and organ model to obtain the adjusted facial model and the adjusted organ model sequence.

[0079] Specifically, facial adjustment metrics refer to parameters or standards used to guide model adjustments, such as proportions, symmetry, and specific aesthetic standards. Different facial adjustment metrics apply to different locations within facial and organ models.

[0080] These facial adjustment indicators are pre-configured by doctors or system developers, such as nasal bridge height, nasal wing width, eye corner opening and closing, and eyelid curvature. Specific values ​​can be determined through the doctor's professional judgment. For the corresponding organ model, there are multiple different selectable values ​​for the same indicator. By adjusting a tracheal model in the model sequence based on each value, different adjusted organ models can be obtained, that is, the model sequence of adjusted organ models.

[0081] Specifically, adjusting facial and organ models using facial adjustment metrics may involve complex algorithms, such as shape optimization, nonlinear deformation, and constraint optimization, to ensure that the adjusted model meets the set metric values.

[0082] In some embodiments, automated algorithms can be applied to automatically adjust the corresponding model based on preset facial adjustment indicators and values, reducing human error and time consumption, improving efficiency, and ensuring the consistency and reliability of the adjustment results.

[0083] S204. Generate a display model based on the model sequence of the adjusted facial model and the adjusted organ model.

[0084] Specifically, based on the adjusted facial and organ models, a final display model can be generated. This display model is used to visually demonstrate the postoperative results.

[0085] By selecting organ models from the model sequence of various organ models and integrating them with the adjusted facial model, a complete 3D display model can be obtained. By selecting different models from the model sequence and integrating them with the adjusted facial model, different 3D display models can be obtained to reflect the display effects of different facial adjustments.

[0086] By generating multiple display models simultaneously based on model sequences, the corresponding display model can be directly called for display according to the patient's surgical selection, thereby facilitating patients to quickly and intuitively understand the surgical effect, improving display efficiency, and enhancing the user experience for both patients and doctors.

[0087] The process of integrating and adjusting the model requires ensuring seamless connection between the various models. Therefore, it is necessary to combine model merging technology to form a complete three-dimensional structure by combining different organ models with the facial model.

[0088] In some embodiments, during the model merging process, attention needs to be paid to the proportions and symmetry of the models to ensure that the final displayed model has a natural and realistic visual effect.

[0089] In some embodiments, after the models are merged, texture mapping and lighting processing can be performed to enhance the realism of the models. Texture mapping applies the color information of the original image to the 3D model, making it look more realistic, while lighting processing enhances the model's three-dimensionality and detail by simulating real lighting conditions. The combination of these techniques makes the display model not only accurate in shape but also highly realistic in visual effect.

[0090] In some embodiments, virtual reality (VR) or augmented reality (AR) technologies can also be used to provide an immersive viewing experience, allowing patients to view postoperative results in a virtual environment, enhancing the accuracy of their understanding and decision-making. This not only increases patient engagement but also improves the efficiency and effectiveness of communication between doctors and patients.

[0091] The model generation method provided in this application involves determining a facial image for generating a display model, generating a base model based on the facial image, extracting organ models from the base model, generating a model sequence for the organ models, adjusting the model sequences of the facial and organ models based on set facial adjustment indicators, obtaining adjusted facial and organ model sequences, and finally generating a display model based on the adjusted facial and organ model sequences. This method significantly improves the intuitiveness of postoperative results by generating a three-dimensional display model, solving the problem of insufficient two-dimensional image display. By generating a base model from a facial image and extracting and adjusting the model sequences of organ models, doctors can precisely adjust the face and organs in three-dimensional space. This allows doctors to generate a postoperative effect display model that better meets the patient's expectations based on set facial adjustment indicators, reducing patient misunderstandings about postoperative results, lowering the risk of unsatisfactory surgery, and improving surgical success rate and patient satisfaction.

[0092] Figure 3a Schematic diagram of the model generation process provided in this application Figure 2 ,like Figure 3a As shown, in this embodiment... Figure 2 Based on the examples, the specific implementation process of the model generation method is described in detail, which includes:

[0093] S301. Model the face images in the preset face image library to obtain a preliminary face model.

[0094] Specifically, this embodiment provides a detailed explanation of the actions related to determining the basic model, adjusting the facial model, and adjusting the organ model during the model generation process.

[0095] When determining the facial images used to generate the base model, since cosmetic surgery needs to reflect the patient's aesthetic preferences, the facial images can be determined based on public or open-source databases of photos of beautiful people in China to reflect the general aesthetic tendencies of patients and ensure that the display model selected and generated from them can better conform to the aesthetics of the general subjective evaluation of patients.

[0096] After obtaining facial images, these two-dimensional images can be transformed into three-dimensional models using computer vision techniques and 3D reconstruction algorithms. For example, multi-view stereo vision (MVS) or structured light scanning techniques can be used to capture the depth information of the face in the facial images, thereby generating a 3D model.

[0097] In some embodiments, existing facial modeling software can also be used to complete the process of generating a preparatory facial model from a face image, such as using FaceGen Modeller software for modeling.

[0098] A database of photos of beautiful people in China can reflect the aesthetic preferences of the Chinese public in the current era, and the models selected from it can better match the public's subjective aesthetic evaluation.

[0099] S302. Select the models with a frontal view among the prepared face models as candidate models.

[0100] Specifically, in the aforementioned steps, multiple preliminary face models can be generated in batches based on selected face images, and then the base model used to generate the display model can be determined from the facial index parameters of the preliminary face models.

[0101] In this process, models with a frontal orientation from the preliminary face models need to be identified as candidate models. These candidate models represent those selected from the preliminary face models that meet specific orientation requirements, and are used for subsequent metric comparison and adjustment.

[0102] Therefore, it is necessary to identify the facial orientation in the preliminary face model to filter out models with a frontal orientation. This can be achieved by analyzing the facial feature points of the model. By calculating the relative positions and angles of the facial feature points, it can be determined whether the model is facing forward.

[0103] A frontal view model typically provides more symmetrical and complete facial information, which helps improve the accuracy of structural indicator comparisons in subsequent steps.

[0104] S303. Determine the deviation between the candidate face structure index values ​​in each candidate model and the pre-determined face structure reference values.

[0105] Specifically, after determining the candidate model, the deviations of the values ​​of various facial structure indicators of the candidate face can be further compared with the pre-determined facial structure reference values. The pre-determined facial structure reference values ​​can be determined by directly referring to the corresponding standard documents, such as "GB / T2428-2024 Adult Head and Face Dimensions" (or statistical analysis results of a large amount of facial data in the local area or similar statistical documents can be used) to ensure that the values ​​of various facial structure feature indicators of different patients' faces are consistent.

[0106] As shown in Table 1 below, by measuring the measured values ​​of the candidate's facial structure index and the predetermined facial structure reference values, the deviation values ​​of the corresponding types of facial structure indexes can be obtained.

[0107] Table 1. Examples of measured values ​​and corresponding offset values ​​for facial structure indicators in candidate models.

[0108]

[0109] Based on the offset values ​​of each candidate model, the candidate models can be further determined.

[0110] S304. The candidate model with the smallest deviation value is determined as the basic model.

[0111] Specifically, only one base model is needed. Therefore, the candidate model with the smallest deviation value (sum or average) obtained from the above steps can be directly determined as the base model.

[0112] The implementation process includes first selecting the model with the smallest deviation value, and then optional implementation methods include using a symmetry detection algorithm to automatically evaluate the symmetry of the model, or through expert review combined with automated tools, to ensure that the selection of the basic model is accurate and meets aesthetic standards.

[0113] In some embodiments, if there are at least two candidate models with the smallest deviation value, the candidate model with the highest facial symmetry among the candidate models with the smallest deviation value is determined as the base model, wherein the facial symmetry is determined based on the difference in facial structure index between the two sides of the candidate face.

[0114] Specifically, in addition to comparing deviation values, facial symmetry can be further compared. Facial symmetry can be determined by calculating the difference in structural indices between the two sides of the face. This dual screening mechanism ensures that the selected base model not only conforms to structural reference values ​​but also possesses good symmetry.

[0115] In some embodiments, the base model can be determined by first using an image recognition algorithm specifically designed for face recognition to analyze the facial orientation and facial structure index values ​​of each face image from a pre-defined face image library. The algorithm then selects face images with a frontal orientation, the smallest deviation between the facial structure index values ​​and pre-determined facial structure reference values, and the highest facial symmetry. The base model is then derived by modeling based on these face images. This face image selection process can also be implemented using a pre-trained neural network model.

[0116] Those skilled in the art can choose this implementation method or the implementation methods in steps S301 to S304 according to actual needs, and there are no restrictions here.

[0117] S305. Extract organ models from the basic model and generate organ model sequences.

[0118] The model sequence includes at least two models corresponding to each organ model.

[0119] Specifically, after determining the basic model, facial models and organ models can be extracted from it. By changing the facial structure indicators corresponding to the organ models, the model sequence of organ models can be obtained. (In this step, the individual organ models in the model sequence can be initially adjusted and further adjusted in subsequent steps. Alternatively, the individual organ models can be directly copied in this step and adjusted all at once in subsequent steps.)

[0120] In some embodiments, the organ models include eye models, mouth models, and nose models.

[0121] Specifically, the organ models separated from the base model are mainly the eye model, mouth model, and nose model, which are the three most frequently involved wonders in cosmetic surgery. These organ models can be extracted from the base model using feature segmentation algorithms.

[0122] In some embodiments, such as Figure 3b As shown, it is a structural diagram of the nose model. The part of the diagram that does not include the grid lines is the area corresponding to the nose model.

[0123] The separated organ models are not separated from the facial model, but remain on the facial model. During the numerical adjustment process, only the organ models are adjusted in a targeted manner. For example, the eyes and mouth remain unchanged, while the nose is adjusted to generate multiple models containing different nose models (at this time, the model also contains various organ models and facial models at the same time), which are the organ sequences corresponding to these different nose models.

[0124] S306. Based on the difference between the facial structure index value of the facial model and the predetermined facial structure reference value, adjust the facial model to obtain the adjusted facial model.

[0125] Specifically, the obtained basic model will still differ from the predetermined facial structure reference values. Therefore, the facial model can be adjusted directly based on the differences.

[0126] Referring to the national standard GB / T 2428-2024 Adult head and face dimensions, the model parameters were sculpted and modified to conform to the average value of Chinese women.

[0127] Furthermore, such as Figure 3c The diagram shows a flowchart of the facial model adjustment method. The facial structure reference values ​​include the mean and standard deviation of each facial structure index. Therefore, the specific facial model adjustment method can include the following steps:

[0128] S3061. Rotate the facial model until the eye and ear plane is perpendicular to the coronal axis.

[0129] Specifically, the oculomotor plane and coronal axis are specialized terms used in the field of cosmetic surgery to describe locations.

[0130] Since the generated base model and facial model are usually not in a completely frontal state (usually slightly tilted, because completely frontal face images are generally only used for ID photos, the images in the preset face image library are usually slightly tilted relative to a completely frontal state).

[0131] Therefore, before making any facial contouring adjustments, it is necessary to standardize the facial model's posture to facilitate subsequent precise adjustments.

[0132] By identifying key points on the facial model, such as the corners of the eyes, earlobes, and the tip of the chin, the position of the eye-ear plane is calculated. Then, through mathematical transformation, the facial model is rotated to a position where the plane is perpendicular to the coronal axis (an automated posture adjustment algorithm can also be preset to automatically complete the rotation process according to preset standards).

[0133] S3062. Scale the facial model based on the head size reference value in the facial structure reference value.

[0134] Specifically, since the facial model size obtained at this time will differ from the facial size (head size reference value) in the facial structure reference value, it is necessary to adjust the size of the facial model by scaling it up in order to further adjust the facial model.

[0135] At this point, the current size of the facial model needs to be calculated and compared with a head size reference. Based on the comparison result, a scaling factor (such as reducing by a factor of 0.84 or other factors) is determined to adjust the facial model to a size that conforms to the reference standard. The scaling process needs to maintain the model's proportions and symmetry to ensure that the adjusted model looks natural and meets aesthetic standards.

[0136] S3063. Determine the difference between the facial structure index value of the facial model and the average value of the corresponding category in the facial structure reference value.

[0137] Specifically, after scaling, the difference between the facial structure index values ​​of the facial model and the average values ​​of the corresponding categories in the facial structure reference values ​​can be calculated. The facial structure index values ​​refer to the geometric feature parameters of the facial model, such as the height of the bridge of the nose, the distance between the eyes, and the width of the chin.

[0138] S3064. The ratio of the difference to the standard deviation of the corresponding type in the face structure reference value is determined as the offset.

[0139] Specifically, the ratio of the difference to the standard deviation of the corresponding category in the face structure reference value is determined as the offset to ensure that the face structure index value of the face model differs from the face structure reference value by 1 standard deviation.

[0140] In some embodiments, steps S3063 to S3064 can also be performed continuously, without having to be broken down into multiple steps.

[0141] S3065. Based on the offset, adjust the corresponding facial structure index values ​​in the facial model.

[0142] Specifically, based on the offset, various structural parameters of the facial model are adjusted accordingly. These adjustments may involve shape deformation, size scaling, and positional movement to ensure that the adjusted model appears visually natural and meets the expected results.

[0143] In some embodiments, automated adjustment tools can also be used to automatically complete the adjustment process in steps S3061 to S3065.

[0144] These steps ensure that the adjusted facial model not only conforms to the reference values ​​for facial structure, but also has good visual effects and patient acceptance.

[0145] S307. Based on the range of regulation parameters corresponding to the organ model, determine the adjustment value of each organ model in the corresponding model sequence.

[0146] Specifically, each organ model corresponds to different facial structure parameters and has a certain adjustment range, that is, the acceptable adjustment range of each organ model, to ensure that the adjusted model looks natural and meets aesthetic standards. For example, the ideal range for the height of the bridge of the nose is 10 to 14 millimeters, so nose models with different bridge heights can be generated at 0.5-millimeter intervals.

[0147] The specific range of adjustment parameters can be determined based on clinical experience or statistical data (such as the numerical range in the aforementioned facial structure reference values).

[0148] Based on these parameter ranges, the adjustment values ​​for each organ model in the model sequence can be calculated to guide the subsequent model adjustment process and ensure that the adjusted model meets the expected results.

[0149] S308. Based on the adjustment value, adjust the organ models respectively to obtain the model sequence of the adjusted organ models.

[0150] Specifically, the model sequence contains multiple adjusted organ models with different adjustment values ​​corresponding to the same organ model, in order to demonstrate the postoperative effects of different adjustments.

[0151] Specific adjustments can include shape deformation, size scaling, and position movement, to ensure that the adjusted model looks natural and meets the expected results.

[0152] like Figure 3dAs shown, this diagram illustrates the different states corresponding to varying adjusted values ​​of the lip protrusion parameter in the mouth model. By adjusting the facial structure parameters in the organ model, various organ models in different states can be obtained.

[0153] S309. Replace the double eyelid texture in the eye model with a single eyelid texture.

[0154] Specifically, single eyelids are used because, statistically, the proportion of patients with single eyelids is greater than that with double eyelids. Depending on the actual situation, all single eyelid stickers can also be replaced with double eyelid stickers.

[0155] In practical implementation, for the eye model, the texture map needs to be replaced to change the model's appearance, replacing the double eyelid map with a single eyelid map. The system can identify the double eyelid map in the eye model and then select a pre-drawn single eyelid map for replacement. The system can use texture editing software to manually replace the map, or it can use automated tools to automatically complete the map replacement process according to preset standards, thereby improving efficiency and consistency.

[0156] This allows for the adjustment of facial and organ models.

[0157] S310. Select the adjusted organ model from each model sequence for placement in the facial model.

[0158] Specifically, when providing models to patients to demonstrate surgical results, usually only one model (such as directly showing the post-operative state through a model) or two models (such as showing the pre-operative state through one model and the post-operative state through one model) are displayed. In this case, the system can directly select the facial model and combine it with the organ model that best fits the reference value of the facial structure index to obtain the display model. Alternatively, it can combine the most common surgical types and the corresponding facial structure index adjustment amounts to determine the corresponding display model.

[0159] In some embodiments, due to the aforementioned steps adjusting the organ models, it is usually inconvenient to keep only one model fixed. However, in actual model fusion, it is usually the fusion of multiple different organ models. It may also include the fusion of organ models in the base model with the adjusted facial model, as well as selecting the most common model from the organ sequence (e.g., selecting one most common modified state for each parameter, and selecting one organ model for each parameter type) and combining it with the facial model. Therefore, model fusion also involves adjusting the model position and connection points.

[0160] The aforementioned method yields ten to twenty model combinations (more models can be created, and then the most common combinations can be extracted based on their frequency of occurrence, such as procedures like canthoplasty, double eyelid surgery, and nose bridge reduction). These combinations can generate display models for subsequent presentation based on user needs.

[0161] In some embodiments, the adjusted model of each organ model in the model sequence can be evaluated based on preset selection criteria (such as the deviation value between the organ model and the reference value of facial structure indicators) to select the model that best meets the aesthetic standards. In addition to the above criteria, evaluation indicators such as proportion and visual effect can also be combined to achieve automated selection.

[0162] In some embodiments, unselected organ models are also stored in the system, and the corresponding organ model is selected (a label with corresponding facial structure indicators can be added to the organ model) according to the type of surgery and adjustment required by the patient, replacing the organ model in the current display model, thereby quickly generating a new display model to show the effect after surgery.

[0163] S311. Move the adjusted organ model into the adjusted facial model to generate the display model.

[0164] Specifically, the selected adjusted organ model is integrated with the adjusted facial model to ensure a seamless connection between the various models, resulting in the final display model.

[0165] Therefore, by generating display models, it serves two purposes: firstly, it provides aesthetic evaluation research, facilitating researchers to compare different display models, score different model combinations, and understand people's aesthetic preferences; secondly, it has clinical applications in cosmetic surgery, quickly displaying corresponding models based on the patient's desired type of surgery. Even if no similar model exists, a corresponding display model can be quickly obtained by selecting from organ sequences, generating display animations and images for users to choose from. This ensures that the display models are tailored to the patient's characteristics, conveniently demonstrate the effects of cosmetic surgery, and enhances the efficiency and effectiveness of communication between doctors and patients.

[0166] The model generation method provided in this application ensures model accuracy by generating a base model from high-resolution facial images and extracting and adjusting organ models. Next, by setting facial adjustment indicators and spatial ranges, the model undergoes translation and gradation processing, seamlessly integrating the organ and facial models. Finally, through symmetry processing and the combination of hair models, the generated display model appears more natural and complete. This approach not only improves the intuitiveness of postoperative results but also enhances communication efficiency and decision-making accuracy between doctors and patients, reduces the risk of unsatisfactory surgery, and increases patient satisfaction. Through automated and intelligent technologies, the solution effectively reduces human error and improves the efficiency and consistency of model generation.

[0167] Figure 4 The flowchart for generating the model provided in this application is shown in Figure 3. Figure 4 As shown, in this embodiment... Figure 2 and Figure 3a Based on the examples, the part of the model generation method that generates a display model after model adjustment is further described in detail. This method includes:

[0168] S401. Determine the face image used to generate the display model, and generate the base model based on the face image.

[0169] The basic model includes a facial model and an organ model. There are at least two types of organ models. The facial model, organ model, and display model are all three-dimensional models.

[0170] Specifically, this embodiment describes the process of generating a display model after adjusting the facial and organ models.

[0171] S402. Extract organ models from the basic model and generate organ model sequences.

[0172] The model sequence includes at least two models corresponding to each organ model.

[0173] S403. Based on the set facial adjustment index, adjust the model sequences of the facial model and organ model to obtain the adjusted facial model and the adjusted organ model sequence.

[0174] Specifically, steps S401 to S403 are the same as the corresponding steps in the aforementioned embodiments, and will not be repeated here.

[0175] S404. Determine the spatial range of the adjusted organ model.

[0176] Specifically, after adjusting the facial and organ models, it's necessary to determine how to combine them to obtain the display model. At this point, the spatial extent of the adjusted organ model must first be determined. Spatial extent refers to the position and range of the adjusted organ model in three-dimensional space, used to guide subsequent translation and integration operations.

[0177] The system needs to calculate the bounding box (BGB) of each adjusted organ model to determine its position and size in 3D space. This can be achieved by analyzing the vertex coordinates of the model, ensuring accurate translation and integration of the model in subsequent steps.

[0178] S405. Based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model, translate the adjusted organ model.

[0179] Specifically, the system needs to move the organ model in three-dimensional space to align it correctly with the facial model. This process involves determining the vertex coordinates of the adjusted organ model, calculating the differences between these vertices and their corresponding vertices in the facial model, calculating the translation vector based on these differences, and then translating the organ model accordingly in three-dimensional space.

[0180] This ensures that the organ model is accurately aligned with the facial model in terms of position, laying the foundation for subsequent integration and display.

[0181] S406. Apply a gradient treatment to the boundary region within a set distance of the vertex coordinates.

[0182] Specifically, during the fusion process, since it is necessary to merge organ models from different models together, a gradient processing based on the contact area is required.

[0183] In practice, after the translation is completed, the vertex coordinates in a certain range (such as an area with a width of 8mm) around the boundary where the organ model and the face model meet need to be gradient-processed to make the model look smooth.

[0184] In some embodiments, this can be achieved by adjusting the normal direction and color value of the vertices within the boundary region, ensuring that the final display model appears visually seamless and natural. The specific adjustment formula can be:

[0185] ;

[0186] Where x is the distance moved along the normal direction from the boundary region of the organ model; the greater the distance moved, the lighter the color displayed.

[0187] In some embodiments, automated gradient processing tools can also be used to automatically complete the gradient process according to preset standards.

[0188] S407. Adjust the model vertices of the adjusted organ model to the average value of the face normals of the face model to obtain the moved organ model.

[0189] Specifically, the normals corresponding to the vertices of the adjusted organ model are adjusted to the average value of the face normals to obtain the moved organ model, ensuring that the model rendering result is normal.

[0190] The system needs to calculate the normals at each vertex of the facial model, and then apply the average of these normals to the corresponding organ model vertices. By adjusting the direction of the normals, it can be ensured that the organ models are consistent with the facial model in terms of lighting and shadow effects, thereby improving the visual consistency and realism of the displayed models.

[0191] S408. Generate a display model based on the moved organ model and the adjusted facial model.

[0192] Specifically, after the preceding processing, the system will integrate the moved organ model with the adjusted facial model to ensure a seamless connection between the various models, forming a complete three-dimensional structure.

[0193] Furthermore, the specific integration steps in generating the display model also include the following:

[0194] Step A1: Based on the sagittal plane, remove the right half of the face from the moved organ model and the adjusted facial model.

[0195] Specifically, the sagittal plane, a vertical plane passing through the midline of the human body, divides the body into left and right parts. By removing the right half of the face, a basis is provided for subsequent symmetry processing, ensuring that the left half of the face in the model can be accurately mirrored.

[0196] Step A2: Symmetry processing is performed on the left half of the face of the moved organ model and the adjusted facial model.

[0197] Specifically, by mirroring, one side of the model is copied to the other side to ensure good visual symmetry, thereby improving the aesthetic effect of the displayed model and patient satisfaction.

[0198] Step A3: Import the hair model and combine it with the face model to obtain the display model.

[0199] Specifically, based on the aforementioned operations, a hair model is imported and combined with a facial model to obtain a display model, thereby improving the completeness and realism of the display model.

[0200] As a result, the display model is not only structurally complete, but also more realistic and natural in visual effects, possessing a high degree of realism and aesthetic appeal.

[0201] In some embodiments, after generating the display model, a facial photo from the target angle can be obtained based on the display model, or a corresponding display animation can be generated based on the display model and set facial movements.

[0202] Specifically, to facilitate the quick display of the model to patients, a screenshot can be taken directly after the model is generated to obtain a front and side view photo, so as to quickly provide patients with a two-dimensional display of the surgical effect.

[0203] In addition, the system can automatically record and display animated GIFs of the model. By following the pre-defined facial movements (such as common actions like turning the head, closing the eyes, and opening the mouth, which can be achieved through 3D model debugging software), and adjusting the recording frame rate and number of frames (e.g., recording as an AVI file at 15fps and 135fps, and then converting it to GIF format), a display animation can be generated, resulting in a better playback animation.

[0204] When presenting the system to patients, combining two-dimensional photos with three-dimensional animations allows users to obtain the best display effect of the system and enhances patients' understanding of the surgical outcome.

[0205] The model generation method provided in this application generates a basic model from a facial image and extracts and adjusts organ models to ensure the basic accuracy of the model. Then, by setting the spatial range and performing vertex translation and gradient processing, seamless integration of the organ model and facial model is achieved, enhancing the visual naturalness of the model. Further symmetry processing and the combination with the hair model make the final displayed model more structurally complete and realistic. This solution not only improves the visualization accuracy of postoperative results but also promotes effective communication between doctors and patients, helping patients better understand and anticipate surgical outcomes. At the same time, automated processing steps reduce errors from human intervention, improve the efficiency and consistency of the overall operation, and ultimately increase patient satisfaction and surgical success rate.

[0206] Figure 5 A schematic diagram of the structure of the model generation device provided in this application is shown below. Figure 5 As shown, the model generation apparatus 500 provided in this embodiment includes:

[0207] The acquisition module 510 is used to determine the face image used to generate the display model, and generate a basic model based on the face image. The basic model includes a facial model and an organ model. There are at least two types of organ models. The facial model, organ model and the display model are all three-dimensional models.

[0208] Extraction module 520 is used to extract organ models from the base model and generate a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model;

[0209] The adjustment module 530 is used to adjust the model sequences of the facial model and the organ model based on the set facial adjustment index, so as to obtain the adjusted facial model and the adjusted organ model sequence.

[0210] The generation module 540 is used to generate the display model based on the model sequence of the adjusted facial model and the adjusted organ model.

[0211] In one possible implementation, the acquisition module 510 is specifically used to: model face images in a preset face image library to obtain a preliminary face model; identify models with a frontal view among the preliminary face models as candidate models; determine the deviation values ​​of the candidate face structure index values ​​in each candidate model from the predetermined face structure reference values; and identify the candidate model with the smallest deviation value as the basic model.

[0212] In one possible implementation, the acquisition module 510 is specifically used to determine the candidate model with the highest facial symmetry among the candidate models with the smallest deviation value if there are at least two candidate models with the smallest deviation value. The facial symmetry is determined based on the difference in facial structure index values ​​between the two sides of the candidate face.

[0213] In one possible implementation, the acquisition module 510 specifically includes organ models including an eye model, a mouth model, and a nose model.

[0214] In one possible implementation, the adjustment module 530 is specifically used to: adjust the facial model based on the difference between the facial structure index value of the facial model and the predetermined facial structure reference value to obtain the adjusted facial model; determine the adjustment value of each organ model in the corresponding model sequence based on the adjustment parameter range corresponding to the organ model; and adjust the organ models respectively based on the adjustment value to obtain the model sequence of the adjusted organ models.

[0215] In one possible implementation, the adjustment module 530 is specifically configured to: if the facial structure reference values ​​include the average and standard deviation of each facial structure index, rotate the facial model so that the eye-ear plane is perpendicular to the coronal axis; scale the facial model based on the head size reference values ​​in the facial structure reference values; determine the difference between the facial structure index values ​​of the facial model and the average values ​​of the corresponding categories in the facial structure reference values; determine the ratio of the difference to the standard deviation of the corresponding categories in the facial structure reference values ​​as the offset; and adjust the corresponding facial structure index values ​​in the facial model based on the offset.

[0216] In one possible implementation, the adjustment module 530 is further configured to, based on the adjustment value, adjust the organ models respectively, and after obtaining the model sequence of the adjusted organ models, replace the double eyelid texture in the eye model with the single eyelid texture.

[0217] In one possible implementation, the generation module 540 is specifically used to select an adjusted organ model from each model sequence for placement in the facial model; move the adjusted organ model into the adjusted facial model to generate a display model.

[0218] In one possible implementation, the generation module is specifically used to: determine the spatial range of the adjusted organ model; translate the adjusted organ model based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model; perform gradient processing on the boundary regions within a set distance of the vertex coordinates; adjust the model vertices of the adjusted organ model to the average value of the surface normals of the facial model to obtain the moved organ model; and generate a display model based on the moved organ model and the adjusted facial model.

[0219] In one possible implementation, the generation module 540 is specifically used to: remove the right half of the face of the moved organ model and the adjusted facial model based on the sagittal plane; perform symmetry processing on the left half of the face of the moved organ model and the adjusted facial model; import the hair model and combine the hair model with the facial model to obtain the display model.

[0220] In one possible implementation, the generation module 540 is further configured to, after generating a display model based on the model sequence of the adjusted facial model and the adjusted organ model, include at least one of the following: obtaining a facial photograph at a target angle based on the display model; and generating a corresponding display animation based on the display model and set facial movements.

[0221] The model generation device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0222] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the device 60 further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus 604.

[0223] In a specific implementation, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to perform the above-described method.

[0224] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0225] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0226] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0227] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0228] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0229] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0230] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0231] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0232] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0233] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0234] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0235] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0236] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0237] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A model generation method, characterized in that, include: A face image is determined for generating the display model, and a base model is generated based on the face image. The base model includes a facial model and an organ model. There are at least two types of organ models. The facial model, organ model, and display model are all three-dimensional models. Extract organ models from the base model and generate a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model; Based on the set facial adjustment index, the model sequences of the facial model and the organ model are adjusted to obtain the adjusted facial model and the adjusted organ model sequence. The display model is generated based on the model sequence of the adjusted facial model and the adjusted organ model; The process of adjusting the model sequences of the facial model and the organ model based on set facial adjustment indicators to obtain adjusted facial model and adjusted organ model model sequences includes: The facial model is rotated until the eye-ear plane is perpendicular to the coronal axis; the facial model is scaled based on the head size reference value in the facial structure reference value; the difference between the facial structure index value of the facial model and the average value of the corresponding type in the facial structure reference value is determined; the ratio of the difference to the standard deviation of the corresponding type in the facial structure reference value is determined as the offset; based on the offset, the corresponding facial structure index value in the facial model is adjusted to obtain the adjusted facial model; Based on the range of adjustment parameters corresponding to the organ model, determine the adjustment value of each organ model in the corresponding model sequence; Based on the adjustment values, the organ models are adjusted respectively to obtain the model sequence of the adjusted organ models.

2. The method according to claim 1, characterized in that, The step of determining the face image used to generate the display model and generating the base model based on the face image includes: Model the face images in the preset face image library to obtain the preliminary face model; The models with a frontal view among the preliminary face models are identified as candidate models; Determine the deviation between the candidate face structure index values ​​in each candidate model and the pre-determined face structure reference values; the face structure reference values ​​include the mean and standard deviation corresponding to each face structure index; The candidate model with the smallest deviation value is determined as the base model.

3. The method according to claim 2, characterized in that, The step of determining the candidate model with the smallest deviation value as the basic model includes: If there are at least two candidate models with the smallest deviation value, then the candidate model with the highest facial symmetry among the candidate models with the smallest deviation value is determined as the base model, wherein the facial symmetry is determined based on the difference in facial structure index between the two sides of the candidate face.

4. The method according to claim 1, characterized in that, The organ models include eye models, mouth models, and nose models.

5. The method according to claim 4, characterized in that, After adjusting the organ models based on the adjustment values ​​to obtain the adjusted organ model sequence, the process further includes: Replace the double eyelid texture in the eye model with a single eyelid texture.

6. The method according to claim 1, characterized in that, The generation of the display model based on the model sequence of the adjusted facial model and the adjusted organ model includes: Select the adjusted organ model for placement in the facial model from each model sequence; The adjusted organ model is moved into the adjusted facial model to generate the display model.

7. The method according to claim 6, characterized in that, The step of moving the adjusted organ model into the adjusted facial model to generate the display model includes: Determine the spatial extent of the adjusted organ model; Based on the vertex coordinates of the model vertices in the spatial range and the corresponding vertex positions in the facial model, the adjusted organ model is translated. The boundary region within a set distance of the vertex coordinates is subjected to a gradient process; The model vertices of the adjusted organ model are adjusted to the average value of the surface normals of the facial model to obtain the moved organ model; The display model is generated based on the moved organ model and the adjusted facial model.

8. The method according to claim 7, characterized in that, The generation of the display model based on the moved organ model and the adjusted facial model includes: Based on the sagittal plane, the right half of the face of the moved organ model and the adjusted facial model is removed; Symmetry processing is performed only on the left half of the face of the moved organ model and the adjusted facial model; Import the hair model and combine it with the face model to obtain the display model.

9. The method according to any one of claims 1 to 8, characterized in that, After generating the display model based on the model sequence of the adjusted facial model and the adjusted organ model, the method further includes at least one of the following: A facial photograph from the target angle is obtained based on the displayed model; Based on the display model and the set facial movements, a corresponding display animation is generated.

10. A model generation apparatus, characterized in that, include: The acquisition module is used to determine the face image used to generate the display model, and generate a basic model based on the face image. The basic model includes a facial model and an organ model. There are at least two types of organ models. The facial model, organ model and the display model are all three-dimensional models. An extraction module is used to extract organ models from the base model and generate a model sequence of the organ models, wherein the model sequence includes at least two models corresponding to each organ model; The adjustment module is used to adjust the model sequences of the facial model and the organ model based on the set facial adjustment index, so as to obtain the adjusted facial model and the adjusted organ model sequence. A generation module is used to generate the display model based on the model sequence of the adjusted facial model and the adjusted organ model; The adjustment module is specifically used to rotate the facial model so that the eye-ear plane is perpendicular to the coronal axis; scale the facial model based on the head size reference value in the facial structure reference value; determine the difference between the facial structure index value of the facial model and the average value of the corresponding type in the facial structure reference value; determine the ratio of the difference to the standard deviation of the corresponding type in the facial structure reference value as the offset; and adjust the corresponding facial structure index value in the facial model based on the offset to obtain the adjusted facial model. Based on the range of adjustment parameters corresponding to the organ model, determine the adjustment value of each organ model in the corresponding model sequence; Based on the adjustment values, the organ models are adjusted respectively to obtain the model sequence of the adjusted organ models.

11. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.

Images