Model processing method and apparatus, electronic device, computer-readable storage medium, and computer program product

Abstract

The application provides a model processing method and device, electronic equipment, computer readable storage medium and computer program product; the method comprises: obtaining a first face model and a second face model to be processed, and performing global rigid alignment on the first face model and the second face model to obtain an initial alignment result; for each first regional model, performing first rigid alignment on the first regional model to obtain a regional alignment result corresponding to the first regional model; performing first smoothing processing on the regional alignment result based on the regional weight corresponding to the first regional model to obtain a smoothed regional alignment result; performing second rigid alignment on each accessory component corresponding to the first regional model to obtain a target alignment result corresponding to the first regional model; and performing second smoothing processing on each target alignment result corresponding to the first regional model to obtain a target model corresponding to the second face model. Through the application, the accuracy and fineness of face model processing can be improved.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a model processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In the creation of 3D digital content, the refined modeling and fusion technology of 3D digital human facial meshes is becoming increasingly important. For example, in the practical application of building personalized digital virtual humans, it is usually necessary to transfer and fuse facial features based on specific objects onto a basic mesh model with a standard topological structure to achieve a high degree of fidelity in local anatomical features such as facial features and cheeks while maintaining the overall proportional harmony of the face. However, facial model fusion and alignment technologies in related fields often struggle to balance the accuracy of global alignment with the precision of local deformation when dealing with complex spatial pose differences, leading to problems such as geometric distortion or visual unnaturalness in the final generated facial model. Summary of the Invention

[0003] This application provides a model processing method, apparatus, computer-readable storage medium, and computer program product that can improve the accuracy and precision of facial model processing.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a model processing method, the method comprising: A first facial model and a second facial model to be processed are obtained, and the first facial model and the second facial model are globally rigidly aligned to obtain an initial alignment result. The initial alignment result includes a first region model corresponding to each of multiple facial features. For each of the first region models, a first rigid alignment is performed on the first region model to obtain the region alignment result corresponding to the first region model; Based on the region weights corresponding to the first region model, the region alignment result is subjected to a first smoothing process to obtain a smoothed region alignment result. A second rigid alignment is performed on each of the subordinate components corresponding to the first region model to obtain the target alignment result corresponding to the first region model, wherein the rigid transformation parameters used for the first rigid alignment and the second rigid alignment are the same; A second smoothing process is performed on the target alignment result corresponding to each of the first region models to obtain the target model corresponding to the second facial model.

[0005] This application provides a model processing apparatus, including: The alignment module is used to acquire a first facial model and a second facial model to be processed, and to perform global rigid alignment on the first facial model and the second facial model to obtain an initial alignment result. The initial alignment result includes a first region model corresponding to each of multiple facial features. The alignment module is further configured to perform a first rigid alignment on the corresponding first region model for each first region model, so as to obtain the region alignment result corresponding to the first region model. The smoothing module is used to perform a first smoothing process on the region alignment result based on the region weights corresponding to the first region model, so as to obtain a smoothed region alignment result. The alignment module is further configured to perform a second rigid alignment on each of the subordinate components corresponding to the first region model to obtain a target alignment result corresponding to the first region model, wherein the rigid transformation parameters used for performing the first rigid alignment and the second rigid alignment are the same. The smoothing module is further configured to perform a second smoothing process on the target alignment result corresponding to each of the first region models to obtain the target model corresponding to the second facial model.

[0006] In the above scheme, the alignment module is further configured to determine the region mask corresponding to each facial feature; divide the first facial model based on each region mask to obtain the divided first facial model; determine the global transformation parameters between the first facial model and the second facial model; and perform a global rigid transformation on the divided first facial model or the second facial model based on the global transformation parameters to obtain the initial alignment result.

[0007] In the above scheme, the alignment module is further configured to determine the first mesh data corresponding to the first facial model and the second mesh data corresponding to the second facial model; determine the first mapping relationship between the first facial model and the second facial model based on the first mesh data and the second mesh data; and determine the global transformation parameters based on the first mapping relationship, wherein the global transformation parameters include at least one of the following: rotation parameters, translation parameters, and uniform scale parameters.

[0008] In the above scheme, the alignment module is further configured to determine the first mesh data corresponding to the first facial model and the semantic partition information of the first facial model; for each facial feature, based on the semantic partition information, determine the target vertex from the first mesh data, the target vertex being located in the second region model corresponding to the facial feature; set the mask value of the target vertex to a first value, and set the mask values ​​of the first other vertices besides the target vertex to a second value, to obtain the region mask corresponding to the facial feature.

[0009] In the above scheme, the alignment module is further configured to determine a third region model in the second facial model corresponding to the facial features; determine the rigid transformation parameters between the first region model and the third region model; and perform rigid transformation processing on the first region model based on the rigid transformation parameters to obtain the region alignment result.

[0010] In the above scheme, the alignment module is further configured to determine the first vertex data corresponding to the first region model from the first mesh data corresponding to the first face model; determine the second vertex data corresponding to the third region model from the second mesh data corresponding to the second face model; and determine the rigid transformation parameters based on the first vertex data and the second vertex data.

[0011] In the above scheme, the alignment module is further configured to determine a second mapping relationship between the first vertex data and the second vertex data; based on the second mapping relationship, construct an objective function to characterize the alignment error between the first region model and the third region model; and obtain the rigid transformation parameters by solving the objective function.

[0012] In the above scheme, the first region model includes multiple first vertices, and the region weight includes the vertex weight corresponding to each first vertex; the smoothing module is further configured to, for each first vertex, determine the first coordinate information of the first vertex in the initial alignment result and the second coordinate information of the first vertex in the region alignment result; based on the vertex weight corresponding to the first vertex, perform a weighted summation of the first coordinate information and the second coordinate information to obtain the first target coordinate corresponding to the first vertex; and construct the smoothed region alignment result based on the first target coordinate corresponding to each first vertex.

[0013] In the above scheme, the smoothing module is further configured to: determine the center vertex corresponding to the first region model from multiple first vertices; determine the first vertices other than the center vertex as second other vertices; determine the vertex weight corresponding to the center vertex as a first value; determine the distance information between the second other vertex and the center vertex for each second other vertex; and determine the vertex weight corresponding to the second other vertex based on the distance information, wherein the vertex weight corresponding to the second other vertex is negatively correlated with the distance information and the vertex weight corresponding to the second other vertex is less than the first value.

[0014] In the above scheme, the alignment module is further configured to, for each of the auxiliary components, determine the rigid transformation parameters used by the first region model corresponding to the auxiliary component when performing the first rigid alignment; determine the constraint conditions of the auxiliary component, the constraint conditions being used to constrain the deformation degree of the auxiliary component; and perform a second rigid alignment on the auxiliary component based on the rigid transformation parameters and the constraint conditions to obtain the target alignment result corresponding to the first region model.

[0015] In the above scheme, the alignment module is further configured to determine the initial relative position information between the vertices of each component in the auxiliary component; determine the deformation threshold corresponding to the auxiliary component based on each initial relative position information; and construct the constraint condition based on the deformation threshold, wherein the constraint condition is used to constrain that the relative position change between the vertices of each component is less than or equal to the deformation threshold when performing the second rigid alignment.

[0016] In the above scheme, the smoothing module is further configured to: determine the third coordinate information of each second vertex included in the target alignment result for each target alignment result; determine the coverage state between the second vertex and each other target alignment result for each second vertex; determine the smoothed second target coordinates of the second vertex based on the coverage state and the third coordinate information; and construct the target model based on the second target coordinates corresponding to each vertex.

[0017] In the above scheme, the smoothing module is further configured to: determine the third coordinate information as the second target coordinates when the coverage state indicates that the second vertex is not covered by other target alignment results; determine the first region model corresponding to each target alignment result that covers the second vertex as the target region model when the coverage state indicates that the second vertex is covered by other target alignment results; and determine the second target coordinates of the second vertex based on the region weight corresponding to each target region model.

[0018] In the above scheme, the smoothing module is further configured to determine the vertex weight corresponding to the second vertex from the region weights corresponding to each target region model; normalize each vertex weight corresponding to the second vertex to obtain the target weight; determine the fourth coordinate information of the second vertex in the target alignment result corresponding to each target region model; and perform weighted summation on each fourth coordinate information based on each target weight to obtain the second target coordinate of the second vertex.

[0019] In the above scheme, the second facial model includes multiple models. The alignment module is further used to obtain the fusion weight corresponding to each second facial model and the target model corresponding to each second facial model. Based on each fusion weight and each target model, the first facial model is deformed to obtain the fusion model.

[0020] In the above scheme, the alignment module is further configured to: determine the vertex offset between each vertex in the target model and each vertex in the first face model for each target model; perform weighted summation on each vertex offset based on the fusion weight corresponding to each target model to obtain the target offset; and superimpose the target offset onto each vertex of the first face model to obtain the fusion model.

[0021] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the model processing method provided in the embodiments of this application.

[0022] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the model processing method provided in this application when executed by a processor.

[0023] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the model processing method provided in this application.

[0024] The embodiments of this application have the following beneficial effects: In this embodiment, firstly, global rigid alignment is used to unify the first and second facial models to the same spatial coordinate system, eliminating translation and rotation errors at the macro level. Next, a first rigid alignment is performed independently on each of the first region models defined in the initial result, significantly improving the local registration accuracy of key facial regions. Then, a first smoothing process based on region weights is introduced to ensure seamless fusion of the boundaries of each independent region, effectively eliminating discontinuities and breaks in the model surface and ensuring the naturalness and continuity of the facial skin topology. Subsequently, a second rigid alignment is performed on facial appendages, reusing the same rigid transformation parameters for their corresponding regions, ensuring the relative spatial relationship between appendages and the facial epidermis, and avoiding the problems of internal component detachment, misalignment, or clipping that easily occur during complex deformations. Finally, a second smoothing process is performed on the target result after integrating appendages, further eliminating minor deformation defects accumulated during local assembly and component integration at the global level. The embodiments of this application not only take into account the high-precision alignment of macro contours and local details, but also solve the technical defects of mesh tearing and clipping of auxiliary components that are common in the processing of complex facial models. The final output is a target facial model with correct structure, high fidelity and natural and smooth surface transition, which improves the accuracy and precision of facial model processing. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the structure of the model processing system provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application; Figure 3 This is a flowchart illustrating the model processing method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the global rigid alignment process provided in the embodiments of this application; Figure 5 This is a schematic diagram of the first rigid alignment process provided in the embodiments of this application; Figure 6 This is a schematic diagram of the second rigid alignment process provided in the embodiments of this application; Figure 7 This is a schematic flowchart of the second smoothing process provided in the embodiments of this application; Figure 8 This is a schematic diagram of facial feature region division provided in an embodiment of this application; Figure 9 This is a schematic diagram of global rigid alignment provided in an embodiment of this application; Figure 10 This is a schematic diagram of the multi-region facial model alignment process provided in the embodiments of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0029] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0030] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0031] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0032] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0033] 1) Rigid alignment: used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which multiple operations are performed.

[0034] 2) Facial features: These refer to local geometric regions in a facial model that have specific anatomical semantics, including but not limited to the nose, mouth, eyes, cheekbones, and jaw. They are used to characterize the local anatomical structure features, vertex distribution range, and semantic partition boundaries of the 3D facial model during model processing.

[0035] 3) Auxiliary components: These refer to the auxiliary three-dimensional geometric structures that have spatial pose constraints with specific facial feature areas and are independent of the main facial mesh model. These include, but are not limited to, eyeball components, tooth components, tongue components, and eyelash components. During the model processing, they need to be synchronously pose-adjusted with the associated facial feature areas to ensure the relative spatial position consistency between the auxiliary structures and the main facial structure and to prevent clipping defects.

[0036] 4) Mesh data: refers to a set of numerical information used to characterize the geometric shape and topological connection relationship of a 3D model. The core includes a vertex list consisting of the 3D spatial coordinates of multiple vertices, and an index list used to define the topological connection relationship between vertices. Depending on the processing requirements, it may also include normal vectors and coordinate information to accurately construct and quantitatively describe the surface geometric contour and topological structure of the facial model in virtual 3D space.

[0037] In related technologies, parametric region blending under a standardized ecosystem is commonly used, or facial mesh alignment and fusion are achieved based on global rigid bodies and affine registration. However, this approach has at least the following problems: On the one hand, global registration schemes have obvious defects in local accuracy. They can only complete the overall uniform transformation and cannot achieve high-precision fitting for key local areas such as eyes, mouth, and nose. Moreover, the boundaries of these areas are prone to stretching and seams, making it difficult to form a natural transition and directly affecting the final presentation effect. On the other hand, the schemes lack versatility and stability. Some technologies are highly customized, resulting in poor consistency of alignment effects. Some schemes also rely too much on fixed standard topologies, which have limited adaptability to non-standard assets. They generally suffer from component drift and unstable results, making it difficult to simultaneously achieve alignment accuracy, effect consistency, processing efficiency, and asset compatibility.

[0038] This application provides a model processing method, apparatus, device, computer-readable storage medium, and computer program product, which can improve the accuracy and precision of facial model processing. The following describes exemplary applications of the electronic devices provided in this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. Exemplary applications of the electronic devices implemented as servers will be described below.

[0039] See Figure 1 , Figure 1 This is a schematic diagram of the structure of the model processing system provided in the embodiments of this application. Figure 1 The system involves a database 100, a server 200, a network 300, and a terminal 400. Terminal 400 is connected to server 200 via network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both. The trained prediction model can be stored in database 100, which can be independent of server 200 or deployed on server 200. Figure 1 The database 100 is shown as an example, independent of the server 200.

[0040] The terminal 400 provides an interactive entry point for users, receives model processing instructions triggered by users, acquires a first facial model and a second facial model to be processed, and uploads the two types of model data to the server 200 via the network 300. The server 200 receives the first and second facial models uploaded by the terminal 400, or acquires the two types of facial models from the database 100, and then performs the following processing steps in sequence: First, the first and second facial models are globally rigidly aligned to obtain an initial alignment result containing multiple facial features corresponding to first region models; then, a first rigid alignment is performed on each first region model to obtain the region alignment result corresponding to each region; then, according to the preset region weights of each region, a first smoothing process is performed on the region alignment result to eliminate splicing marks between regions; simultaneously, for each auxiliary component corresponding to the first region model, a second rigid alignment is performed using the same rigid transformation parameters as the first rigid alignment of the first region model to obtain the target alignment result corresponding to each region; finally, a second smoothing process is performed on the target alignment results corresponding to all first region models to generate the target model corresponding to the second facial model. After the model is processed, the server 200 sends the target model to the terminal 400 via the network 300, and displays the target model to the user through the display interface of the terminal 400.

[0041] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0042] Taking the server mentioned above as an example, which is the electronic device used for model processing, see [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application. Figure 2 The server 200 shown includes at least one processor 210, memory 230, and at least one network interface 220. The various components of server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 240.

[0043] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0044] The memory 230 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 230 may optionally include one or more storage devices physically located away from the processor 210.

[0045] The memory 230 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 230 described in this application embodiment is intended to include any suitable type of memory.

[0046] In some embodiments, memory 230 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0047] Operating system 231 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 232 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0048] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A model processing device 233 stored in memory 230 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: alignment module 2331 and smoothing module 2332. These modules are logically related and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0049] The model processing method provided in the embodiments of this application will be described below. As mentioned above, the electronic device implementing the model processing method of the embodiments of this application can be a terminal, a server, or a combination of both. The model processing method provided in the embodiments of this application will be described below with reference to exemplary applications and implementations of the server provided in the embodiments of this application. Therefore, the executing entity of each step will not be repeated below.

[0050] It should be noted that the model processing examples below are based on facial models. Those skilled in the art can apply the model processing methods provided in the embodiments of this application to alignment, fusion and processing scenarios of other types of models, including but not limited to full-body human models, virtual animal models, virtual prop models, industrial 3D models, etc., and can achieve similar high-precision alignment, refined regional processing and smooth fusion effects.

[0051] Based on this, the technical solutions provided in this application can be widely applied to various application scenarios. For example, the technical solutions provided in this application can be applied to game development and interactive entertainment scenarios. For instance, in the process of highly customizable character creation or batch generation of non-player character assets in games, the model processing method provided in this application can be used to process personalized facial feature data, accurately mapping and smoothly integrating it into the common base mesh of the game engine. This effectively avoids problems such as clipping, geometric distortion, or animation breakdown when dealing with complex facial geometric differences, improving the fidelity and animation expressiveness of game characters. The model processing method provided in this application can also be applied to film and television special effects production scenarios. For example, in the production of digital stunt doubles or special effects characters, it is often necessary to match the high-precision facial scan mesh of a live actor with the base mesh of a specific special effects character. Through the model processing method provided in this application, when dealing with significant differences in spatial pose and anatomical structure between the two, the accuracy of global alignment and the fineness of local deformation can be balanced, ensuring that the final generated special effects character face retains the actor's performance characteristics while naturally integrating into the film-grade rendering pipeline. The model processing method provided in this application can also be applied to scenarios such as virtual reality and augmented reality. For example, in immersive virtual meetings, virtual makeup try-on, virtual try-on, or holographic communication applications, the model processing method provided in this application can be used to quickly align and fuse real facial geometric features into a standardized avatar model in virtual space, ensuring that the digital image in the virtual environment is not distorted and providing a more natural and highly immersive visual interaction experience. The model processing method provided in this application can also be applied to personalized virtual human or digital human construction scenarios. For example, when creating virtual anchors, virtual customer service representatives, or digital idols, the model processing method provided in this application can be used to align and fuse 3D facial data obtained from human scanning with high precision into a digital human basic mesh model with a standard topology. This can not only highly restore the local anatomical features of real people, such as facial features and cheeks, but also ensure that the fused model has a standardized wiring structure, thereby perfectly supporting subsequent facial skeleton binding, fusion deformation generation, and expression driving.

[0052] The scope of protection of the embodiments of this application is not limited to the specific application scenario of facial models. Any extension, modification, and adaptation based on the technical solution of this application are within the scope of protection of this application. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. Those skilled in the art can flexibly adjust the application scenario according to actual needs in actual application, and all such modifications, equivalent substitutions, and improvements should be included within the scope of protection of this application.

[0053] See Figure 3 , Figure 3This is a flowchart illustrating the model processing method provided in the embodiments of this application, which will be combined with... Figure 3 The steps shown are explained.

[0054] In step 101, a first facial model and a second facial model to be processed are obtained, and the first facial model and the second facial model are globally rigidly aligned to obtain an initial alignment result. The initial alignment result includes a first region model corresponding to each of the multiple facial features.

[0055] Here, the first facial model refers to the 3D facial model used as the alignment reference, the second facial model refers to the facial mesh model that needs to be aligned with the first facial model, and global rigid alignment refers to the overall rotation, translation, and scaling transformation of the two facial models in 3D space, without changing the geometry of the models themselves or the relative positions between vertices during the transformation. The initial alignment result refers to the intermediate processing result obtained after global rigid alignment. Facial features refer to the key anatomical structures and corresponding regions of the face, such as the nose, eyes, mouth, and cheekbones. The first region model refers to the facial sub-models separated from the first facial model in the initial alignment result, each corresponding to a single facial feature. In this step, the reference model and the model to be processed are first obtained, and then the two models are matched in 3D space through global rigid transformation. At the same time, the matched model is split into multiple independent region sub-models according to facial features to prepare for subsequent local fine-tuning processing. For example, if the first facial model is a standard neutral 3D facial model and the second facial model is a personalized 3D facial model, the two models are first imported, and then global rigid alignment is used to make the personalized facial model roughly match the standard neutral facial model in terms of spatial position and overall scale. At the same time, the first facial model will be split into multiple first region models such as nose model, eye model, and mouth model to complete the initial alignment process.

[0056] In some embodiments, see Figure 4 The step 101, "performing global rigid alignment of the first facial model and the second facial model to obtain the initial alignment result," can be achieved through steps 1011 to 1014, including: In step 1011, the region mask corresponding to each facial feature is determined.

[0057] Here, a region mask refers to a numerically labeled matrix used to define the region range corresponding to different facial features in the facial model. It corresponds one-to-one with the number of vertices in the facial model, using different numerical labels to distinguish whether a vertex belongs to the region corresponding to a particular facial feature. In this step, a unique region mask is generated for each facial feature, clearly defining the vertex range and region boundaries of each facial feature in the first facial model, providing a precise numerical basis for subsequent model region division. For example, for the facial feature of the nose, a corresponding nose region mask is generated. The mask labels vertices belonging to the nose with a specific value and vertices not belonging to the nose with another value. This mask clearly defines the vertex range of the nose in the first facial model. Similarly, corresponding region masks are generated for other facial features such as the eyes and mouth.

[0058] In some embodiments, "determining the region mask corresponding to each facial feature" in step 1011 can be implemented through the following process, including: Determine the first grid data corresponding to the first facial model, as well as the semantic partition information of the first facial model; for each facial feature, based on the semantic partition information, determine the target vertex from the first grid data, and the target vertex is located in the second region model corresponding to the facial feature; set the mask value of the target vertex to the first value, and set the mask values ​​of the first other vertices besides the target vertex to the second value to obtain the region mask corresponding to the facial feature.

[0059] Here, the first mesh data refers to the core data representing the geometry of the first facial model, including the 3D coordinates of all vertices and the topological connections between them. Semantic partitioning information refers to the annotation information that semantically divides the first facial model according to facial anatomical features, clarifying the approximate area of ​​each facial feature within the model. The target vertex refers to the vertex belonging to the region corresponding to a specific facial feature. The second region model refers to the original sub-model corresponding to a specific facial feature in the first facial model. The mask value refers to the numerical value in the region mask used to identify the state of a vertex. The first value and the second value refer to two different mask values ​​used to distinguish the target vertex from other vertices in the first facial model. The other vertices refer to all vertices in the first facial model except the target vertex. This step is the specific process of generating the region mask. The core is to accurately select the target vertices corresponding to each facial feature based on the mesh data and semantic partitioning information of the first facial model, and then construct the region mask by setting different mask values. For example, first obtain the mesh data of a standard neutral face model, which contains the 3D coordinates and connectivity of all vertices of the model. At the same time, obtain the semantic partition information of the model, which marks the area of ​​the eyes in the model. Based on this information, select all vertices belonging to the eyes from the mesh data as target vertices, set the mask value of these target vertices to 1, and set the mask value of the remaining vertices in the model to 0. In this way, the region mask corresponding to the eyes is obtained.

[0060] In this embodiment, the mesh data and semantic partitioning information of the first facial model are first obtained. Then, for each facial feature, target vertices within the corresponding region are selected from the mesh data. By setting different mask values ​​for the target vertices and other vertices, region masks corresponding to each facial feature are generated. In this way, the precise definition of facial feature regions is achieved based on semantic partitioning and vertex masks, providing a clear basis for subsequent model region division.

[0061] In step 1012, the first facial model is segmented based on the region mask corresponding to each facial feature to obtain the segmented first facial model.

[0062] Here, the segmentation process refers to the process of dividing the complete first facial model into multiple independent sub-models corresponding to each facial feature based on the region masks of each facial feature. The segmented first facial model refers to the overall first facial model composed of multiple sub-models corresponding to facial features after the segmentation process. In this step, the generated facial feature region masks are used to segment the complete first facial model, so that each facial feature corresponds to an independent sub-model, achieving fine-grained region segmentation of the model. For example, using the generated region masks for the nose, eyes, mouth, and cheekbones, the nose sub-model, eye sub-model, mouth sub-model, and cheekbone sub-model are extracted from the standard neutral facial model. These sub-models together form the segmented standard neutral facial model, and the vertex range of each sub-model is precisely defined by the corresponding region mask.

[0063] In step 1013, global transformation parameters between the first facial model and the second facial model are determined.

[0064] Here, global transformation parameters refer to the parameters used to achieve global rigid alignment of the first and second facial models. These include one or more of rotation parameters, translation parameters, and uniform scale parameters. Rotation parameters represent the angle information of the model's rotation around the coordinate axes in 3D space; translation parameters represent the distance information of the model's movement along the coordinate axes in 3D space; and uniform scale parameters represent the scale information of the overall scaling of the model. Solving for the global transformation parameters that enable the first and second facial models to achieve overall positional matching in 3D space provides a numerical basis for subsequent global rigid transformations. For example, by calculating and solving for a set of parameters where the rotation parameters are 5 degrees around the X-axis and 3 degrees around the Y-axis, the translation parameters are 2 units along the X-axis and 1 unit along the Z-axis, and the uniform scale parameter is 1.05, this set of parameters represents the global transformation parameters that enable the first and second facial models to achieve global positional matching.

[0065] In some embodiments, the "determining global transformation parameters between the first facial model and the second facial model" in step 1013 can be achieved through the following process, including: Determine the first mesh data corresponding to the first facial model and the second mesh data corresponding to the second facial model; based on the first mesh data and the second mesh data, determine the first mapping relationship between the first facial model and the second facial model; based on the first mapping relationship, determine the global transformation parameters, which include at least one of the following: rotation parameters, translation parameters, and uniform scale parameters.

[0066] Here, the second mesh data refers to the core data characterizing the geometry of the second facial model, including the 3D coordinates of all vertices and the topological connections between them. The first mapping relationship refers to the corresponding association between the vertices of the two facial models, established based on the first and second mesh data. First, the mesh data of the two facial models is extracted. Then, the first mapping relationship between the vertices is established based on the mesh data. Finally, the corresponding global transformation parameters are solved using this mapping relationship. For example, the first mesh data of a standard neutral facial model and the second mesh data of a personalized facial model are extracted first. Then, through feature point matching and other methods, a one-to-one first mapping relationship is established for the key vertices of the two models. Based on this relationship, the rotation parameters, translation parameters, and uniform scale parameters that enable global alignment of the two models are calculated using an algorithm.

[0067] In this embodiment, the mesh data corresponding to the first and second facial models are first extracted. Then, a first mapping relationship between the two models is established based on the two sets of mesh data. Finally, the global transformation parameters, including at least one of rotation, translation, and uniform scaling, are solved based on this mapping relationship. In this way, the relationship between the models is established at the mesh data level, ensuring that the solved global transformation parameters can accurately adapt to the global alignment requirements of the two facial models.

[0068] In step 1014, based on the global transformation parameters, a global rigid transformation is performed on the segmented first or second facial model to obtain the initial alignment result.

[0069] Here, global rigid transformation refers to performing overall rotation, translation, and scaling operations on the segmented first or second facial model based on global transformation parameters, while maintaining the model's own geometry and the relative positions between vertices. The solved global transformation parameters are applied to the segmented first or second facial model, and the global rigid transformation allows the two models to achieve overall positional matching in 3D space. The matching result is then used as the initial alignment result, providing a basis for subsequent local processing. For example, applying the solved global transformation parameters to the segmented first facial model yields a first region model containing multiple regions such as the nose and eyes. Then, this model is rotated, translated, and scaled overall to roughly match the second facial model in spatial position and overall scale, obtaining the initial alignment result.

[0070] In this embodiment, a corresponding region mask is matched for each facial feature. Based on the mask, the region of the first facial model is divided. Then, the global transformation parameters between the first and second facial models are solved. Finally, a global rigid transformation is performed on the divided first or second facial model using these parameters to obtain the initial alignment result containing the first region model corresponding to each facial feature. This achieves accurate region division and preliminary global alignment based on facial features, improving the refinement and accuracy of the model processing.

[0071] See also Figure 3 In step 102, for each first region model, a first rigid alignment is performed on the first region model to obtain the region alignment result corresponding to the first region model.

[0072] Here, the first rigid alignment refers to the local rigid alignment processing performed on each independent first region model. Only the sub-model corresponding to a single facial feature is subjected to stretching, rotation, translation, and scaling transformations. The region alignment result refers to the local processing result obtained after the first rigid alignment of a single first region model. Each first region model separated from the initial alignment result is then subjected to local refined rigid alignment, ensuring that the sub-model corresponding to each facial feature can accurately match the corresponding sub-model of the baseline model, thus improving the alignment accuracy of local regions.

[0073] In some embodiments, see Figure 5 The step 102, "performing a first rigid alignment on the first region model to obtain the region alignment result corresponding to the first region model," can be achieved through steps 1021 to 1023, including: In step 1021, a third region model corresponding to the facial features in the second facial model is determined.

[0074] Here, the third region model refers to the sub-model in the second facial model corresponding to a certain facial feature. It is the corresponding matching object of the first region model in the second facial model. The third region model corresponding to each facial feature is extracted from the second facial model, and a precise local alignment matching object is determined for each first region model. For example, the nose sub-model extracted from the second facial model is the third region model corresponding to the nose first region model in the initial alignment result. Similarly, a corresponding third region model can be determined for each first region model, such as the eyes and mouth.

[0075] In step 1022, the rigid transformation parameters between the first region model and the third region model are determined.

[0076] Here, rigid transformation parameters refer to the parameters used to achieve local rigid alignment between a single first region model and its corresponding third region model. Similar to global transformation parameters, they include parameters related to rotation, translation, and scaling, and may also include parameters such as stretching and deformation, suitable for alignment processing of local region models. In this step, specific local rigid transformation parameters are solved for each first region model and its corresponding third region model, enabling precise position and shape matching of the local region models. For example, for the first region model of the nose and its corresponding third region model of the nose, the algorithm solves for rigid transformation parameters that are only applicable to the alignment of this nose model, allowing the two nose models to achieve fine-grained local alignment.

[0077] In some embodiments, the "determining the rigid transformation parameters between the first region model and the third region model" in step 1022 can be achieved through the following process, including: From the first mesh data corresponding to the first face model, determine the first vertex data corresponding to the first region model; from the second mesh data corresponding to the second face model, determine the second vertex data corresponding to the third region model; based on the first vertex data and the second vertex data, determine the rigid transformation parameters.

[0078] Here, the first vertex data refers to the vertex-related data belonging to a certain first region model in the first mesh data, including the 3D coordinates of all vertices in that region. The second vertex data refers to the vertex-related data belonging to a certain third region model in the second mesh data, including the 3D coordinates of all vertices in that region. This step is the specific solution process for the local rigid transformation parameters. The core is to extract the vertex data of the corresponding local regions from the mesh data of the two facial models, and then solve for the specific rigid transformation parameters based on these local vertex data. For example, the coordinates of all vertices of the first region model of the nose are extracted from the first mesh data of the standard neutral facial model as the first vertex data, and the coordinates of all vertices of the third region model of the nose are extracted from the second mesh data of the personalized facial model as the second vertex data. Based on these two sets of vertex data, the rigid transformation parameters that enable the two nose models to achieve local precise alignment are calculated through algorithms.

[0079] In this embodiment, vertex data corresponding to the first and third region models are extracted from the mesh data of the first and second facial models, respectively. Then, the rigid transformation parameters between the two region models are directly solved based on the two sets of vertex data. In this way, the parameter solution is performed by focusing on the vertex data within the region, making the rigid transformation parameters more closely match the alignment requirements of the local region, thereby improving the accuracy and rationality of determining the rigid transformation parameters.

[0080] In some embodiments, the "determining rigid transformation parameters based on the first vertex data and the second vertex data" in the above embodiments can be implemented through the following process, including: A second mapping relationship is determined between the first vertex data and the second vertex data; based on the second mapping relationship, an objective function is constructed to characterize the alignment error between the first region model and the third region model; by solving the objective function, the rigid transformation parameters are obtained.

[0081] Here, the second mapping relationship refers to the corresponding association between the vertices of the first and third region models, established based on the first and second vertex data. Alignment error refers to the positional deviation between corresponding vertices in the first and third region models when they are not aligned. The objective function is a mathematical function used to quantify the overall alignment error between the first and third region models. In this step, the local rigid transformation parameters are determined through optimization. The core is to first establish the second mapping relationship between the vertices of the two local region models, then construct the objective function to quantify the alignment error, and finally obtain the optimal rigid transformation parameters by solving for the parameters that minimize the objective function value. For example, first, a one-to-one second mapping relationship is established for the key vertices of the first and second vertices of the nose. Then, an objective function is constructed with the sum of the squares of the positional deviations of the corresponding vertices of the two nose models as its core. The parameters that minimize this objective function value are then solved using algorithms such as gradient descent. These parameters are the rigid transformation parameters that minimize the alignment error between the two nose models.

[0082] In this embodiment, a second mapping relationship is first established between the first and second vertex data. Then, an objective function that reflects the alignment error between the two region models is constructed based on this relationship. Finally, the rigid transformation parameters are obtained by solving the objective function. In this way, by minimizing the alignment error as the objective of parameter solving, the alignment deviation between region models is effectively reduced, thereby improving the accuracy of local model alignment.

[0083] In step 1023, the first region model is subjected to rigid transformation based on the rigid transformation parameters to obtain the region alignment result.

[0084] Here, rigid transformation processing refers to operations such as local rotation, translation, scaling, stretching, and deformation performed on a single first region model based on local rigid transformation parameters. During the operation, the geometry of the region model itself and the relative positions between vertices can remain unchanged or be adaptively deformed. In this step, the solved local rigid transformation parameters are applied to the corresponding first region model, and the first region model is precisely aligned with the third region model through local rigid transformation, resulting in the corresponding region alignment result. For example, the solved nose rigid transformation parameters are applied to the first region model of the nose, and local transformation processing is performed on this model to achieve precise position and shape matching with the third region model of the nose. The nose model obtained after matching is the corresponding region alignment result for the nose.

[0085] In this embodiment, a third region model matching the facial features corresponding to the first region model in the second facial model is first found. Then, the rigid transformation parameters between the two region models are solved. Finally, the first region model is rigidly transformed using these parameters to obtain the region alignment result. In this way, refined rigid alignment at the facial feature region level is achieved, improving the alignment accuracy of local regions.

[0086] See also Figure 3 In step 103, based on the region weights corresponding to the first region model, the region alignment result is subjected to a first smoothing process to obtain a smoothed region alignment result.

[0087] Here, region weights refer to the weight values ​​set for each vertex in the first region model, used to adjust the degree of vertices blending during the smoothing process. The first smoothing process refers to the local smoothing and blending process performed on the alignment result of a single region. By weighting the vertex weights, the vertex coordinates within the region achieve a natural transition. The smoothed region alignment result refers to the local processing result obtained after the first smoothing process on the alignment result of a single region. The core of this step is to use the region weights set for the first region model to perform local smoothing on the region alignment result after the first rigid alignment, allowing the vertex coordinates within the region to achieve a natural transition, avoiding coordinate abrupt changes, and improving the morphological coherence of the local region model. For example, for the region alignment result of the nose, the region weights of the first region model of the nose are used to perform a weighted sum of the coordinates of each vertex of the nose after the initial alignment and the region alignment, obtaining the smoothed coordinates of each vertex. The nose model composed of these coordinates is the smoothed nose region alignment result.

[0088] In some embodiments, the first region model includes multiple first vertices, and the region weights include the vertex weights corresponding to each first vertex. Step 103, "based on the region weights corresponding to the first region model, performing a first smoothing process on the region alignment result to obtain a smoothed region alignment result," can be implemented through the following process: For each first vertex, determine the first coordinate information of the first vertex in the initial alignment result and the second coordinate information of the first vertex in the region alignment result; based on the vertex weight corresponding to the first vertex, perform weighted summation on the first coordinate information and the second coordinate information to obtain the first target coordinates corresponding to the first vertex; based on the first target coordinates corresponding to each first vertex, construct the smoothed region alignment result.

[0089] Here, "first vertex" refers to all vertices contained in the first region model; "vertex weight" refers to the weight value individually set for each first vertex in the region weight; "first coordinate information" refers to the 3D coordinate data of the first vertex in the initial alignment result; "second coordinate information" refers to the 3D coordinate data of the first vertex in the region alignment result; "weighted summation" refers to the process of weighting and summing the first and second coordinate information according to the vertex weight; and "first target coordinates" refers to the smoothed 3D coordinate data of the first vertex after weighted summation. This step is the specific implementation process of the first smoothing process. Its core is to obtain the coordinate information of each first vertex in the two different processing stages, then perform weighted fusion using vertex weights to obtain the smoothed target coordinates, and finally construct the smoothed region alignment result based on all target coordinates.

[0090] In this embodiment, for each vertex in the first region model, its coordinate information in the initial alignment and region alignment results is obtained. Then, the two coordinates are weighted and summed according to the weight of the vertex to obtain the target coordinates of the vertex. Finally, a smoothed region alignment result is constructed based on the target coordinates of all vertices. In this way, the region alignment result is smoothed by weighted fusion of vertex weights, avoiding abrupt changes in vertex coordinates within the region. This improves the morphological coherence and visual naturalness of the aligned local regions of the facial model, effectively eliminating the vertex position jump problem that may occur within the region after rigid alignment, making the geometric transition of the facial feature region smoother, and improving the overall alignment quality of the model.

[0091] In some embodiments, the region weights in the above embodiments can be determined by the following process, including: From multiple first vertices, determine the center vertex corresponding to the first region model, and determine the first vertices other than the center vertex as second other vertices; determine the vertex weight corresponding to the center vertex as the first value; for each second other vertex, determine the distance information between the second other vertex and the center vertex; based on the distance information, determine the vertex weight corresponding to the second other vertex, wherein the vertex weight corresponding to the second other vertex is negatively correlated with the distance information, and the vertex weight corresponding to the second other vertex is less than the first value.

[0092] Here, the central vertex refers to the vertex located at the core of the first region model, corresponding to the geometric center or feature center of the region model. Other vertices refer to the remaining vertices in the first region model excluding the central vertex. Distance information refers to the 3D straight-line distance between the other vertices and the central vertex. Negative correlation means that the weight of the other vertices decreases as their distance from the central vertex increases. This step is the specific process of determining the region weights. The core is to determine the central vertex of the first region model and set its highest weight, then set gradient weight values ​​based on the distances of other vertices to the central vertex, constructing a gradient weight system based on vertex positions. The first value is usually set to 1. For example, for the first region model of the nose, the vertex at the geometric center of the nose is determined as the central vertex, and its weight is set to 1. Then, the 3D spatial distance between each other vertex of the nose and the central vertex is calculated. The vertex 1 unit away from the central vertex has a weight of 0.8, the vertex 2 units away has a weight of 0.6, and the weight decreases with increasing distance. Corresponding weight values ​​are then set for all other vertices.

[0093] In this embodiment, the central vertex of the first region model is first determined and assigned a weight with a first value. Then, the weights of other vertices are determined based on their distance from the central vertex, with the weight decreasing as the distance increases and negatively correlated with the distance. In this way, by constructing a gradient weight system based on vertex positions, the smoothing effect within the region can better match the spatial distribution of facial features.

[0094] See also Figure 3 In step 104, a second rigid alignment is performed on the subordinate components corresponding to each first region model to obtain the target alignment result corresponding to the first region model. The rigid transformation parameters used for the first rigid alignment and the second rigid alignment are the same.

[0095] Here, "attached components" refers to facial geometric structures that match the facial feature regions. These are sub-models independent of the main facial mesh but spatially constrained by it, such as the eyeball component corresponding to the eyes, the tooth component corresponding to the mouth, and the lip component. "Second rigid alignment" refers to the rigid alignment processing performed on the attached components, sharing the same rigid transformation parameters as the first rigid alignment of the corresponding first region model. "Target alignment result" refers to the local processing result obtained after integrating the first region model (after first smoothing) with the corresponding attached components that have undergone second rigid alignment. The purpose of this step is to perform second rigid alignment on each attached component corresponding to the first region model using the same rigid transformation parameters as the first rigid alignment of that region, allowing the attached components to transform synchronously with the corresponding facial feature regions, and then integrating the two to obtain the target alignment result. For example, for the first region model of the eyes, its corresponding attached component is the eyeball component. The rigid transformation parameters of the first rigid alignment of the eyes are applied to the eyeball component for second rigid alignment, allowing the eyeball component to transform synchronously with the eye region model. Then, the smoothed eye region model is integrated with the aligned eyeball component to obtain the target alignment result for the eyes.

[0096] In some embodiments, see Figure 6 Step 104, "perform a second rigid alignment on the subordinate components corresponding to each first region model to obtain the target alignment result corresponding to the first region model", can be implemented through steps 1041 to 1043, including: In step 1041, for each auxiliary component, the rigid transformation parameters used in the first rigid alignment of the first region model corresponding to the auxiliary component are determined.

[0097] Here, the first rigid transformation parameters of the corresponding first region model are matched for each auxiliary component to ensure that the alignment transformation of the auxiliary component is consistent with the corresponding facial feature region. For example, the eyeball component corresponds to the first region model of the eye, so the rigid transformation parameters used by the first region model of the eye during the first rigid alignment are directly extracted as the parameter basis for the second rigid alignment of the eyeball component. The tooth component corresponds to the first region model of the mouth, so the rigid transformation parameters of the first rigid alignment of the mouth are extracted as the alignment parameters of the tooth component.

[0098] In step 1042, the constraints of the auxiliary components are determined, and the constraints are used to constrain the degree of deformation of the auxiliary components.

[0099] Here, constraints refer to the deformation limit rules set for auxiliary components. These rules prevent excessive deformation of the auxiliary components during rigid alignment, ensuring the integrity of their original structure and form. The degree of deformation refers to the extent to which the geometry and relative positions between vertices of the auxiliary component change during the transformation. The core of this step is to formulate specific deformation constraints for each auxiliary component, clearly defining its upper limit of deformation during the second rigid alignment process, thus avoiding excessive deformation and structural damage. For example, constraints are set for the eyeball component, stipulating that the relative position change between vertices during the transformation cannot exceed 0.1 units; constraints are set for the tooth component, stipulating that its overall scaling ratio cannot exceed 1.1 times, thereby limiting the degree of deformation of the auxiliary components.

[0100] In some embodiments, the "determining the constraints of the auxiliary components" in step 1042 can be achieved through the following process, including: Determine the initial relative position information between the vertices of each component in the auxiliary component; based on each initial relative position information, determine the deformation threshold corresponding to the auxiliary component; construct constraints based on the deformation threshold, which are used to constrain that the relative position change between the vertices of each component is less than or equal to the deformation threshold during the second rigid alignment.

[0101] Here, component vertices refer to all vertices contained in the subordinate component. Initial relative position information refers to the 3D spatial relative positions and distances between the vertices of each component when the subordinate component has not undergone any transformation. Deformation threshold refers to the maximum allowable value for the relative position change between vertices of the subordinate component, which is the critical value for measuring the degree of deformation of the subordinate component. This step is the specific construction process of the subordinate component constraints. First, the initial relative position information of the subordinate component is extracted, then a reasonable deformation threshold is determined based on this information, and finally, the constraints are constructed with the deformation threshold as the core. For example, first, the initial relative position information between all vertices of the eyeball component is obtained, including the 3D spatial distance between any two vertices. Then, based on the structural characteristics of the eyeball component, the deformation threshold for the relative position change between each vertex is set to 0.1 units. Constraints are constructed based on this threshold, stipulating that during the second rigid alignment of the eyeball component, the relative position change between all component vertices cannot exceed this threshold.

[0102] In this embodiment, the initial relative position information between the vertices of the auxiliary component is first extracted. Then, the deformation threshold of the auxiliary component is determined based on this information. Finally, constraints are constructed based on the threshold to limit the range of change in the relative position of the vertices during the alignment process. In this way, by formulating deformation constraints at the level of relative vertex position, the degree of deformation of the auxiliary component can be precisely controlled, ensuring its original structural characteristics and preventing excessive deformation of the auxiliary component.

[0103] In step 1043, based on the rigid transformation parameters and constraints, the auxiliary components are subjected to a second rigid alignment to obtain the target alignment result corresponding to the first region model.

[0104] Here, the matched rigid transformation parameters are combined with the constructed constraints to perform a second rigid alignment on the auxiliary components. This ensures that the auxiliary components transform synchronously with the corresponding first region model while strictly controlling their deformation degree. The aligned auxiliary components are then integrated with the smoothed first region model to obtain the target alignment result. For example, the rigid transformation parameters of the eye are applied to the eyeball component, and the deformation constraints of the eyeball component are followed to perform a second rigid alignment. This ensures that the eyeball component does not undergo excessive deformation while transforming synchronously. The aligned eyeball component is then integrated with the smoothed first region model of the eye to obtain the target alignment result for the eye.

[0105] In this embodiment, a first rigid transformation parameter is matched to the corresponding first region model for each auxiliary component. Simultaneously, conditions constraining the deformation degree of the auxiliary components are set. Then, a second rigid alignment is performed on the auxiliary components based on the parameters and conditions to obtain the target alignment result. In this way, by allowing the auxiliary components to transform synchronously with the corresponding region model, excessive deformation of the auxiliary components can be avoided, ensuring the morphological integrity of the auxiliary components.

[0106] In step 105, the target alignment result corresponding to each first region model is subjected to a second smoothing process to obtain the target model corresponding to the second face model.

[0107] Here, the second smoothing process refers to the global smoothing and fusion processing performed on the target alignment results of all first region models. This ensures a natural connection between the target alignment results of each region at the overall level, eliminating stitching artifacts between regions. The target model refers to the final 3D facial mesh model obtained after all processing steps of the second facial model; it is the output of the model processing workflow. Global smoothing and fusion processing is performed on the target alignment results of all facial feature regions to achieve a natural connection between regions, eliminating stitching artifacts and ultimately obtaining a target model with a coherent shape and complete structure. For example, the target alignment results of all regions such as the eyes, nose, mouth, and cheekbones are integrated, and then the second smoothing process ensures a natural transition at the junctions of each region, eliminating discontinuities and abrupt changes between regions, ultimately resulting in a personalized facial target model with a complete shape and natural transitions.

[0108] In some embodiments, see Figure 7 The step 105, "performing a second smoothing process on the target alignment result corresponding to each first region model to obtain the target model corresponding to the second face model," can be implemented through steps 1051 to 1054, including: In step 1051, for each target alignment result, the third coordinate information of each second vertex included in the target alignment result is determined.

[0109] Here, the second vertex refers to all vertices included in the target alignment results, covering the vertices of each first region model and smoothed auxiliary components, and the third coordinate information refers to the three-dimensional coordinate data of the second vertex in its respective target alignment result.

[0110] In step 1052, for each second vertex, the coverage state between the second vertex and the alignment results of each other target is determined.

[0111] Here, "other target alignment results" refers to all target alignment results except for the target alignment result to which a certain second vertex belongs. Coverage status refers to the state used to characterize whether a single second vertex simultaneously belongs to multiple target alignment results, divided into "covered" and "uncovered." "Covered" means the second vertex belongs to two or more target alignment results simultaneously, representing an overlapping vertex between regions. "Uncovered" means the second vertex belongs to only one target alignment result, representing an independent vertex within that region. The core of this step is to determine whether each second vertex is an overlapping vertex between regions and to ascertain its corresponding coverage status, providing a basis for subsequent differential coordinate processing. For example, a vertex at the nasal wing belongs to both the nasal and cheekbone target alignment results, so its coverage status is "covered." Conversely, a vertex at the nasal tip belongs only to the nasal target alignment result, so its coverage status is "uncovered."

[0112] In step 1053, the smoothed second target coordinates of the second vertex are determined based on the coverage state and the third coordinate information.

[0113] Here, the second target coordinates refer to the final 3D coordinate data obtained after the second vertex undergoes the second smoothing process. This is the core coordinate basis for constructing the target model. The core of this step is to determine the smoothed second target coordinates based on the coverage status of each second vertex, combined with its third coordinate information, using a differentiated processing method. This allows overlapping vertices between regions to achieve a natural transition, while independent vertices within a region retain their original coordinates. For example, for the uncovered tip of the nose vertex, its third coordinate information is directly used as the second target coordinates; for the covered alar vertices, the smoothed second target coordinates are calculated by combining its third coordinate information from the alignment results of the nose and cheekbone regions, as well as the weights of the two regions.

[0114] In some embodiments, the step 1053, "determining the smoothed second target coordinates of the second vertex based on the coverage state and the third coordinate information," can be implemented through the following process, including: When the coverage state indicates that the second vertex is not covered by other target alignment results, the third coordinate information is determined as the second target coordinate; when the coverage state indicates that the second vertex is covered by other target alignment results, the first region model corresponding to each target alignment result that covers the second vertex is determined as the target region model; based on the region weight corresponding to each target region model, the second target coordinate of the second vertex is determined.

[0115] Here, the target region model refers to the first region model that covers all target alignment results corresponding to a covered second vertex; it is the region model to which the overlapping vertex belongs. This step is the specific process of determining the second target coordinates. The core is to adopt a differentiated coordinate determination strategy for uncovered and covered second vertices. Uncovered vertices directly use their original coordinates, while covered vertices calculate new coordinates based on the weights of all target region models to which they belong. For example, if a vertex belongs only to the eye target alignment result and is not covered by other regions, its third coordinate in the eye target alignment result is directly used as the second target coordinate. If a vertex belongs to both the nose and cheekbone target alignment results and is covered by both regions, then the first region models of the nose and cheekbone are determined as the target region models. Then, based on the region weights of the nose and cheekbone, the third coordinate of the vertex in the target alignment results of the two regions is weighted and calculated to obtain the second target coordinate.

[0116] In this embodiment, different coordinate determination strategies are adopted based on whether a vertex is covered by the alignment results of other targets. When not covered, the original coordinates are directly used; when covered, the target region model covering the vertex is determined, and then the vertex target coordinates are determined based on the region weights of these models. In this way, differentiated and precise processing is performed for different coverage states of vertices, which effectively improves the naturalness of coordinate transitions at the fusion of multiple regions and makes the connection of vertex positions between regions smoother.

[0117] In some embodiments, the "determining the second target coordinates of the second vertex based on the region weights corresponding to each target region model" in the above embodiments can be implemented through the following process, including: From the region weights corresponding to each target region model, determine the vertex weights corresponding to the second vertex; normalize each vertex weight corresponding to the second vertex to obtain the target weights; determine the fourth coordinate information of the second vertex in the target alignment result corresponding to each target region model; based on each target weight, perform weighted summation on each fourth coordinate information to obtain the second target coordinates of the second vertex.

[0118] Here, normalization refers to the standardization process of processing multiple vertex weights so that their sum is 1. Target weight refers to the normalized weight value obtained after normalization of the vertex weight corresponding to the second vertex. The fourth coordinate information refers to the 3D coordinate data of the second vertex in the target alignment result corresponding to each target region model. In this step, the vertex weights of the covered vertices in each target region model are first extracted and normalized, and then combined with their coordinate information in each region for weighted summation to obtain the smoothed target coordinates. For example, a certain overlapping vertex has a vertex weight of 1 in the nose target region model and a vertex weight of 1 in the cheekbone target region model. After normalization, the target weights are 0.5 and 0.5 respectively. The fourth coordinate of this vertex in the nose is (2, 3, 4), and the fourth coordinate of the cheekbone is (4, 3, 6). The second target coordinates are calculated by weighted summation as (2×0.5+4×0.5, 3×0.5+3×0.5, 4×0.5+6×0.5), which is (3, 3, 5).

[0119] In this embodiment, the vertex weights corresponding to each target region model covering the vertex are first extracted. After normalizing the weights, the target coordinates of the vertex are calculated by weighted summation, combining the coordinate information of the vertex in the target alignment results of each model. In this way, accurate coordinate fusion of vertices covering multiple regions is achieved based on normalized weights, effectively ensuring the rationality of the fused coordinates, while significantly improving the smoothness of coordinates at the junctions of multiple regions.

[0120] In step 1054, a target model is constructed based on the second target coordinates corresponding to each vertex.

[0121] Here, the second target coordinates obtained after the second smoothing process of all the second vertices are integrated, and combined with the topological connection relationship of the facial model, to construct a final target model with a coherent shape and complete structure. This is the final implementation step of the second smoothing process. For example, by integrating the second target coordinates of all vertices of the face, maintaining the original topological connection relationship between the vertices, and using a 3D modeling algorithm to transform these coordinate data into a 3D mesh model, this model is the final target model corresponding to the second facial model.

[0122] In this embodiment, the coordinate information of all vertices in each target alignment result is first obtained. Then, the coverage state of each vertex with other target alignment results is determined. Next, the target coordinates after vertex smoothing are determined by combining the coverage state and coordinate information. Finally, the final target model is constructed based on the target coordinates of all vertices. In this way, the overall smooth fusion of multi-region target alignment results is achieved from the perspective of vertex coverage, effectively improving the coherence and integrity of the overall shape of the facial model, making the connection between various facial feature regions more natural, and greatly improving the overall alignment fusion effect and visual smoothness of the model.

[0123] In this embodiment, firstly, global rigid alignment is used to unify the first and second facial models to the same spatial coordinate system, eliminating translation and rotation errors at the macro level. Next, a first rigid alignment is performed independently on each of the first region models defined in the initial result, significantly improving the local registration accuracy of key facial regions. Then, a first smoothing process based on region weights is introduced to ensure seamless fusion of the boundaries of each independent region, effectively eliminating discontinuities and breaks in the model surface and ensuring the naturalness and continuity of the facial skin topology. Subsequently, a second rigid alignment is performed on facial appendages, reusing the same rigid transformation parameters for their corresponding regions, ensuring the relative spatial relationship between appendages and the facial epidermis, and avoiding the problems of internal component detachment, misalignment, or clipping that easily occur during complex deformations. Finally, a second smoothing process is performed on the target result after integrating appendages, further eliminating minor deformation defects accumulated during local assembly and component integration at the global level. The embodiments of this application not only take into account the high-precision alignment of macro contours and local details, but also cleverly solve the common problems of mesh tearing and attachment component clipping in the processing of complex facial models. The final output is a target facial model with correct structure, high fidelity and extremely natural and smooth surface transition, which improves the accuracy and precision of facial model processing.

[0124] In some embodiments, the second facial model includes multiple models. The method in this application embodiment further includes the following steps: Obtain the fusion weights corresponding to each second facial model and the target model corresponding to each second facial model; based on each fusion weight and each target model, perform deformation processing on the first facial model to obtain the fusion model.

[0125] Here, fusion weight refers to the weight value set for the target model of each second facial model, used to adjust the deformation contribution of the target model in the fusion process. Deformation processing refers to the multi-model fusion deformation operation performed on the first facial model based on multiple target models and corresponding fusion weights. The fusion model refers to the new facial 3D mesh model obtained after the first facial model undergoes multi-target model fusion deformation processing. In the embodiments of this application, when there are multiple second facial models, fusion weights are set for the target models of each second facial model, and then the first facial model is deformed by combining all target models and fusion weights to achieve multi-model fusion and obtain a fusion model that combines the features of multiple second facial models. For example, there are two second facial models, namely second facial model A and second facial model B, and their corresponding target models are target model A and target model B. The fusion weight of target model A is set to 0.6, and the fusion weight of target model B is set to 0.4. The first facial model is deformed by combining these two target models and fusion weights to obtain a fusion model that combines the features of model A and model B.

[0126] In this embodiment, when multiple second facial models exist, the fusion weights corresponding to each second facial model and its processed target model are first obtained. Then, the first facial model is deformed by combining all fusion weights and the target model to obtain the fused model. This supports the fusion processing of multiple second facial models with the first facial model, enriching the application scenarios of model processing and improving the applicability and flexibility of the model processing method.

[0127] In some embodiments, the process of "deforming the first facial model based on each fusion weight and each target model to obtain a fusion model" in the above embodiments can be implemented through the following process, including: For each target model, determine the vertex offset between each vertex in the target model and each vertex in the first face model; based on the fusion weight corresponding to each target model, perform a weighted summation of each vertex offset to obtain the target offset; superimpose the target offset onto each vertex of the first face model to obtain the fusion model.

[0128] Here, vertex offset refers to the 3D coordinate difference between a vertex in the target model and the corresponding vertex in the first facial model, representing the degree of deformation influence of a single target model on a vertex of the first facial model. Target offset refers to the total coordinate difference obtained by weighted summation of the vertex offsets of all target models after fusion weights, representing the comprehensive deformation influence of multiple target models on a vertex of the first facial model. Superposition refers to the operation of adding the target offset to the original coordinates of the vertex of the first facial model. In the embodiments of this application, the vertex offset of each target model relative to the first facial model is first calculated, then the offsets are weighted and summed using fusion weights to obtain the total target offset, and finally the total offset is superimposed on the vertex of the first facial model to obtain the fused model.

[0129] In this embodiment, the offset of each target model vertex relative to the vertex of the first facial model is first calculated. Then, the offsets are weighted and summed according to the fusion weight of each target model to obtain the overall target offset. Finally, this offset is superimposed on each vertex of the first facial model to generate the fused model. In this way, weighted fusion based on vertex offsets achieves accurate deformation fusion of multiple models, effectively ensuring the shape fit between the fused model and each target model, and improving the accuracy and naturalness of multi-model fusion.

[0130] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0131] This application provides a multi-region face mesh feature fusion, alignment, and one-to-one reconstruction technique: Under the premise of unified coordinates and topological invariance, the face is first divided into facial feature regions such as nose, mouth, eyes, cheekbones, and jaw, and corresponding vertex weight masks (corresponding to region masks in other embodiments) are loaded. For each region, similarity transformations (rotation, translation, and unified scale) are solved based on the corresponding point set. Then, linear interpolation is performed on the "transformed position" and "original position" after weight normalization to achieve high-precision fitting and smooth boundary transition within the region. At the same time, a "region → accessory" mapping is established, and the same transformation is applied to the vertices of accessories such as eyeballs and teeth (corresponding to auxiliary components in other embodiments, with a weight of 1) to ensure strict pose synchronization. After feature fusion and alignment are completed, a vertex-by-vertex difference field is calculated and deposited in the form of deformed targets and wrapped deformations to achieve a replayable one-to-one transfer of the target facial features to the reference model. This application supports bidirectional feature fusion and alignment from target to reference and from reference to target, configurable batch processing, and quality verification, thereby achieving high consistency, high efficiency, and stable reuse in large-scale production.

[0132] This application's embodiment is a low-level capability module for "multi-region facial fusion" for character factories and digital human production. It can run independently or be embedded into existing digital content creation tools and engine pipelines. In terms of capability layer positioning, this application's embodiment provides low-level capabilities for model multi-region fusion and alignment. It first globally aligns the entire head region, then performs local alignment and fusion based on region masks and weights, synchronizing accessories and outputting a replayable difference channel. It sits between "geometric normalization → deformation channel generation," connecting upstream scanning, asset libraries, and downstream binding, rendering, and engines, maintaining consistent alignment and fusion across the toolchain.

[0133] The input data in this embodiment includes: (1) the vertex position of a source model (corresponding to the first face model in other embodiments) (corresponding to the first mesh data in other embodiments); (2) the vertex position of multiple target models (corresponding to the second face model in other embodiments) (corresponding to the second mesh data in other embodiments); (3) the region mask (corresponding to the region mask in other embodiments) and the region weight, as well as the "region → accessory" mapping (eyeball / teeth, etc.); (4) multiple values ​​of the mixed weight of each region (baseline + number of targets), automatically normalized.

[0134] The processing flow of this application embodiment includes: (1) Global rigid alignment: First, perform a similar rigid body alignment on the entire head region of the source model and the target model (corresponding to global rigid alignment in other embodiments, including rotation, translation, and optional unified scale) to bring the overall coordinates and pose closer; (2) Region non-rigid alignment (corresponding to the first rigid alignment in other embodiments): First, use region rigid alignment to match the vertex position of the region to the target, and then use the aligned vertex position to perform smooth deformation according to the vertex weight in the mask (corresponding to the first smoothing process in other embodiments). The deformation intensity is determined by the weight, so that the details fit and the boundary is more natural; (3) Accessory synchronization: Apply the same region rigid body transformation to the accessory vertex with weight = 1 (corresponding to the second rigid alignment in other embodiments) to ensure that the eye / tooth pose is consistent; (4) Vertex-level weight blending (corresponding to the second smoothing process in other embodiments): The aligned reference / target vertices are linearly blended according to the region blending weight; the uncovered vertices remain unchanged; the overlapping areas are automatically normalized, and the boundary is smooth and seamless.

[0135] See Figure 8 , Figure 8 This is a schematic diagram of facial feature region division provided in an embodiment of this application. For example... Figure 8 As shown, the right cheek region of the facial model is divided into an independent facial feature region 81.

[0136] See Figure 9 , Figure 9 This is a schematic diagram of global rigid alignment provided in an embodiment of this application. For example... Figure 9As shown, the two spatially separated facial models shown in the first graphic frame 91, after global rigid alignment processing, can achieve the effect shown in the second graphic frame 92, realizing the initial matching and alignment of the facial models in spatial position and overall posture.

[0137] Finally, the data output by the embodiments of this application may include: (1) fused mesh vertices (i.e., mesh data of the target model); (2) per-vertex difference field (which can be stored in the deformable target / wrap deformation / parameter file channel) for easy playback and batch reuse.

[0138] On the technical side, this embodiment proposes a low-level technology stack for "one-to-one restoration of face mesh feature fusion and alignment for multiple regions." The core idea is: given that the face is divided into several functional regions (nose, mouth, eyes, cheekbone / jaw, etc.), each region is first independently solved for similarity transformations (rotation, translation, unified scale), and the transformation results are linearly interpolated with the original geometry using a vertex-level fusion ratio, thereby achieving "high-precision fitting within the region and smooth transition between regions." Simultaneously, based on a "region → accessory" mapping mechanism, synchronous transformations are applied to accessories such as eyeballs and teeth; after alignment, a vertex-by-vertex difference field is calculated and deposited in the form of deformation target / wrap deformation or parameter file channels, supporting bidirectional alignment (target to reference / reference to target).

[0139] The data and resource model in the embodiments of this application will be described below: (1) Mesh data: The core includes a vertex list, index, and optional normal information. The vertex list consists of N vertices, each with three-dimensional coordinates (x, y, z), used to accurately represent the geometric structure of the face mesh; the index is used to define the connection relationship between vertices, ensuring the integrity and rationality of the mesh structure; the normal is an optional configuration and can be flexibly added according to actual application needs, mainly used to assist in subsequent mesh rendering, geometric feature analysis and other related processing.

[0140] (2) Region weight mask: The mapping form of "region name → vertex weight set" is adopted, where the vertex weight set is presented in the form of {vertex identifier: weight value}, and the value range of the weight value is strictly limited to [0, 1]. In order to achieve a smooth transition between different facial regions, the boundary weight of the mask adopts a gradual attenuation design; at the same time, the weight masks of different functional regions such as nose, mouth, and eyes are allowed to overlap to adapt to the fusion and alignment requirements of complex facial regions.

[0141] (3) Region → Accessory Mapping: The mapping structure of “region name → accessory model name set” is adopted. Its core purpose is to realize the synchronous control of accessories. Through this mapping relationship, the transformation of the corresponding facial region can be synchronously applied to the associated accessory model (such as eyeballs, teeth, etc.), ensuring that the posture of the accessory is strictly consistent with the corresponding facial region, and effectively avoiding accessory drift problem.

[0142] See Figure 10 , Figure 10 This is a schematic diagram of the multi-region facial model alignment process provided in an embodiment of this application. For example... Figure 10 As shown, the specific steps include: In step 1001, the first facial model and the second facial model are input.

[0143] Here, the two models can be 3D facial assets from different sources, such as scanned and reconstructed real face models, virtual character models, or models of the same character in different poses and expressions. The purpose is to provide the raw data foundation for subsequent facial alignment and fusion, serving as the starting point for the entire process.

[0144] In step 1002, the coordinates of the first facial model and the second facial model are unified with the assets.

[0145] Here, the units, up axis, orientation, and naming are standardized for both models to achieve consistency between the baseline and target models. This eliminates differences in coordinate systems and asset formats, preventing deviations in subsequent alignment calculations due to inconsistent basic standards. Furthermore, a head reference vertex set can be selected for center alignment, improving subsequent registration reliability.

[0146] In step 1003, the facial model's auxiliary components are arranged.

[0147] Here, the model structure is standardized. Facial ancillary components typically include sub-models independent of the main facial mesh, such as eyeballs, teeth, tongue, and eyelashes. The process involves clarifying the hierarchical relationship between the ancillary components and the main facial model, standardizing the component coordinate references, and marking the list of components that need synchronous transformation, preparing for subsequent region mapping and synchronous transformation.

[0148] In step 1004, the first facial model and the second facial model are globally rigidly aligned.

[0149] Here, a global rigid transformation (such as the Kabsch algorithm or Procrustes analysis) is used to solve for the rotation and translation matrices between the two models (some scenes may involve uniform scaling). This quickly reduces the differences in overall pose and position between the two models, allowing their faces to roughly overlap, thus simplifying subsequent fine-grained region alignment.

[0150] Specifically, under two sets of corresponding points or nearest neighbor correspondence, the target point set is rigidly aligned to the reference point set by solving for three parameters: rotation, translation, and scaling. First, the geometric centers of the reference point set X and the target point set Y are calculated separately. Then, each point in X and Y is subtracted from its center to perform a decentering process, eliminating interference from translation. Next, the covariance matrix is ​​calculated using the decentered X and Y. Then, perform singular value decomposition on H to obtain , The rotation parameters are shown in formula (1): (1) The uniform scale is shown in formula (2): (2) The translation parameters are shown in formula (3): (3) In step 1005, the region mask and region weight are obtained.

[0151] Here, the face is divided into multiple semantic feature regions (such as the forehead, eye area, nose area, mouth area, cheeks, etc.), and each vertex is labeled with a mask to indicate the region it belongs to. Differentiated weights are assigned to different regions (e.g., higher weights for facial features and lower weights for cheek edges) for subsequent weighted alignment to ensure the alignment accuracy of key facial features.

[0152] In step 1006, a mapping table between each facial feature region and its associated components is established.

[0153] Here, the facial feature region to which each accessory component (such as the left eye corresponding to region A around the eye, and teeth corresponding to region C around the mouth) belongs is recorded. This ensures that after subsequent region transformation calculations are completed, the region transformation can be accurately transferred to the corresponding accessory components, avoiding misalignment between the components and the main face.

[0154] In step 1007, multiple facial feature regions are processed in parallel.

[0155] Here, by building a parallel architecture, different facial feature regions (A / B / C / D, etc.) are allocated to multi-process / distributed nodes for independent computation. Parallel computing power is used to improve alignment efficiency and avoid single-region computation blocking the overall process, making it suitable for processing high-polygon facial models.

[0156] In step 1008, each facial feature region is individually aligned.

[0157] Here, for each predefined feature region (e.g., region A, region B), weighted rigid alignment (e.g., weighted Kabsch algorithm) is performed based on the region weights from step 1005. The independent rotation, translation, and scaling parameters for each region are solved to achieve precise alignment of local features, addressing the problem that global alignment cannot take local details into account.

[0158] Specifically, for each independent region r, the corresponding set of similarity transformations—rotation, translation, and scaling—are solved. First, the weighted centroid is calculated. The center of the reference point is obtained by weighted averaging of the coordinates of all points, and the center of the target point is calculated using the same weighted averaging method. Next, the weighted covariance matrix H is calculated by subtracting the corresponding center point from the coordinates of each point, multiplying them according to their corresponding weights, and summing them up. Then, using the same process as the conventional Kabusch algorithm, the covariance matrix H is solved by singular value decomposition to obtain the rotation matrix. The scale parameter is obtained by dividing the sum of singular values ​​by the weighted variance, thereby calculating the rotation, translation, and scaling parameters specific to this region. Finally, this set of region transformations is applied to all vertices, and each vertex will participate in the subsequent overall result summarization and fusion based on its own weight value in the region mask.

[0159] In step 1009, the alignment results of each facial feature region are fused.

[0160] Here, based on vertex weights or region boundary transition rules, the independent transformation results of multiple regions are blended. The aim is to eliminate problems such as vertex tearing and facial deformation at region splicing points, ensuring the overall continuity of the face.

[0161] Specifically, the vertex normalization fusion (feature fusion) operation involves, for each vertex i, first collecting its weight values ​​in each region. If a vertex is not in a certain region, its weight is recorded as 0; then the total weight of this vertex across all regions is calculated. This is used as the total weight. Next, we first calculate the position of each region after aligning it with this vertex, and then calculate the normalization coefficient for each region. This coefficient is the weight of the vertex in this region divided by the total weight. (To avoid the total weight being 0, a minimum value ε will be used as a lower limit constraint (or as a minimum value compensation)), as shown in formula (4): (4) Specifically, if a vertex is covered by multiple regions at the same time, the position of the vertex after alignment in each region is calculated by weighted average based on its weight ratio in each region to obtain the final position result.

[0162] In step 10010, a globally continuous vertex transformation field is generated.

[0163] Here, the vertex transformation and difference are calculated using a fusion formula to obtain a continuous transformation field interpolated per vertex (each vertex corresponds to a set of rotation, translation, and scaling transformations). This transformation field describes the vertex-by-vertex position change rules from the source facial model to the target facial model and is the core basis for subsequent displacement field generation.

[0164] In step 10011, the auxiliary components undergo synchronous rigid transformation.

[0165] Here, based on the mapping table established in step 1006, the transformation parameters of the corresponding regions are applied to the auxiliary components (e.g., the transformation of the periorbital region is synchronized to the eyeball, and the transformation of the periorbital region is synchronized to the teeth). This ensures that the relative position and pose of the auxiliary components remain consistent with the main facial model, avoiding problems such as the eyeball detaching from its socket or the teeth becoming misaligned.

[0166] In step 10012, the position difference before and after the vertex transformation is calculated to generate the vertex displacement field.

[0167] Here, for each vertex, a three-dimensional displacement vector is obtained by calculating the position difference between the transformed position and the original position. The set of displacement vectors of all vertices constitutes the vertex displacement field, which intuitively describes the deformation amplitude and direction from the source model to the target model, and is the core data for facial animation and retargeting.

[0168] In step 10013, the displacement field is precipitated into a standard data channel and converted into a universal format.

[0169] Here, the displacement field is stored as a custom data channel of the model (such as vertex color channel, UV channel, and custom attribute channel) for easy direct reading. Through format conversion, the displacement field is exported to an industry-standard format, compatible with downstream workflows such as rendering, animation, and game engines.

[0170] In step 10014, the aligned target model is output.

[0171] Here, the output model includes the aligned main face mesh, synchronously updated auxiliary components, bound vertex displacement fields, and transformation fields. This model can be directly used for subsequent applications such as face retargeting, expression transfer, and character generation, and is the final result of the entire process.

[0172] The following describes the input / output specifications and interfaces in the embodiments of this application. The input includes three types of files: first, a model or parameter file used to obtain vertex position information; second, a region weight file containing weight data for each vertex in each region; and third, a mapping table between regions and accessories used to associate each region with its corresponding accessory component. The process output includes three parts: first, an aligned mesh, i.e., a mesh that has been initially aligned to the target after global or regional rigid body alignment, but has not yet undergone region weight blending; second, a blended mesh, which, based on the aligned mesh, linearly blends the reference and target vertices according to the region mask weights, where overlapping regions are automatically normalized according to their weight proportions to make the mesh boundaries smoother; and third, per-vertex differences and corresponding channels, obtained by calculating the difference between the aligned or blended vertex position and the source model vertex position, which can be stored as deformation targets, wrap deformations, or parameter file channels for subsequent deformation playback, overlay, and batch reuse. In addition, users can input custom weights for each region to control the degree of blending in that region. These weights will be normalized and then participate in vertex-level linear blending. Overlapping regions will automatically share the blending ratio according to their weight proportions, while vertices not covered by any region will remain in their original state, thus achieving fine adjustment of the final effect of each region.

[0173] This application's embodiment achieves high-precision local alignment and natural boundary transitions for facial meshes through the core technology of "multi-region similarity transformation + vertex weight fusion." Its technical effects are significant and comprehensive: In terms of alignment accuracy and consistency, key regions such as the nose, mouth, eyes, and cheekbones can achieve near-perfect matching. The continuous weight decay design allows for smooth transitions between regions, effectively avoiding stretching and seam issues. Simultaneously, the accessory synchronization mechanism ensures that the geometry of peripheral elements such as eyeballs and teeth undergoes strict synchronous transformation with their corresponding regions, guaranteeing a perfect fit between the overall effect and the target. This embodiment supports bidirectional alignment from arbitrary topology and target to reference, and from reference to target, without relying on fixed standard topologies. It can adapt to meshes from multiple sources, significantly reducing pre-modification costs and covering more production scenarios. In terms of functional practicality, the normalized multi-region weight fusion eliminates the influence of processing order, improving the repeatability and testability of the process. The output vertex-by-vertex difference field can be stored as standard channels such as deformation targets, wrap deformations, and parameter files, enabling one-to-one restoration, playback, overlay, and version management of deformations. At the engineering application level, by adopting regional parallelism, quality gating and automatic rollback mechanisms, the processing time of a single model can be significantly reduced, and the parallel processing capability can reach hundreds of models per hour. Moreover, it is easy to quickly replicate and standardize the deployment. The modules can be embedded into mainstream digital content creation tools and pipelines in the form of standalone, plug-in or software development kit. The standard channel output can be directly imported into various asset processes, realizing consistent transmission across toolchains, and providing stable and reliable underlying support for large-scale digital human production.

[0174] The following description continues to illustrate the exemplary structure of the model processing device 233 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the model processing device 233 of the memory 230 may include: Alignment module 2331 is used to acquire a first facial model and a second facial model to be processed, and to perform global rigid alignment on the first facial model and the second facial model to obtain an initial alignment result, wherein the initial alignment result includes a first region model corresponding to each of multiple facial features; The alignment module 2331 is further configured to perform a first rigid alignment on the corresponding first region model for each first region model, so as to obtain the region alignment result corresponding to the first region model. The smoothing module 2332 is used to perform a first smoothing process on the region alignment result based on the region weights corresponding to the first region model, so as to obtain a smoothed region alignment result. The alignment module 2331 is further configured to perform a second rigid alignment on each of the auxiliary components corresponding to the first region model to obtain a target alignment result corresponding to the first region model, wherein the rigid transformation parameters used for performing the first rigid alignment and performing the second rigid alignment are the same. The smoothing module 2332 is further configured to perform a second smoothing process on the target alignment result corresponding to each of the first region models to obtain the target model corresponding to the second facial model.

[0175] In some embodiments, the alignment module 2331 is further configured to determine a region mask corresponding to each facial feature; divide the first facial model based on each region mask to obtain a divided first facial model; determine global transformation parameters between the first facial model and the second facial model; and perform a global rigid transformation on the divided first facial model or the second facial model based on the global transformation parameters to obtain the initial alignment result.

[0176] In some embodiments, the alignment module 2331 is further configured to determine first mesh data corresponding to the first facial model and second mesh data corresponding to the second facial model; determine a first mapping relationship between the first facial model and the second facial model based on the first mesh data and the second mesh data; and determine the global transformation parameters based on the first mapping relationship, wherein the global transformation parameters include at least one of the following: rotation parameters, translation parameters, and uniform scale parameters.

[0177] In some embodiments, the alignment module 2331 is further configured to determine the first mesh data corresponding to the first facial model and the semantic partition information of the first facial model; for each facial feature, based on the semantic partition information, determine a target vertex from the first mesh data, the target vertex being located in the second region model corresponding to the facial feature; set the mask value of the target vertex to a first value, and set the mask values ​​of the first other vertices besides the target vertex to a second value, to obtain the region mask corresponding to the facial feature.

[0178] In some embodiments, the alignment module 2331 is further configured to determine a third region model in the second facial model corresponding to the facial features; determine rigid transformation parameters between the first region model and the third region model; and perform rigid transformation processing on the first region model based on the rigid transformation parameters to obtain the region alignment result.

[0179] In some embodiments, the alignment module 2331 is further configured to determine, from the first mesh data corresponding to the first region model, the first vertex data corresponding to the first region model; determine, from the second mesh data corresponding to the second face model, the second vertex data corresponding to the third region model; and determine the rigid transformation parameters based on the first vertex data and the second vertex data.

[0180] In some embodiments, the alignment module 2331 is further configured to determine a second mapping relationship between the first vertex data and the second vertex data; based on the second mapping relationship, construct an objective function to characterize the alignment error between the first region model and the third region model; and obtain the rigid transformation parameters by solving the objective function.

[0181] In some embodiments, the first region model includes a plurality of first vertices, and the region weights include vertex weights corresponding to each first vertex; the smoothing module 2332 is further configured to, for each first vertex, determine first coordinate information of the first vertex in the initial alignment result and second coordinate information of the first vertex in the region alignment result; perform weighted summation processing on the first coordinate information and the second coordinate information based on the vertex weights corresponding to the first vertex to obtain first target coordinates corresponding to the first vertex; and construct the smoothed region alignment result based on the first target coordinates corresponding to each first vertex.

[0182] In some embodiments, the smoothing module 2332 is further configured to: determine the center vertex corresponding to the first region model from a plurality of first vertices; determine the first vertices other than the center vertex as second other vertices; determine the vertex weight corresponding to the center vertex as a first value; for each second other vertex, determine the distance information between the second other vertex and the center vertex; and determine the vertex weight corresponding to the second other vertex based on the distance information, wherein the vertex weight corresponding to the second other vertex is negatively correlated with the distance information, and the vertex weight corresponding to the second other vertex is less than the first value.

[0183] In some embodiments, the alignment module 2331 is further configured to, for each of the auxiliary components, determine the rigid transformation parameters used by the first region model corresponding to the auxiliary component when performing the first rigid alignment; determine the constraint conditions of the auxiliary component, the constraint conditions being used to constrain the degree of deformation of the auxiliary component; and perform a second rigid alignment on the auxiliary component based on the rigid transformation parameters and the constraint conditions to obtain the target alignment result corresponding to the first region model.

[0184] In some embodiments, the alignment module 2331 is further configured to determine initial relative position information between the vertices of each component in the auxiliary component; determine a deformation threshold corresponding to the auxiliary component based on each initial relative position information; and construct the constraint condition based on the deformation threshold, wherein the constraint condition is used to constrain that the relative position change between the vertices of each component is less than or equal to the deformation threshold when performing the second rigid alignment.

[0185] In some embodiments, the smoothing module 2332 is further configured to: determine, for each of the target alignment results, the third coordinate information of each second vertex included in the target alignment result; determine, for each second vertex, the coverage state between the second vertex and each other target alignment result; determine, based on the coverage state and the third coordinate information, the smoothed second target coordinates of the second vertex; and construct the target model based on the second target coordinates corresponding to each vertex.

[0186] In some embodiments, the smoothing module 2332 is further configured to: determine the third coordinate information as the second target coordinates when the coverage state indicates that the second vertex is not covered by other target alignment results; determine the first region model corresponding to each target alignment result that covers the second vertex as the target region model when the coverage state indicates that the second vertex is covered by other target alignment results; and determine the second target coordinates of the second vertex based on the region weight corresponding to each target region model.

[0187] In some embodiments, the smoothing module 2332 is further configured to determine the vertex weight corresponding to the second vertex from the region weights corresponding to each target region model; normalize each vertex weight corresponding to the second vertex to obtain the target weight; determine the fourth coordinate information of the second vertex in the target alignment result corresponding to each target region model; and perform weighted summation on each fourth coordinate information based on each target weight to obtain the second target coordinate of the second vertex.

[0188] In some embodiments, the second facial model includes multiple models, and the alignment module 2331 is further configured to obtain the fusion weight corresponding to each second facial model and the target model corresponding to each second facial model; based on each fusion weight and each target model, the first facial model is deformed to obtain a fusion model.

[0189] In some embodiments, the alignment module 2331 is further configured to: for each target model, determine the vertex offset between each vertex in the target model and each vertex in the first face model; perform weighted summation on each vertex offset based on the fusion weight corresponding to each target model to obtain a target offset; and superimpose the target offset onto each vertex of the first face model to obtain the fusion model.

[0190] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the model processing method described above in this application.

[0191] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the model processing method provided in this application. For example, ... Figure 3 The model processing method is shown.

[0192] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0193] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0194] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored in a portion of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0195] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0196] In summary, through the embodiments of this application, firstly, global rigid alignment is used to unify the first and second facial models to the same spatial coordinate system, eliminating translation and rotation errors at the macro level. Next, independent first rigid alignment is performed on each first region model defined in the initial result, significantly improving the local registration accuracy of key facial regions. Then, a first smoothing process based on region weights is introduced to ensure seamless fusion of the boundaries of each independent region, effectively eliminating discontinuities and fragmentations on the model surface and ensuring the naturalness and continuity of the facial skin topology. Subsequently, a second rigid alignment is performed on facial accessory components, reusing the same rigid transformation parameters for their corresponding regions, ensuring the relative spatial positional relationship between accessory organs and the facial epidermis, avoiding the problems of internal component detachment, misalignment, or clipping that easily occur during complex deformation of the model. Finally, a second smoothing process is performed on the target result after integrating accessory components, further eliminating minor deformation defects accumulated during local assembly and component integration at the global level. The embodiments of this application not only take into account the high-precision alignment of macro contours and local details, but also cleverly solve the common problems of mesh tearing and attachment component clipping in the processing of complex facial models. The final output is a target facial model with correct structure, high fidelity and extremely natural and smooth surface transition, which improves the accuracy and precision of facial model processing.

[0197] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A model processing method, characterized in that, The method includes: A first facial model and a second facial model to be processed are obtained, and the first facial model and the second facial model are globally rigidly aligned to obtain an initial alignment result. The initial alignment result includes a first region model corresponding to each of multiple facial features. For each of the first region models, a first rigid alignment is performed on the first region model to obtain the region alignment result corresponding to the first region model; Based on the region weights corresponding to the first region model, the region alignment result is subjected to a first smoothing process to obtain a smoothed region alignment result. The first smoothing process is a local smoothing fusion process performed on a single region alignment result. A second rigid alignment is performed on each of the auxiliary components corresponding to the first region model to obtain the target alignment result corresponding to the first region model. The rigid transformation parameters used for the first rigid alignment and the second rigid alignment are the same. The auxiliary component is a facial auxiliary geometry that matches the facial feature region corresponding to the first region model. A second smoothing process is performed on the target alignment result corresponding to each of the first region models to obtain the target model corresponding to the second face model. The second smoothing process is a global smoothing fusion process performed on the target alignment results of each of the first region models.

2. The method according to claim 1, characterized in that, The step of performing global rigid alignment on the first facial model and the second facial model to obtain an initial alignment result includes: Determine the region mask corresponding to each of the facial features; Based on each region mask, the first facial model is segmented to obtain the segmented first facial model. Determine the global transformation parameters between the first facial model and the second facial model; Based on the global transformation parameters, a global rigid transformation is performed on the segmented first facial model or the second facial model to obtain the initial alignment result.

3. The method according to claim 2, characterized in that, Determining the global transformation parameters between the first facial model and the second facial model includes: Determine the first mesh data corresponding to the first facial model and the second mesh data corresponding to the second facial model; Based on the first grid data and the second grid data, a first mapping relationship is determined between the first facial model and the second facial model; Based on the first mapping relationship, the global transformation parameters are determined, and the global transformation parameters include at least one of the following: rotation parameters, translation parameters, and uniform scale parameters.

4. The method according to claim 2, characterized in that, Determining the region mask corresponding to each facial feature includes: Determine the first grid data corresponding to the first facial model, and the semantic partition information of the first facial model; For each facial feature, a target vertex is determined from the first grid data based on the semantic partitioning information, and the target vertex is located in the second region model corresponding to the facial feature; The mask value of the target vertex is set to a first value, and the mask values ​​of the first other vertices besides the target vertex are set to a second value to obtain the region mask corresponding to the facial feature.

5. The method according to claim 1, characterized in that, The step of performing a first rigid alignment on the first region model to obtain the region alignment result corresponding to the first region model includes: Determine the third region model in the second facial model that corresponds to the facial features; Determine the rigid transformation parameters between the first region model and the third region model; The first region model is subjected to rigid transformation based on the rigid transformation parameters to obtain the region alignment result.

6. The method according to claim 5, characterized in that, Determining the rigid transformation parameters between the first region model and the third region model includes: From the first mesh data corresponding to the first facial model, determine the first vertex data corresponding to the first region model; The second vertex data corresponding to the third region model is determined from the second mesh data corresponding to the second facial model; The rigid transformation parameters are determined based on the first vertex data and the second vertex data.

7. The method according to claim 6, characterized in that, Determining the rigid transformation parameters based on the first vertex data and the second vertex data includes: Determine a second mapping relationship between the first vertex data and the second vertex data; Based on the second mapping relationship, an objective function is constructed to characterize the alignment error between the first region model and the third region model; The rigid transformation parameters are obtained by solving the objective function.

8. The method according to claim 1, characterized in that, The first region model includes multiple first vertices, and the region weights include the vertex weights corresponding to each first vertex. The first smoothing process, based on the region weights corresponding to the first region model, is applied to the region alignment result to obtain a smoothed region alignment result, including: For each of the first vertices, determine the first coordinate information of the first vertex in the initial alignment result and the second coordinate information of the first vertex in the region alignment result; Based on the vertex weight corresponding to the first vertex, the first coordinate information and the second coordinate information are weighted and summed to obtain the first target coordinates corresponding to the first vertex. Based on the first target coordinates corresponding to each first vertex, the smoothed region alignment result is constructed.

9. The method according to claim 8, characterized in that, The method further includes: From the plurality of first vertices, determine the center vertex corresponding to the first region model, and determine the first vertices other than the center vertex as the second other vertices; The vertex weight corresponding to the center vertex is determined to be the first value; For each of the second other vertices, determine the distance information between the second other vertex and the center vertex; Based on the distance information, the vertex weight corresponding to the second other vertex is determined, wherein the vertex weight corresponding to the second other vertex is negatively correlated with the distance information, and the vertex weight corresponding to the second other vertex is less than the first value.

10. The method according to claim 1, characterized in that, The step of performing a second rigid alignment on each of the subordinate components corresponding to the first region model to obtain the target alignment result corresponding to the first region model includes: For each of the aforementioned auxiliary components, determine the rigid transformation parameters used in the first region model corresponding to the auxiliary component when performing the first rigid alignment; Determine the constraints on the auxiliary component, the constraints being used to limit the degree of deformation of the auxiliary component; Based on the rigid transformation parameters and the constraints, a second rigid alignment is performed on the auxiliary components to obtain the target alignment result corresponding to the first region model.

11. The method according to claim 10, characterized in that, The constraints for determining the auxiliary components include: Determine the initial relative position information between the vertices of each component in the auxiliary component; Based on the initial relative position information for each of the above, the deformation threshold corresponding to the auxiliary component is determined; The constraint condition is constructed based on the deformation threshold, and the constraint condition is used to constrain that the relative position change between the vertices of the components is less than or equal to the deformation threshold when performing the second rigid alignment.

12. The method according to claim 1, characterized in that, The step of performing a second smoothing process on the target alignment result corresponding to each of the first region models to obtain the target model corresponding to the second facial model includes: For each target alignment result, determine the third coordinate information of each second vertex included in the target alignment result; For each of the second vertices, determine the coverage state between the second vertex and the alignment results of each of the other targets; Based on the coverage state and the third coordinate information, determine the smoothed second target coordinates of the second vertex; The target model is constructed based on the second target coordinates corresponding to each vertex.

13. The method according to claim 12, characterized in that, The step of determining the smoothed second target coordinates of the second vertex based on the coverage state and the third coordinate information includes: When the coverage state indicates that the second vertex is not covered by other target alignment results, the third coordinate information is determined as the coordinates of the second target; When the coverage state indicates that the second vertex is covered by the other target alignment results, the first region model corresponding to each target alignment result that covers the second vertex is determined as the target region model; Based on the region weights corresponding to each target region model, the second target coordinates of the second vertex are determined.

14. The method according to claim 13, characterized in that, Determining the second target coordinates of the second vertex based on the region weights corresponding to each target region model includes: From the region weights corresponding to each target region model, determine the vertex weights corresponding to the second vertex; The weight of each vertex corresponding to the second vertex is normalized to obtain the target weight; Determine the fourth coordinate information of the second vertex in the target alignment result corresponding to each of the target region models; Based on each of the target weights, the fourth coordinate information of each of the four coordinates is weighted and summed to obtain the second target coordinates of the second vertex.

15. The method according to any one of claims 1 to 14, characterized in that, The second facial model includes multiple models, and the method further includes: Obtain the fusion weights corresponding to each of the second facial models, and the target model corresponding to each of the second facial models; Based on each of the fusion weights and each of the target models, the first facial model is deformed to obtain the fusion model.

16. The method according to claim 15, characterized in that, The process of deforming the first facial model based on each of the fusion weights and each of the target models to obtain the fusion model includes: For each target model, determine the vertex offset between each vertex in the target model and each vertex in the first facial model; Based on the fusion weights corresponding to each target model, the vertex offsets of each vertex are weighted and summed to obtain the target offsets. The target offset is superimposed onto each vertex of the first facial model to obtain the fused model.

17. A model processing device, characterized in that, The device includes: The alignment module is used to acquire a first facial model and a second facial model to be processed, and to perform global rigid alignment on the first facial model and the second facial model to obtain an initial alignment result. The initial alignment result includes a first region model corresponding to each of multiple facial features. The alignment module is further configured to perform a first rigid alignment on the corresponding first region model for each first region model, so as to obtain the region alignment result corresponding to the first region model. The smoothing module is used to perform a first smoothing process on the region alignment result based on the region weights corresponding to the first region model, so as to obtain a smoothed region alignment result. The first smoothing process is a local smoothing fusion process performed on a single region alignment result. The alignment module is further configured to perform a second rigid alignment on each of the auxiliary components corresponding to the first region model to obtain the target alignment result corresponding to the first region model. The rigid transformation parameters used for the first rigid alignment and the second rigid alignment are the same. The auxiliary component is a facial auxiliary geometric structure that matches the facial feature region corresponding to the first region model. The smoothing module is further configured to perform a second smoothing process on the target alignment result corresponding to each of the first region models to obtain the target model corresponding to the second face model. The second smoothing process is a global smoothing fusion process performed on the target alignment results of each of the first region models.

18. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 16.

19. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method described in any one of claims 1 to 16.

20. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method according to any one of claims 1 to 16.

Images