Three-dimensional model processing method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a three-dimensional model processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps that a first three-dimensional model and a second three-dimensional model which are used for representing the same object are obtained, and a first vertex in the first three-dimensional model and preset skeleton parameters in skeletons have an incidence relation; extracting a first feature vector of each first vertex in the first three-dimensional model and a second feature vector of each second vertex in the second three-dimensional model; determining a second vertex corresponding to each first vertex based on the first feature vector and the second feature vector; and associating the skeleton parameters having the association relationship with each first vertex in the first three-dimensional model with a second vertex corresponding to the first vertex, and taking the associated second three-dimensional model as a third three-dimensional model. According to the invention, the generation efficiency of the skeleton parameters in the three-dimensional model can be improved.

Description

Technical Field

[0001] This application relates to computer graphics technology, and particularly to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for processing three-dimensional models. Background Art

[0002] In the fields of computer graphics and 3D modeling, skeleton binding and skinning of 3D models are a key step, especially in animation production and character design. Skeleton binding and skinning involve associating virtual skeleton parameters with the geometry of 3D models so as to control the movement of 3D models through skeleton animation, thereby achieving natural animation effects. Related technologies often set skeleton parameters manually, resulting in low model generation efficiency, or automatically obtain skeleton parameters by training a deep neural network, and the training process depends on the quality of training data, resulting in poor adaptability of the model. Summary of the Invention

[0003] Embodiments of this application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for processing three-dimensional models, which can improve the generation efficiency of skeleton parameters in three-dimensional models.

[0004] The technical solution of the embodiments of this application is implemented as follows:

[0005] Embodiments of this application provide a method for processing three-dimensional models, the method including:

[0006] Obtain a first three-dimensional model and a second three-dimensional model for representing the same object, wherein a first vertex in the first three-dimensional model has an association relationship with preset skeleton parameters in a skeleton;

[0007] Extract a first feature vector of each first vertex in the first three-dimensional model and a second feature vector of each second vertex in the second three-dimensional model;

[0008] Based on the first feature vector and the second feature vector, determine a second vertex corresponding to each first vertex;

[0009] Associate the skeleton parameters in the first three-dimensional model that have the association relationship with each first vertex with the second vertex corresponding to the first vertex, and use the second three-dimensional model after the association as a third three-dimensional model.

[0010] Embodiments of this application provide a device for processing three-dimensional models, the device including:

[0011] A model acquisition module, configured to acquire a first 3D model and a second 3D model for representing the same object, wherein a first vertex in the first 3D model has an associated relationship with a preset bone parameter in a skeleton;

[0012] A feature extraction module, configured to extract a first feature vector of each first vertex in the first 3D model and a second feature vector of each second vertex in the second 3D model;

[0013] A vertex matching module, configured to determine, based on the first feature vector and the second feature vector, a second vertex corresponding to each first vertex;

[0014] A model generation module, configured to associate the bone parameter in the first 3D model that has the associated relationship with each first vertex with the second vertex corresponding to the first vertex, and use the second 3D model after the association as a third 3D model.

[0015] An embodiment of the present application provides an electronic device, where the electronic device includes:

[0016] A memory, configured to store computer-executable instructions or a computer program;

[0017] A processor, configured to implement the 3D model processing method provided by the embodiment of the present application when executing the computer-executable instructions or the computer program stored in the memory.

[0018] An embodiment of the present application provides a computer-readable storage medium, storing a computer program or computer-executable instructions, which are configured to implement the 3D model processing method provided by the embodiment of the present application when being executed by a processor.

[0019] An embodiment of the present application provides a computer program product, including a computer program or computer-executable instructions, where the computer program or computer-executable instructions implement the 3D model processing method provided by the embodiment of the present application when being executed by a processor.

[0020] The embodiment of the present application has the following beneficial effects:

[0021] By extracting the feature vectors of the first 3D model and the second 3D model, and determining the second vertices respectively corresponding to each first vertex in the first 3D model, the automatic transfer and reuse of bone parameters are realized, which reduces the setting time and workload compared with manually setting bone parameters, and improves the generation efficiency of the third 3D model. The first 3D model and the second 3D model only represent the same object, and the postures or sizes of the first 3D model and the second 3D model are not limited, which improves the flexibility and adaptability of model processing. Description of the Drawings

[0022] Figure 1 It is a schematic architecture diagram of a three - dimensional model processing system 100 provided by an embodiment of the present application;

[0023] Figure 2 It is a schematic structural diagram of an electronic device 500 provided by an embodiment of the present application;

[0024] Figure 3A It is a first process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0025] Figure 3B It is a second process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application.

[0026] Figure 3C It is a third process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0027] Figure 3D It is a fourth process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0028] Figure 3E It is a fifth process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0029] Figure 3F It is a sixth process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0030] Figure 3G It is a seventh process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0031] Figure 3H It is an eighth process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0032] Figure 3I It is a ninth process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0033] Figure 3J It is a tenth process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0034] Figure 3K It is an eleventh process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0035] Figure 3L It is a twelfth process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0036] Figure 3M It is a thirteenth process schematic diagram of a three - dimensional model processing method provided by an embodiment of the present application;

[0037] Figure 3N is the fourteenth process schematic diagram of the three-dimensional model processing method provided by the embodiments of the present application;

[0038] Figure 3O is the fifteenth process schematic diagram of the three-dimensional model processing method provided by the embodiments of the present application;

[0039] Figure 4 is the schematic diagram of the first three-dimensional model and the second three-dimensional model provided by the embodiments of the present application;

[0040] Figure 5A is the schematic diagram of the global sample vertex rotation provided by the embodiments of the present application;

[0041] Figure 5B is the schematic diagram of the global sample vertex translation provided by the embodiments of the present application;

[0042] Figure 6 is the schematic diagram of the segmented sample three-dimensional model provided by the embodiments of the present application;

[0043] Figure 7A is the schematic diagram of the local sample vertex rotation provided by the embodiments of the present application;

[0044] Figure 7B is the schematic diagram of the local sample vertex translation provided by the embodiments of the present application;

[0045] Figure 8 is the structural schematic diagram of the relationship matrix provided by the embodiments of the present application;

[0046] Figure 9 is the schematic diagram of the segmented three-dimensional model provided by the embodiments of the present application;

[0047] Figure 10 is the process schematic diagram of the model generation provided by the embodiments of the present application;

[0048] Figure 11 is the process schematic diagram of the processing template model provided by the embodiments of the present application;

[0049] Figure 12 is the schematic diagram of the part-by-part scaling of the template model provided by the embodiments of the present application;

[0050] Figure 13 is the process schematic diagram of the bone binding and skinning provided by the embodiments of the present application.

[0051] It should be noted that the above "first" and "second" are only used to distinguish different solutions, and do not represent the distinction of the quality or priority in the implementation process of the solutions. Detailed implementation manners

[0052] In order to make the objectives, technical solutions, and advantages of this application clearer, the following will further describe this application in detail in conjunction with the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

[0053] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0054] In the following description, the terms "first / second / third" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of this application described here can be implemented in an order other than that illustrated or described here.

[0055] In the embodiments of this application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the function of that module or unit.

[0056] Unless otherwise specified, the at least one described below refers to the case of one or more, and "multiple" can refer to the case of two or more.

[0057] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as can be understood by those skilled in the technical field to which this application belongs. The terms used in the embodiments of this application are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] In the embodiments of this application, when collecting and processing relevant data in practical applications, the informed consent or separate consent of the personal information subject should be obtained in strict accordance with the requirements of relevant laws and regulations, and subsequent data use and processing behaviors should be carried out within the scope authorized by laws and regulations and the personal information subject.

[0059] Before further elaborating on the embodiments of this application, the nouns and terms involved in the embodiments of this application are explained. The nouns and terms involved in the embodiments of this application are subject to the following explanations.

[0060] 1) Responsive to: Used to represent the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more of the executed operations can be real-time or have a set delay; without special instructions, there is no restriction on the execution order of multiple executed operations.

[0061] 2) Human-computer interaction interface, an interface for providing human-computer interaction functions / an interface for displaying 3D models and animations.

[0062] For example, graphical user interface (GUI) display, such as augmented reality (AR) interface, virtual reality (VR) interface, voice user interface (VUI), interactive projection interface (using projection technology to display information on a flat surface), eye movement detection interface (an interface controlled by detecting the user's line of sight), holographic interface (a three-dimensional hologram formed by projecting an image through holographic projection technology, allowing a stereoscopic image to be seen without wearing special glasses), multimodal interface (an interactive interface that combines multiple interaction methods such as touch, vision, and hearing), brain-machine interface (BMI) interface, etc.

[0063] 3) 3D model, a mathematical model used in computer graphics to represent an object or a scene. It consists of a set of geometric data (such as vertices, edges, faces) and additional attributes (such as colors, textures, materials), and is used to render a three-dimensional image on a computer screen or other display devices.

[0064] 4) Bones, in 3D animation and modeling, refer to a set of connected nodes or joints used to define and control the movement and deformation of 3D models. Bones can form a hierarchical structure, and each bone node can have one or more child nodes, allowing each part of the model to move and rotate in a natural way.

[0065] 5) Bone parameters refer to the data that define the position, orientation, and relative relationships of bones in 3D space. Bone parameters include bone positions and bone weights. Among them, bone positions refer to the coordinate positions of bones in 3D space, which can be defined as the positions of the root nodes or joints of bones; bone weights define how the vertices of the model are affected by bone movements. Each vertex can have multiple bone weights, indicating the degree to which it is affected by multiple bones.

[0066] 6) Global features refer to feature vectors that can describe the entire 3D model or its significant parts. These features can be extracted from the geometric structure, topological relationships, or other properties of the model, and can capture the overall shape, pose, or other global properties of the model. Mathematically, global features can be represented as a vector, where each element represents a specific property or feature of the model. Global features include at least one of the following: geometric features, semantic features, context features, animation features, and texture features.

[0067] 7) Local features refer to feature vectors that can describe local regions or specific parts in a 3D model. These features can be extracted from the geometric structure, texture, color, or other properties of the model, and can capture the local shape, texture pattern, or other local properties of the model. Mathematically, local features can be represented as a vector, where each element represents a specific local property or feature of the model. After dividing the 3D model into multiple sub-models, the features extracted from the vertices in the sub-models are used as local features.

[0068] 8) A configuration file is a document that includes specific instructions and parameters and is used to guide the operation of a computer program or system. The configuration file includes the segmentation order and multiple segmentation positions for segmenting the 3D model. This information can be stored in the form of key-value pairs or structured data so that the program can read and execute the corresponding operations.

[0069] 9) Pose parameters refer to the parameters that describe the position and orientation of a model in 3D space. Pose parameters can include position parameters and orientation parameters. The position parameters are used to describe the coordinate position of the model in space, and the orientation parameters are used to describe the orientation or rotation state of the model.

[0070] 10) Animation data refers to a set of information used to describe the movement and deformation of a 3D model over time. Animation data includes the movement trajectories of bones, the weight distribution of vertices, the poses of key frames, and other parameters that affect the appearance and behavior of the model. Animation data is the key to realizing 3D character animation, which enables the model to move and deform in a natural and realistic way. Mathematically, animation data can be represented as a series of bone transformation matrices or vertex position vectors that change over time.

[0071] Related technologies often set bone parameters manually, resulting in low model generation efficiency. Or, they automatically obtain bone parameters by training a deep neural network, and the training process depends on the quality of the training data, resulting in poor model adaptability.

[0072] Based on the above analysis, the applicant found that the method for processing a 3D model in the related art cannot quickly set the bone parameters in the 3D model. To address the above problem, the embodiments of the present application provide a method for processing a 3D model, which can improve the generation efficiency of the bone parameters in the 3D model.

[0073] The following describes the exemplary applications of the electronic device provided in the embodiments of the present application. The electronic device provided in the embodiments of the present application can be implemented as various types of terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart TV, a vehicle-mounted terminal, etc., or can be implemented as a server. Next, the exemplary applications will be described when the electronic device is implemented as a terminal or a server.

[0074] See Figure 1 , Figure 1 FIG. 100 is a schematic architecture diagram of a 3D model processing system 100 provided in the embodiments of the present application. To implement a 3D model processing application, the terminal 400 is connected to the server 200 through the network 300. The network 300 can be a wide area network, a local area network, or a combination of the two.

[0075] The method for processing a 3D model provided in the embodiments of the present application can be applied to various scenarios that require generating a 3D model, such as character customization, medical image analysis, etc. The following is an example.

[0076] 1) Online character customization: The terminal 400 is used to upload a first 3D model and a second 3D model. The server 200 is used to apply the bone parameters of the first vertex of the first 3D model to the corresponding second vertex to generate a third 3D model. The terminal 400 can view and download the customized third 3D model in the human-computer interaction interface 410.

[0077] 2) Virtual fitting platform: The terminal 400 is used to upload a 3D human body model corresponding to the target object (i.e., the second 3D model). The server 200 performs feature matching between the 3D human body model and the clothing model (i.e., the first 3D model), and applies the bone parameters of the first vertex of the 3D human body model to the corresponding second vertex in the clothing model to generate a third 3D model wearing the clothing. The terminal 400 can view the fitting effect in the human-computer interaction interface 410 and select a suitable piece of clothing.

[0078] 3) Game character synchronization: The terminal 400 is used to create or select a game character model of the target object (i.e., the second 3D model). The server 200 performs feature matching between the game character model and the standard character model in the game (i.e., the first 3D model), and associates the bone parameters of the standard character to the vertices of the player model to generate a character model in the game (i.e., the third 3D model). The terminal 400 can view and control the game character of the target object in the human-computer interaction interface 410 to play the game.

[0079] 4) Medical image analysis, terminal: The terminal 400 is used to upload the medical image data of the patient and generate a second 3D model. The server 200 performs feature matching between the second 3D model of the patient and the standard model (i.e., the first 3D model), associates the bone parameters of the standard model to the vertices of the second 3D model of the patient, and generates a third 3D model that can be used for surgical planning. The terminal 400 can view the analysis results in the human-computer interaction interface 410 and perform surgical planning.

[0080] The terminal 400 is used to display the first 3D model and the second 3D model in the human-computer interaction interface 410. The server 200 is used to apply the bone parameters of each first vertex in the first 3D model to the corresponding second vertex to obtain a third 3D model.

[0081] In some embodiments, the server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, which are not limited in the embodiments of the present application.

[0082] See Figure 2 , Figure 2 is a schematic structural diagram of the electronic device 500 provided by the embodiments of the present application. Figure 2 The illustrated electronic device 500 can be Figure 1 the terminal 400 or the server 200 in Figure 2 . The electronic device 500 includes: at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. Each component in the terminal 500 is coupled together through a bus system 540. It can be understood that the bus system 540 is used to implement the connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in

[0083] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0084] The user interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, and other input buttons and controls.

[0085] The memory 550 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, etc. The memory 550 optionally includes one or more storage devices that are physically remote from the processor 510.

[0086] The memory 550 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0087] In some embodiments, the memory 550 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustrated below.

[0088] An operating system 551, including system programs for handling various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and handling hardware-based tasks;

[0089] A network communication module 552 for reaching other electronic devices via one or more (wired or wireless) network interfaces 520. Exemplary network interfaces 520 include: Bluetooth, Wi-Fi (Wireless Fidelity), and Universal Serial Bus (USB), etc.;

[0090] A presentation module 553 for enabling the presentation of information (such as a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 associated with the user interface 530 (such as a display screen, a speaker, etc.);

[0091] An input processing module 554 for detecting and translating one or more user inputs or interactions from one of the one or more input devices 532.

[0092] In some embodiments, the device provided by the embodiments of the present application can be implemented in software. Figure 2 Fig. shows a processing device 555 for a three-dimensional model stored in a memory 550, which can be software in the form of a program and a plug-in, etc., including the following software modules: a model acquisition module 5551, a feature extraction module 5552, a vertex matching module 5553, and a model generation module 5554. These modules are logical, and thus can be combined arbitrarily or further split according to the functions to be implemented. The functions of each module will be described below.

[0093] In some embodiments, a terminal or a server can implement the method for processing a three-dimensional model provided by the embodiments of the present application by running various computer-executable instructions or computer programs. For example, the computer-executable instructions can be commands at the microprogram level, machine instructions, or software instructions. The computer program can be a native program or a software module in an operating system; it can be a local (Native) application (APPlication, APP), that is, a program that needs to be installed in an operating system to run, such as a game APP or a video APP; it can also be a small program that can be embedded in any APP, that is, a program that only needs to be downloaded to a browser environment to run. In short, the above computer-executable instructions can be instructions in any form, and the above computer programs can be application programs, modules, or plug-ins in any form.

[0094] Next, the method for processing a three-dimensional model provided by the embodiments of the present application will be described. As mentioned above, the electronic device implementing the method for processing a three-dimensional model provided by the embodiments of the present application can be a terminal or a server, or a combination of both. Therefore, the execution subject of each step will not be repeated below.

[0095] See Figure 3A , Figure 3A which is a first flowchart of the method for processing a three-dimensional model provided by the embodiments of the present application and can be executed by the above terminal or server, and will be described in combination with the steps shown in Figure 3A .

[0096] In step 101, a first three-dimensional model and a second three-dimensional model for characterizing the same object are obtained, where a first vertex in the first three-dimensional model has an association relationship with a preset bone parameter in the bone.

[0097] In some embodiments, the first 3D model is a pre-generated standard model. Each first vertex of the first 3D model has an associated relationship with preset bone parameters. The bone parameters include bone positions and bone weights. That is to say, the first 3D model has completed bone binding and skinning. Among them, bone binding refers to the process of creating an internal bone structure for the 3D model and associating the geometric mesh of the 3D model with the bone framework. This structure can be composed of a series of joints (bones), and these joints are connected through a parent-child relationship to form a skeleton; skinning refers to associating the geometric surface of the 3D model with the skeleton bound by the bones, so that when the bones move, the surface of the 3D model can be deformed accordingly, thereby realizing the deformation and animation of the model. The skinning process involves assigning one or more influence weights (weights) of bones, that is, bone weights, to each vertex of the 3D model. These weights determine how the vertex follows the bones when the bones move. Skinning can be implemented through the Linear Blend Skinning (LBS) algorithm, which calculates the position and normal of the vertex in each frame according to the bone weights of each vertex and the transformation matrix of the bones. The second 3D model can be a 3D model designed by the user according to business requirements. Each second vertex of the second 3D model has no associated relationship with the preset bone parameters. That is to say, the second 3D model has not completed bone binding and skinning.

[0098] For example, refer to Figure 4 , Figure 4 is a schematic diagram of the first 3D model and the second 3D model provided by the embodiments of the present application. Figure 4 The left figure of Figure 4 shows the first 3D model,

[0099] In some embodiments, refer to Figure 3B , Figure 3B is the second process schematic diagram of the 3D model processing method provided by the embodiments of the present application. Before step 101, perform Figure 3B steps 201 to 206 of

[0100] In step 201, a preset model is obtained, where the preset model is used to represent a sample object, and the sample object is an object different from the object represented by the first 3D model.

[0101] In some embodiments, the preset model can be manually created using 3D modeling software or generated by scanning real-world objects.

[0102] Exemplarily, creating the preset model manually can be achieved using 3D modeling tools, such as 3D graphics and image software (Blender), 3D animation software (Maya), or 3D modeling, rendering, and production software (Three Dimensions StudioMax, 3DS Max), to construct the geometric shape of the model through techniques such as polygon modeling, Non-Uniform Rational B-Splines (NURBS) modeling, or sculpting. If the preset model is generated by scanning, a three-dimensional (3D) scanner can be used to capture the surface geometry and texture of the object, and then software can be used to convert the scan data into an editable 3D model.

[0103] In step 202, sample object features are extracted from the sample object.

[0104] In some embodiments, the sample object features include at least one of the following: geometric features, texture features, and semantic features. The extraction of sample object features can be automatically completed through deep learning techniques. For example, convolutional neural networks (CNNs) can be used to extract geometric features and texture features. For geometric features, a 3D model can be converted into a point cloud or voxel grid representation. A point cloud convolutional network can be used to process the position information, as well as the normal and color information, of each point. For a voxel grid, a 3D convolutional network can be used to capture spatial information. For texture features, texture feature extraction involves understanding the visual appearance of the surface of a 3D model, such as color, pattern, and texture orientation, which can be achieved by processing a texture image associated with the 3D model. The texture image can be a 2D image of a UV unwrap or an image generated from the 3D model through rendering. Conventional 2D convolutional networks (such as the Visual Geometry Group (VGG), Residual Network (ResNet), or Inception Network) are used to extract texture features from the texture image. For semantic features, through a semantic segmentation task, a class label is assigned to each vertex or voxel. A semantic segmentation network, such as a convolutional neural network for biomedical image segmentation (U-Net) or a deep learning model for object detection and instance segmentation (Mask Region-based Convolutional Neural Network, Mask R-CNN), is used to identify and label different parts of the model.

[0105] Exemplarily, geometric features include vertex positions and normal directions, texture features include colors and patterns, and semantic features include object categories and part relationships, where part relationships refer to the structural and functional connections between the various components of the sample object.

[0106] In step 203, a generator is called based on the sample object features to obtain a prediction model.

[0107] In some embodiments, the generator is part of a Generative Adversarial Networks (GAN) and is used to generate virtual data similar to the real data distribution. By inputting the sample object features into the generator, the sample object features are dimensionally augmented to obtain a prediction model.

[0108] Exemplarily, the fully connected layer of the generator converts the sample object features into a feature map, where the feature map includes the basic shape and structural information of the prediction model; the deconvolution layer and the upsampling layer of the generator are used to increase the dimension of the feature map to obtain the prediction model, where the prediction model can be represented by 3D model data in the form of point clouds, meshes, or other forms.

[0109] In step 204, the discriminator determines the loss between the prediction model and the preset model.

[0110] In some embodiments, the discriminator is a classifier used to distinguish between real data and virtual data generated by the generator. The loss between the prediction model and the preset model can be determined through a loss function.

[0111] Exemplarily, the loss function can be a Binary Cross-Entropy Loss (BCELoss), a Mean Squared Error (MSE) loss function, or a perceptual loss function. Taking the mean squared error loss function as an example, the square of the difference between the prediction model and the preset model is determined, the number of sample objects is counted, the sum of the squares corresponding to multiple sample objects is determined, and the ratio of the sum to the number is determined as the loss between the prediction model and the preset model.

[0112] In step 205, the parameters of the generator and the discriminator are updated based on the loss to obtain a pre-trained generator and a pre-trained discriminator.

[0113] In some embodiments, the backpropagation algorithm is used to calculate the gradients of the loss function with respect to the parameters of the generator and the discriminator. The optimization algorithm is used to update the parameters of the generator and the discriminator according to the calculated gradients. During the training process, the generator and the discriminator are alternately trained. First, the parameters of the generator are fixed, and the discriminator is trained to improve its ability to distinguish between real data and virtual data. Then, the parameters of the discriminator are fixed, and the generator is trained to improve its ability to generate virtual data whose similarity to real data is greater than the similarity threshold. Training stops until the loss functions of the generator and the discriminator reach a balanced state, or the generated prediction model reaches the expected quality level, and the prediction model is obtained.

[0114] Exemplarily, the optimization algorithm can be any of the following: Stochastic Gradient Descent (SGD), Adaptive Moment Estimation (Adam), Conjugate Gradient Method. Taking Adam as an example, the first-order moment and the second-order moment are determined based on the time step and the gradient, the first-order moment and the second-order moment are bias-corrected, and the parameters of the generator and the discriminator are updated through the corrected first-order moment and second-order moment.

[0115] In step 206, based on the object features of the object, the pre-trained generator and the pre-trained discriminator are called to generate the first three-dimensional model.

[0116] Here, the geometric features, texture features, and semantic features of the object are extracted, the geometric features, texture features, and semantic features are combined into object features, and the object features are input into the pre-trained and pre-trained discriminator to obtain the first three-dimensional model.

[0117] In the embodiments of the present application, by automatically learning the sample object features from the sample objects and generating the first three-dimensional model, the time and cost of manual modeling are reduced. The generator can adapt to different three-dimensional model types and features, has strong flexibility, and can be applied to various different objects and scenarios. By using the discriminator to evaluate the quality of the generated model and optimizing the performance of the generator and the discriminator based on the loss function, this helps to improve the quality and stability of the generated model.

[0118] Continue to refer to Figure 3A , in step 102, the first feature vector of each first vertex in the first three-dimensional model and the second feature vector of each second vertex in the second three-dimensional model are extracted.

[0119] In some embodiments, the first feature vector characterizes the depth feature of the first vertex, the second feature vector characterizes the depth feature of the second vertex, and the depth feature refers to the high-level abstract feature representation automatically learned by the deep neural network. The depth feature can be extracted from the vertices of the three-dimensional model and can capture the complex structure and semantic information of the data.

[0120] Exemplarily, the depth features include at least one of the following: geometric features, such as the three-dimensional coordinates, normal direction, curvature, and geometric relationship of adjacent points of each vertex; semantic features, such as semantic information such as the body part, function, or use to which each vertex belongs; context features, such as the position and role of each vertex in the overall model, and the context relationship with other points; animation features, such as the motion information of each vertex, such as speed, acceleration, and motion trajectory; texture features, such as the texture color, texture gradient, and texture pattern of each vertex.

[0121] In some embodiments, referring to Figure 3C , Figure 3C is the third process schematic diagram of the method for processing a three-dimensional model provided by an embodiment of the present application. Figure 3A Step 102 of Figure 3C can be implemented by steps 1021 to 1022 of

[0122] and the following is a specific description.

[0123] Exemplarily, the neural network model can be a CNN, a graph neural network (GNN), a point cloud feature learning network (such as PointNet and PointNet++), an auto-encoder (AE), a variational auto-encoder (VAE), or a transformer.

[0124] In some embodiments, referring to Figure 3D , Figure 3D is the fourth process schematic diagram of the method for processing a three-dimensional model provided by an embodiment of the present application. Before step 1021, steps 301 to 305 of Figure 3D are executed and the following is a specific description.

[0125] In step 301, a sample three-dimensional model is obtained, and a first global sample vertex set is extracted from the sample three-dimensional model.

[0126] In some embodiments, all vertices of the sample three-dimensional model are traversed, and all traversed vertices are combined into a first global sample vertex set.

[0127] Exemplarily, the sample three-dimensional model includes multiple first global sample vertices, such as first global sample vertex A(x1, y1), first global sample vertex B(x2, y2), first global sample vertex C(x3, y3), first global sample vertex D(x4, y4), and the first global sample vertex set is {(x1, y1), (x2, y2), (x3, y3), (x4, y4)}.

[0128] In step 302, the first global sample vertex set is transformed to obtain a second global sample vertex set.

[0129] Here, each first global sample vertex in the first global sample vertex set is transformed to obtain a second global sample vertex corresponding to each first global sample vertex, and multiple second global sample vertices are combined into a second global sample vertex set.

[0130] In some embodiments, step 302, "transform the first global sample vertex set to obtain a second global sample vertex set", can be implemented by performing the following: for each first global sample vertex in the first global sample vertex set, rotate the first global sample vertex to obtain a second global sample vertex corresponding to the first global sample vertex, and combine the multiple second global sample vertices into a second global sample vertex set.

[0131] In some embodiments, the rotation angle can be one or more random values or values in an arithmetic sequence. One first global sample vertex corresponds to one second global sample vertex.

[0132] Exemplarily, refer to Figure 5A , Figure 5A which is a schematic diagram of the rotation of global sample vertices provided by an embodiment of the present application. Figure 5A The left diagram of Figure 5A shows multiple first global sample vertices in the first global sample vertex set before rotation, such as the first global sample vertex A, the first global sample vertex B, and the first global sample vertex C. After rotating the multiple first global sample vertices clockwise by 90 degrees simultaneously, second global sample vertices corresponding to each first global sample vertex are obtained. Refer to

[0133] the right diagram of

[0134] In some embodiments, step 302, "transform the first global sample vertex set to obtain a second global sample vertex set", can be implemented by performing the following: for each first global sample vertex in the first global sample vertex set, translate the first global sample vertex to obtain a second global sample vertex corresponding to the first global sample vertex, and combine the multiple second global sample vertices into a second global sample vertex set.

[0135] Exemplarily, refer to Figure 5B , Figure 5B which is a schematic diagram of the translation of global sample vertices provided by an embodiment of the present application. Figure 5BThe left figure shows multiple first global sample vertices in the first global sample vertex set before translation, such as the first global sample vertex A, the first global sample vertex B, and the first global sample vertex C. After translating the multiple first global sample vertices by a preset distance simultaneously, the second global sample vertices corresponding to each first global sample vertex are obtained. Refer to Figure 5B the right figure. For example, the first global sample vertex A corresponds to the second global sample vertex A’, the first global sample vertex B corresponds to the second global sample vertex B’, and the first global sample vertex C corresponds to the second global sample vertex C’. The second global sample vertices A’, B’, and C’ are combined into a second global sample vertex set.

[0136] Continue to refer to Figure 3D , in step 303, based on the first global sample vertex set, the initialized neural network model is called for feature extraction to obtain the first global features of each first global sample vertex in the first global sample vertex set. Based on the second global sample vertex set, the initialized neural network model is called for feature extraction to obtain the second global features of each second global sample vertex in the second global sample vertex set.

[0137] In some embodiments, the first global feature represents the depth feature of the first global sample vertex, and the second global feature represents the depth feature of the second global sample vertex. The depth feature includes at least one of the following: geometric feature, semantic feature, context feature, animation feature, texture feature.

[0138] Exemplarily, the geometric feature includes the three-dimensional coordinates, normal direction, curvature, and geometric relationship of adjacent points of each vertex; the semantic feature includes semantic information such as the body part, function, or use to which each vertex belongs; the context feature includes the position and role of each vertex in the overall model, as well as the context relationship with other points; the animation feature includes the motion information of each vertex, such as speed, acceleration, motion trajectory, etc.; the texture feature includes the texture color, texture gradient, and texture pattern of each vertex.

[0139] In step 304, for each first global sample vertex, the target second global sample vertex corresponding to the first global sample vertex is determined. A global feature pair is constructed based on the first global feature of the first global sample vertex and the second global feature of the target second global sample vertex. Based on the global feature pair, a preset global loss function is called to obtain the global loss.

[0140] Continuing with the example in step 302 above, for each first global sample vertex A, determine that the target second global sample vertex corresponding to the first global sample vertex is the second global sample vertex A'. Based on the first global feature a of the first global sample vertex A and the second global feature a' of the second global sample vertex A', construct a global feature pair <a, a'>, and call a preset global loss function based on the global feature pair <a, a'> to obtain the global loss.

[0141] In some embodiments, referring to Figure 3E , Figure 3E is the fifth process schematic diagram of the three-dimensional model processing method provided by the embodiments of the present application. Figure 3D Step 304 of " Figure 3E calling a preset global loss function based on the global feature pair to obtain the global loss" can be implemented by

[0142] In step 3041A, for each first global sample vertex, determine the exponent of the product of the first global feature and the second global feature in the global feature pair.

[0143] In some embodiments, the exponent of the product of the first global feature and the second global feature is used to construct a non-linear relationship between the first global feature and the second global feature, and the exponential function can amplify or reduce the difference between the features.

[0144] Continuing with the example in step 304 above, in the global feature pair <a, a'>, the product of the first global feature a and the second global feature a' is a·a', and the exponent of the product is e^(a·a').

[0145] In step 3042A, determine the sum of the exponents corresponding to multiple first global features.

[0146] In some embodiments, add the exponent values corresponding to the first global features of all vertices to obtain the sum of the exponents, and the sum of the exponents reflects the overall influence of the first global features of all vertices.

[0147] Continuing with the examples in step 302 and step 3041A above, the multiple first global features are the first global feature a, the first global feature b, and the first global feature c respectively. The second global feature corresponding to the first global feature a is a', the exponent corresponding to the first global feature a is e^(a·a'), the second global feature corresponding to the first global feature b is b', the exponent corresponding to the first global feature b is e^(b·b'), the second global feature corresponding to the first global feature c is c', and the exponent corresponding to the first global feature c is e^(c·c'). Then the sum of the exponents S is e^(a·a') + e^(b·b') + e^(c·c').

[0148] In step 3043A, determine the ratio of the exponent to the sum of exponents.

[0149] In some embodiments, the ratio of the exponent to the sum of exponents represents the contribution of the characteristic product of a vertex relative to the first global characteristic exponent sum of all vertices.

[0150] Continuing with the example of steps 3041A and 3042A above, the product of the first global characteristic a and the second global characteristic a' is a·a', the sum of exponents S is e^(a·a') + e^(b·b') + e^(c·c'), then the ratio K1 of the exponent to the sum of exponents is (a·a') / S = (a·a') / (e^(a·a') + e^(b·b') + e^(c·c')).

[0151] In step 3044A, determine the first sum of the ratios corresponding to multiple first global characteristics respectively.

[0152] In some embodiments, the first sum of the ratios corresponding to multiple first global characteristics respectively reflects the total contribution of the characteristic products of all vertices relative to the first global characteristic exponent sum of the whole.

[0153] Continuing with the example of step 3043A above, the ratio corresponding to the first global characteristic a is K1, if the ratio corresponding to the first global characteristic b is K2, and the ratio corresponding to the first global characteristic c is K3, then the first sum is K1 + K2 + K3.

[0154] In step 3045A, determine the first ratio of the first sum to the number of multiple first global sample vertices, and use the first ratio as the global loss.

[0155] In some embodiments, the global loss is an average value, representing the average contribution of the characteristic product of each vertex relative to the first global characteristic exponent sum of the whole. By minimizing the global loss, the model can learn the parameters that optimize the relationship between the characteristic product and the first global characteristic exponent sum.

[0156] Continuing with the example of step 3044A above, the first sum is K1 + K2 + K3, and the number of multiple first global sample vertices is 3, then the first ratio is (K1 + K2 + K3) / 3, to be used as the global loss.

[0157] In the embodiments of the present application, by using an exponential function, the loss function can capture the non-linear relationship between global features, enabling the model to better learn and represent complex data distributions. The loss function performs a normalization process by taking the ratio of the exponent of each vertex to the sum of the exponents of all vertices, which helps to eliminate the scale differences in the data and makes the model more robust to features of different scales. The final global loss is the average of the losses of all vertices, reducing the sensitivity of the model to outliers and improving the stability and generalization ability of the model.

[0158] In some embodiments, referring to Figure 3F , Figure 3F is the sixth process schematic diagram of the method for processing a three-dimensional model provided by the embodiments of the present application. Figure 3D Step 304 of " Figure 3F obtaining the global loss by calling a preset global loss function based on the global features" can also be implemented through

[0159] Steps 3041B to 3043B of

[0160] are specifically described below. In step 3041B, the difference between the first global feature and the second global feature in each global feature pair is determined.

[0161] For example, there are three global feature pairs, namely global feature pair 1 (a·a') composed of the first global feature a and the second global feature a', global feature pair 2 (b·b') composed of the first global feature b and the second global feature b', and global feature pair 3 (c·c') composed of the first global feature c and the second global feature c'. Among them, the difference between the first global feature a and the second global feature a' in global feature pair 1 is a - a', the difference between the first global feature b and the second global feature b' in global feature pair 2 is b - b', and the difference between the first global feature c and the second global feature c' in global feature pair 3 is c - c'.

[0161] In step 3042B, the second sum of the differences corresponding to the multiple global feature pairs is determined.

[0162] Continuing with the example in step 3041B above, if the difference corresponding to global feature pair 1 is a - a', the difference corresponding to global feature pair 2 is b - b', and the difference corresponding to global feature pair 3 is c - c', then the second sum H is (a - a')+(b - b')+(c - c').

[0163] In step 3043B, the second ratio of the second sum to the number of the multiple global feature pairs is determined, and the second ratio is used as the global loss.

[0164] Continuing with the example in step 3042B above, the second sum is H, the number of multiple global feature pairs is 3, and the second ratio of the second sum to the number of multiple global feature pairs, H / 3 = ((a - a') + (b - b') + (c - c')) / 3, is used as the global loss.

[0165] In the embodiments of the present application, by directly calculating the difference between global features, the loss function can capture the linear relationship between features. By summing and averaging the differences, the loss function can reduce the impact of outliers on the overall loss and improve the robustness of the model. Since the calculation process of the loss function involves simple addition and division operations, the calculation efficiency is improved, which helps the model to quickly converge to the optimal solution.

[0166] Continuing to refer to Figure 3D , in step 305, based on the global loss, the parameters of the initialized neural network model are updated to obtain a pre-trained neural network model.

[0167] In some embodiments, the global loss is backpropagated to update the parameters of the initialized neural network model. The process of repeatedly calculating the global loss and updating the parameters is iterated until the global loss converges, and the iteration process is stopped to form a pre-trained neural network model.

[0168] Exemplarily, backpropagation is implemented through the backpropagation algorithm. The gradient of each neuron is calculated from the output layer to the input layer, and the weights and biases of the neurons are updated according to the gradient. The parameters are continuously updated in the form of gradient descent to reduce the loss value. Gradient descent can adopt various gradient descent algorithms, such as batch gradient descent algorithm, stochastic gradient descent algorithm, adaptive gradient descent algorithm, and momentum gradient descent algorithm.

[0169] In the embodiments of the present application, by transforming the sample 3D model to generate a second global sample vertex set, the diversity of training data can be increased, and the recognition ability of the neural network model from different perspectives or transformations can be improved. Using the global loss function to optimize the global feature pairs can guide the neural network model to learn more discriminative global features, which helps to improve the performance of the model in tasks such as 3D model recognition and classification. Before the start of training, by preprocessing the sample data and extracting features, better initial parameters can be provided for the model, which helps the model to converge faster and improve the training efficiency.

[0170] In some embodiments, referring to Figure 3G , Figure 3G is the seventh process schematic diagram of the 3D model processing method provided by the embodiments of the present application. After step 305, the steps 306 to 310 of Figure 3G are executed, which will be specifically described below.

[0171] In step 306, the sample three-dimensional model is cut into multiple sub-sample three-dimensional models, and a first local sample vertex set is extracted from each sub-sample three-dimensional model.

[0172] In some embodiments, any of the following cutting methods can be used to cut the sample three-dimensional model into multiple sub-sample three-dimensional models: plane cutting, using one or more planes to cut the sample three-dimensional model, where the plane can be horizontal, vertical, or inclined at any angle, and the sub-sample three-dimensional model can be the upper part, lower part, front part, rear part, etc. of the sample three-dimensional model; region cutting, defining a three-dimensional region (such as a cube, sphere, cylinder, etc.) to cut the sample three-dimensional model; feature-based cutting, defining a cutting region according to the geometric features (such as edges, corners, curved surfaces, etc.) of the sample three-dimensional model, and cutting the sample three-dimensional model according to the cutting region; random cutting, randomly generating a cutting plane or region to cut the sample three-dimensional model; defining a cutting plane according to the symmetry of the sample three-dimensional model. For example, for a symmetric sample three-dimensional model, it can be cut along the axis of symmetry to obtain two symmetric sub-sample three-dimensional models; hierarchical cutting, gradually cutting each layer of the sample three-dimensional model; view-point-based cutting, cutting the sample three-dimensional model from different viewpoints to generate sub-sample three-dimensional models with different perspectives; component-based cutting, defining a cutting region according to the components of the sample three-dimensional model. For example, for a human body model, components such as the head, torso, and limbs can be cut out.

[0173] Exemplarily, refer to Figure 6 , Figure 6 is a schematic diagram of segmenting a sample three-dimensional model provided by an embodiment of the present application. In Figure 6 the left figure of, the sample three-dimensional model is a pentagon with a total of 5 vertices. In Figure 6 the middle figure of, the sample three-dimensional model is cut into multiple sub-sample three-dimensional models, such as sub-sample three-dimensional model 1, sub-sample three-dimensional model 2, and sub-sample three-dimensional model 3. In Figure 6 the right figure of, the first local sample vertex set extracted from sub-sample three-dimensional model 1 is {A, B, C}, the first local sample vertex set extracted from sub-sample three-dimensional model 2 is {A, C, E}, and the first local sample vertex set extracted from sub-sample three-dimensional model 3 is {C, D, E}.

[0174] In step 307, each first local sample vertex set is respectively transformed to obtain multiple second local sample vertex sets.

[0175] In some embodiments, step 307 can be implemented by performing any of the following processes: For each first local sample vertex in the first local sample vertex set, rotate the first local sample vertex to obtain a second local sample vertex corresponding to the first local sample vertex, and combine the multiple second local sample vertices into a second local sample vertex set; For each first local sample vertex in the first local sample vertex set, translate the first local sample vertex to obtain a second local sample vertex corresponding to the first local sample vertex, and combine the multiple second local sample vertices into a second local sample vertex set.

[0176] Exemplarily, the rotation angle can be one or more random values or values in an arithmetic sequence. Taking the first local sample vertex set {A, B, C} corresponding to the sub-sample three-dimensional model 1 as an example, see Figure 7A , Figure 7A which is a schematic diagram of the rotation of local sample vertices provided by an embodiment of the present application. Figure 7A The left diagram of shows multiple first local sample vertices in the first local sample vertex set before rotation. For example, the first local sample vertex A, the first local sample vertex B, and the first local sample vertex C. After rotating the multiple first local sample vertices clockwise by 90 degrees simultaneously, second local sample vertices corresponding to each first local sample vertex are obtained. See Figure 7A the right diagram of . For example, the first local sample vertex A corresponds to the second local sample vertex A', the first local sample vertex B corresponds to the second local sample vertex B', and the first local sample vertex C corresponds to the second local sample vertex C'. Combine the second local sample vertices A', B', and C' into a second local sample vertex set.

[0177] Exemplarily, the translation distance can be one or more random values or one or more values in an arithmetic sequence. Taking the first local sample vertex set {A, B, C} corresponding to the sub-sample three-dimensional model 1 as an example, see Figure 7B , Figure 7B which is a schematic diagram of the translation of local sample vertices provided by an embodiment of the present application. Figure 7B The left diagram of shows multiple first local sample vertices in the first local sample vertex set before translation. For example, the first local sample vertex A, the first local sample vertex B, and the first local sample vertex C. After translating the multiple first local sample vertices by a preset distance simultaneously, second local sample vertices corresponding to each first local sample vertex are obtained. See Figure 7BThe right figure, for example, the first local sample vertex A corresponds to the second local sample vertex A', the first local sample vertex B corresponds to the second local sample vertex B', and the first local sample vertex C corresponds to the second local sample vertex C'. The second local sample vertices A', B', and C' are combined into a second local sample vertex set.

[0178] Here, for each set of first local sample vertices, the following steps 308 to 310 are performed.

[0179] In step 308, based on the set of first local sample vertices, a pre-trained neural network model is called for feature extraction to obtain the first local features of each first local sample vertex in the set of first local sample vertices. Based on the set of second local sample vertices, a pre-trained neural network model is called for feature extraction to obtain the second local features of each second local sample vertex in the set of second local sample vertices.

[0180] In some embodiments, the first local feature represents the depth feature of the first local sample vertex, and the second local feature represents the depth feature of the second local sample vertex. The depth feature includes at least one of the following: geometric feature, semantic feature, context feature, animation feature, texture feature. The pre-trained neural network model can be obtained through the above steps 301 to 305 and will not be elaborated here.

[0181] Exemplarily, the geometric feature includes the three-dimensional coordinates, normal direction, curvature, geometric relationship of adjacent points, etc. of each vertex; the semantic feature includes semantic information such as the body part, function or use to which each vertex belongs; the context feature includes the position and role of each vertex in the overall model, and the context relationship with other points; the animation feature includes the motion information of each vertex, such as speed, acceleration, motion trajectory, etc.; the texture feature includes the texture color, texture gradient, texture pattern, etc. of each vertex.

[0182] In step 309, for each first local sample vertex, the target second local sample vertex corresponding to the first local sample vertex is determined. Based on the first local feature of the first local sample vertex and the second local feature of the target second local sample vertex, a local feature pair is constructed. Based on the local feature pair, a preset local loss function is called to determine the local loss.

[0183] Continuing with the example of step 307 above, for each first local sample vertex A, the target second local sample vertex corresponding to the first local sample vertex is determined to be the second local sample vertex A'. Based on the first local feature m of the first local sample vertex A and the second local feature m' of the second local sample vertex A', a local feature pair <m, m'> is constructed. Based on the local feature pair <m, m'>, a preset local loss function is called to obtain the local loss.

[0184] In some embodiments, step 309, "invoking a preset local loss function based on local feature pairs to determine local loss", may be implemented by performing the following processing: for each first local sample vertex, determining the exponent of the product of the first local feature and the second local feature in the local feature pair; determining the exponential sum of the exponents corresponding to the multiple first local features; determining the ratio of the exponent to the exponential sum; determining the third sum of the ratios corresponding to the multiple first local features; determining the third ratio of the third sum to the number of the multiple first local sample vertices, and taking the third ratio as the local loss.

[0185] For example, in the local feature pair <m, m'>, the product of the first local feature m and the second local feature m' is m·m', and the exponent of the product is e^(m·m'). The multiple first local features are the first local feature m, the first local feature n, and the first local feature p respectively. The second local feature corresponding to the first local feature m is m', the exponent corresponding to the first local feature m is e^(m·m'), the second local feature corresponding to the first local feature n is n', the exponent corresponding to the first local feature n is e^(n·n'), the second local feature corresponding to the first local feature p is p', and the exponent corresponding to the first local feature p is e^(p·p'). Then the exponential sum E is e^(m·m') + e^(n·n') + e^(p·p'). The ratio F1 of the exponent to the exponential sum is (m·m') / E = (m·m') / (e^(m·m') + e^(n·n') + e^(p·p')). If the ratio corresponding to the first local feature n is F2 and the ratio corresponding to the first global feature p is F3, then the third sum is F1 + F2 + F3. The number of the multiple first local sample vertices is 3, so the third ratio is (F1 + F2 + F3) / 3, which is used as the local loss.

[0186] In some embodiments, step 309, "invoking a preset local loss function based on local feature pairs to determine local loss", may also be implemented by performing the following processing: determining the feature difference between the first local feature and the second local feature in each local feature pair; determining the fourth sum of the feature differences corresponding to the multiple local feature pairs; determining the fourth ratio of the fourth sum to the number of the multiple local feature pairs, and taking the fourth ratio as the local loss.

[0187] Exemplarily, there are a total of three local feature pairs, namely local feature pair 1 (m·m’) composed of the first local feature m and the second local feature m’, local feature pair 2 (n·n’) composed of the first local feature n and the second local feature n’, and local feature pair 3 (p·p’) composed of the first local feature p and the second local feature p’. Among them, the feature difference between the first local feature m and the second local feature m’ in local feature pair 1 is m - m’, the feature difference between the first local feature n and the second local feature n’ in local feature pair 2 is n - n’, and the feature difference between the first local feature p and the second local feature p’ in local feature pair 3 is p - p’. Then the fourth sum G is (m - m’) + (n - n’) + (p - p’). The number of multiple local feature pairs is 3, and the fourth ratio G / 3 of the fourth sum to the number of multiple local feature pairs is ((m - m’) + (n - n’) + (p - p’)) / 3, which is used as the local loss.

[0188] In step 310, based on the local loss, update the parameters of the pre-trained neural network model to obtain the updated neural network model.

[0189] In some embodiments, backpropagate the local loss to update the parameters of the pre-trained neural network model. The process of calculating the local loss and updating the parameters iteratively for multiple times until the local loss converges, and then stop the iterative process to form the updated neural network model.

[0190] Exemplarily, backpropagation is implemented through the backpropagation algorithm. Calculate the gradient of each neuron from the output layer to the input layer, and update the weights and biases of the neurons according to the gradient. Continuously update the parameters in the way of gradient descent to reduce the loss value. Gradient descent can adopt various gradient descent algorithms, such as batch gradient descent algorithm, stochastic gradient descent algorithm, adaptive gradient descent algorithm, and momentum gradient descent algorithm.

[0191] In the embodiments of the present application, by cutting the sample three-dimensional model into multiple sub-sample three-dimensional models and extracting the local sample vertex sets from each sub-model, the neural network model can learn the local features of different parts, improving the recognition ability of the neural network model for the local details of the three-dimensional model. By further training on the basis of the pre-trained neural network model, the neural network model can gradually optimize its parameters and improve its performance on specific tasks.

[0192] Continue to refer to Figure 3C , in step 1022, based on the coordinates of multiple second vertices in the second three-dimensional model, call the pre-trained neural network model to extract the second feature vectors of each second vertex.

[0193] Exemplarily, the neural network model can be any one of the following: CNN, GNN, PointNet, PointNet++, AE, VAE, Transformers.

[0194] Continue to refer to Figure 3A , in step 103, based on the first feature vector and the second feature vector, determine the second vertex corresponding to each first vertex.

[0195] In some embodiments, refer to Figure 3H , Figure 3H is the eighth process schematic diagram of the three-dimensional model processing method provided by the embodiments of the present application. Figure 3A Step 103 of Figure 3H can be implemented by steps 1031 to 1032 of

[0196] In step 1031, for each first vertex in the first three-dimensional model, determine the similarity between the first feature vector of the first vertex and the second feature vector of each second vertex.

[0197] In some embodiments, the similarity between the first feature vector of the first vertex and the second feature vector of each second vertex can be determined by any one of the following algorithms: cosine similarity, Euclidean distance, Manhattan distance, or Pearson correlation coefficient.

[0198] Exemplarily, taking the Euclidean distance as an example, if the first feature vector of the first vertex is (x 1 , y 1 ), and the second feature vector of the second vertex is (x 2 , y 2 ), determine the distance between the first feature vector and the second feature vector, that is to be used as the similarity. Taking the cosine similarity as an example, if the first feature vector of the first vertex is m and the second feature vector of the second vertex is n, determine the dot product of the first feature vector and the second feature vector, determine the product of the lengths of the first feature vector and the second feature vector, determine the ratio of the dot product to the product of the lengths, to be used as the similarity. For example, the dot product of the first feature vector and the second feature vector is m*n. The length of the first feature vector is |m|, the length of the second feature vector is |n|, the product of the lengths is |m|*|n|, then the similarity is (m*n) / (|m|*|n|).

[0199] In step 1032, look up the second vertex corresponding to each first vertex from the preset relationship matrix, where each element in the relationship matrix corresponds to a similarity.

[0200] Exemplarily, refer to Figure 8 , Figure 8It is a schematic structural diagram of the relationship matrix provided by an embodiment of the present application. If there are a total of 4 first vertices, such as first vertex 1, first vertex 2, first vertex 3, and first vertex 4, and a total of 4 second vertices, such as second vertex 1, second vertex 2, second vertex 3, and second vertex 4, each row represents the similarity between the first eigenvector of a first vertex and the second eigenvectors of multiple second vertices, and each column represents the similarity between the second eigenvector of a second vertex and the first eigenvectors of multiple first vertices. For example, the similarity between first vertex 1 and second vertex 1 is 0.8, and the similarity between first vertex 2 and second vertex 1 is 0.7.

[0201] In some embodiments, the elements of each row of the relationship matrix represent the similarity between the first eigenvector of any one first vertex and the second eigenvectors of multiple second vertices. Refer to Figure 3I , Figure 3I It is the ninth process schematic diagram of the three-dimensional model processing method provided by an embodiment of the present application. Figure 3H For step 1032 "Find the second vertex corresponding to each first vertex from the preset relationship matrix" of Figure 3I , it can be implemented by executing steps 10321 to 10324 of

[0202] Here, steps 10321 to 10324 of the following text Figure 3I are described by taking the first row of the preset relationship matrix as an example.

[0203] In step 10321, the largest element in the row is used as the target element.

[0204] Continuing with the example of step 1032 above, the largest element in the first row of the preset relationship matrix is 0.8, and 0.8 is used as the target element.

[0205] In step 10322, the first vertex corresponding to the row is used as the target first vertex.

[0206] Continuing with the example of step 1032 above, the first row corresponds to first vertex 1, so first vertex 1 is used as the target first vertex.

[0207] In step 10323, the second eigenvector corresponding to the target element is used as the target second eigenvector, and the second vertex corresponding to the target second eigenvector is determined.

[0208] Continuing with the example of step 10321 above, the target element 0.8 represents the similarity between the first eigenvector of first vertex 1 and the second eigenvector of second vertex 1. Then, the second eigenvector of second vertex 1 is used as the target second eigenvector, and second vertex 1 is the second vertex corresponding to the target second eigenvector.

[0209] In step 10324, the second vertex corresponding to the target second feature vector is used as the second vertex corresponding to the target first vertex.

[0210] Continuing with the example of step 10323 above, the second vertex 2 corresponding to the target second feature vector is used as the second vertex corresponding to the target first vertex (i.e., the first vertex 1).

[0211] In the embodiments of the present application, by constructing a relationship matrix and finding the element with the maximum similarity corresponding to each first vertex, the second vertex corresponding to each first vertex can be quickly determined, simplifying the process of determining the corresponding relationship and improving the accuracy of matching. The relationship matrix can adapt to large-scale data sets, and by calculating similarities in parallel and finding the maximum value, a large number of vertices and feature vectors can be efficiently processed.

[0212] Continue to refer to Figure 3A , in step 104, the bone parameters in the first three-dimensional model that are associated with each first vertex are associated with the second vertex corresponding to the first vertex, and the second three-dimensional model after association is used as the third three-dimensional model.

[0213] Here, for each first vertex, the bone parameters in the first three-dimensional model that are associated with the first vertex are applied to the second vertex corresponding to the first vertex in the second three-dimensional model, thereby updating the second three-dimensional model to obtain the third three-dimensional model.

[0214] In some embodiments, refer to Figure 3J , Figure 3J is the tenth process schematic diagram of the three-dimensional model processing method provided by the embodiments of the present application. Figure 3A For each first vertex in step 104, it can be implemented by executing Figure 3J steps 1041 to 1043 of

[0215] In step 1041, the bone position of the first vertex is applied to the second vertex corresponding to the first vertex.

[0216] Here, the relative position of the bone connected to the first vertex with respect to the overall bone is determined, and the second vertex corresponding to the first vertex reuses this relative position in the second three-dimensional model, that is, the relative position of the bone connected to the second vertex with respect to the overall bone is the same as the relative position corresponding to the first vertex.

[0217] In step 1042, the bone weight of the first vertex is applied to the second vertex corresponding to the first vertex.

[0218] In some embodiments, the bone weights of the first vertices characterize the degree to which the first vertices are affected by each bone. Each first vertex can be associated with multiple bones, and the degree of influence of each bone on the first vertex is represented by the bone weight. The higher the bone weight, the greater the influence of the bone on the first vertex.

[0219] For example, the first vertex A is connected to the bone B and the bone C. The bone weight corresponding to the bone B is 0.8, and the bone weight corresponding to the bone C is 0.5. The second vertex A corresponds to the first vertex A, and the second vertex B is connected to the bone B and the bone C. Then, in the second 3D model, for the second vertex A, the bone weight corresponding to the bone B is also 0.8, and the bone weight corresponding to the bone C is also 0.5.

[0220] In step 1043, based on the updated second vertices, the second 3D model is updated to a third 3D model.

[0221] Here, after assigning the bone positions and bone weights to each second vertex in the second 3D model, the updated second 3D model is used as the third 3D model.

[0222] In the embodiments of the present application, by transferring the bone parameters from the first 3D model to the second 3D model, the bone animation of the first model can be retained, enabling the second model to perform similar animation performances. It can improve the compatibility between different 3D models, enabling the second 3D model without bone binding and skinning to utilize the bone parameters of the first 3D model that has been bone-bound and skinned. By transferring the bone parameters, the animation production process of the new model is simplified, repetitive labor is reduced, and the work efficiency of 3D animation production is improved.

[0223] In some embodiments, refer to Figure 3K , Figure 3K is the eleventh process schematic diagram of the 3D model processing method provided by the embodiments of the present application. Before step 104, perform Figure 3K steps 401 to 404 as follows for specific description.

[0224] In step 401, a first global vertex set is sampled from the first 3D model, where the first global vertex set includes multiple first vertices obtained by globally sampling the first 3D model.

[0225] In some embodiments, global sampling refers to selecting a set of representative first vertices from all the first vertices of the first 3D model, and these first vertices can reflect the overall shape and characteristics of the first 3D model. The sampling method can be uniform sampling, region-based sampling, feature-based sampling, etc.

[0226] For example, the multiple first vertices obtained by globally sampling the first 3D model are respectively the first vertex 1, the first vertex 2, the first vertex 3, and the first vertex 4. Then the first global vertex set is {the first vertex 1, the first vertex 2, the first vertex 3, the first vertex 4}.

[0227] In step 402, a second global vertex set is sampled from the second 3D model, where the second global vertex set includes multiple second vertices obtained by globally sampling the second 3D model.

[0228] For example, the multiple second vertices obtained by globally sampling the second 3D model are respectively the second vertex 1, the second vertex 2, the second vertex 3, and the second vertex 4. Then the second global vertex set is {the second vertex 1, the second vertex 2, the second vertex 3, the second vertex 4}.

[0229] In step 403, the first vertices in the first global vertex set are aligned with the second vertices in the second global vertex set to obtain a first alignment result, where the first alignment result indicates that the first vertices and the second vertices correspond one by one.

[0230] In some embodiments, refer to Figure 3L , Figure 3L which is the twelfth process schematic diagram of the 3D model processing method provided by the embodiments of the present application. Figure 3K The step 403 of "<align the first vertices in the first global vertex set with the second vertices in the second global vertex set to obtain a first alignment result>" in Figure 3L can be implemented by steps 4031 to 4033 in

[0231] In step 4031, multiple candidate alignment methods are generated, where in each candidate alignment method, the first vertices in the first global vertex set and the second vertices in the second global vertex set correspond one by one.

[0232] In some embodiments, the candidate alignment method refers to an arrangement of the correspondence between the first vertices and the second vertices when aligning the first global vertex set and the second global vertex set. Each candidate alignment method defines a one-to-one correspondence between each first vertex in the first global vertex set and a second vertex in the second global vertex set.

[0233] Exemplarily, the first vertices in the first global vertex set include the first vertex 1, the first vertex 2, the first vertex 3, and the first vertex 4, and the second vertices in the second global vertex set include the second vertex 1, the second vertex 2, the second vertex 3, and the second vertex 4. The candidate alignment 1 can be: the first vertex 1 is aligned with the second vertex 1, the first vertex 2 is aligned with the second vertex 2, the first vertex 3 is aligned with the second vertex 3, and the first vertex 4 is aligned with the second vertex 4; the candidate alignment 2 can be: the first vertex 1 is aligned with the second vertex 2, the first vertex 2 is aligned with the second vertex 3, the first vertex 3 is aligned with the second vertex 4, and the first vertex 4 is aligned with the second vertex 1; the candidate alignment 3 can be: the first vertex 1 is aligned with the second vertex 4, the first vertex 2 is aligned with the second vertex 3, the first vertex 3 is aligned with the second vertex 2, and the first vertex 4 is aligned with the second vertex 1.

[0234] In step 4032, obtain a preset objective function, where the objective function is used to calculate a statistical value of the distances between the first vertices in the first global vertex set and the second vertices in the second global vertex set in any candidate alignment.

[0235] In some embodiments, the objective function is used to quantify the quality of the candidate alignment. Common objective functions include the least squares method, whose optimization objective is to minimize the sum of the squares of the distances between all corresponding vertices; or the min-max distance method, whose optimization objective is to minimize the maximum distance between vertices. The choice of the objective function depends on specific application requirements and optimization objectives. The statistical value can be the mean, sum, sum of squares, square root of the sum, etc.

[0236] Exemplarily, taking the least squares method as an example, the objective function can be seen in the following formula (1):

[0237]

[0238] where n is the number of vertices, P i is the i-th first vertex in the first global vertex set, Q i is the second vertex in the second global vertex set corresponding to P i , and ‖P i - Q i ‖ represents the distance between P i and Q i .

[0239] In step 4033, align the first vertices in the first global vertex set and the second vertices in the second global vertex set by the candidate alignment that minimizes the statistical value of the objective function, to obtain a first alignment result.

[0240] Exemplarily, if the statistical value corresponding to candidate alignment method 1 is 2, the statistical value corresponding to candidate alignment method 2 is 8, and the statistical value corresponding to candidate alignment method 3 is 6, then the statistical value corresponding to candidate alignment method 1 is the smallest. Align the first vertex in the first global vertex set with the second vertex in the second global vertex set through candidate alignment method 1 to obtain the first alignment result, that is, the first vertex 1 is aligned with the second vertex 1, the first vertex 2 is aligned with the second vertex 2, the first vertex 3 is aligned with the second vertex 3, and the first vertex 4 is aligned with the second vertex 4.

[0241] Embodiments of the present application can avoid local optimality and improve the robustness of alignment through multiple candidate alignment methods. It is applicable to various types of 3D model alignment problems, not limited to specific model types or application scenarios. The optimization process can be automated, reducing the need for manual adjustment and improving work efficiency.

[0242] Continue to refer to Figure 3K , in step 404, for each first vertex in the first global vertex set, update the pose parameters of the first vertex to the pose parameters of the second vertex corresponding to the first vertex in the first alignment result.

[0243] Here, the pose parameters include position and orientation. Updating the pose parameters means adjusting the position and orientation of the first vertex to be consistent with the position and orientation of the aligned second vertex. The pose parameters of the first vertex can be updated by direct assignment or applying a transformation matrix.

[0244] Embodiments of the present application ensure that the bone parameters of the first vertex of the first 3D model can be accurately bound to the second vertex of the second 3D model through automated sampling and alignment, reducing the workload of manual adjustment and improving work efficiency. By applying the bone parameters to the second 3D model, the interactivity of the model can be enhanced, and there is no need to perform bone binding and skinning for the second 3D model from scratch, saving time and computing resources.

[0245] In some embodiments, refer to Figure 3M , Figure 3M is the thirteenth process schematic diagram of the 3D model processing method provided by embodiments of the present application. After step 404, execute Figure 3M steps 405 to 408, which are specifically described below.

[0246] In step 405, divide the first 3D model into multiple first sub-models according to a preset configuration file, where the configuration file is used to indicate multiple segmentation positions and segmentation orders.

[0247] In some embodiments, the multiple splitting positions and splitting orders may be based on the geometric features, topological structures of the model, or user-defined splitting rules. The splitting process may involve geometric splitting algorithms, such as plane splitting, surface splitting, or feature-based splitting.

[0248] Exemplarily, refer to Figure 9 , Figure 9 which is a schematic diagram of splitting a three-dimensional model provided by an embodiment of the present application. In the left diagram of Figure 9 , the splitting order is from right to left, the multiple splitting positions are respectively the head of the virtual elephant and the middle positions between the front legs and the hind legs of the virtual elephant, and the first three-dimensional model is split into multiple first sub-models, such as the first sub-model 601, the first sub-model 602, and the first sub-model 603.

[0249] In step 406, according to the splitting order, the second three-dimensional model is split into multiple second sub-models at the multiple splitting positions.

[0250] Continuing with the example of the above step 405, also in the order from right to left, taking the head of the virtual elephant and the middle positions between the front legs and the hind legs of the virtual elephant as the splitting positions, the second three-dimensional model is split into multiple second sub-models, such as the second sub-model 701, the second sub-model 702, and the second sub-model 703.

[0251] Here, for each first sub-model, the following steps 407 and 408 are executed.

[0252] In step 407, the first sub-model and the second sub-model corresponding to the first sub-model are aligned to obtain a second alignment result, where the second alignment result represents that the multiple first sub-models and the multiple second sub-models correspond one by one.

[0253] In some embodiments, refer to Figure 3N , Figure 3N which is the fourteenth process schematic diagram of the method for processing a three-dimensional model provided by an embodiment of the present application. Figure 3M Step 407 of Figure 3N “Align the first sub-model and the second sub-model corresponding to the first sub-model to obtain a second alignment result” can be implemented by steps 4071 to 4073 of

[0254] In step 4071, a first local vertex set is sampled from the first sub-model, where the first local vertex set includes multiple first sub-vertices sampled from the first sub-model.

[0255] In some embodiments, local sampling refers to selecting a set of representative first sub-vertices from all the first sub-vertices of the first sub-model, and these first sub-vertices can reflect the shape and characteristics of the first sub-model. The sampling method can be uniform sampling, region-based sampling, feature-based sampling, etc.

[0256] For example, if the multiple first sub-vertices obtained by local sampling of the first sub-model are respectively the first sub-vertex 1, the first sub-vertex 2, and the first sub-vertex 3, then the first local vertex set is {the first sub-vertex 1, the first sub-vertex 2, the first sub-vertex 3}.

[0257] In step 4072, determine the target second sub-model corresponding to the first sub-model, and sample the second local vertex set from the target second sub-model, where the second local vertex set includes multiple second sub-vertices obtained by sampling the target second sub-model.

[0258] For example, if the multiple second sub-vertices obtained by local sampling of the second sub-model are respectively the second sub-vertex 1, the second sub-vertex 2, and the second sub-vertex 3, then the second local vertex set is {the second sub-vertex 1, the second sub-vertex 2, the second sub-vertex 3}.

[0259] In step 4073, align the first sub-vertices in the first local vertex set with the second sub-vertices in the second local vertex set to obtain the second alignment result.

[0260] In some embodiments, generate multiple alternative alignment methods. Among them, in each alternative alignment method, the first sub-vertices in the first local vertex set correspond one-to-one with the second sub-vertices in the second local vertex set; obtain a preset local objective function, where the local objective function is used to calculate the statistical value of the distances between the first sub-vertices in the first local vertex set and the second sub-vertices in the second local vertex set in any one of the alternative alignment methods; align the first sub-vertices in the first local vertex set with the second sub-vertices in the second local vertex set by the alternative alignment method that minimizes the statistical value of the local objective function to obtain the second alignment result.

[0261] Exemplarily, the first sub-vertices in the first local vertex set include the first sub-vertex 1, the first sub-vertex 2, and the first sub-vertex 3, and the second sub-vertices in the second local vertex set include the second sub-vertex 1, the second sub-vertex 2, and the second sub-vertex 3. The alternative alignment method 1 can be: the first sub-vertex 1 is aligned with the second sub-vertex 1, the first sub-vertex 2 is aligned with the second sub-vertex 2, and the first sub-vertex 3 is aligned with the second sub-vertex 3; the alternative alignment method 2 can be: the first sub-vertex 1 is aligned with the second sub-vertex 2, the first sub-vertex 2 is aligned with the second sub-vertex 3, and the first sub-vertex 3 is aligned with the second sub-vertex 1; the alternative alignment method 3 can be: the first sub-vertex 1 is aligned with the second sub-vertex 3, the first sub-vertex 2 is aligned with the second sub-vertex 1, and the first sub-vertex 3 is aligned with the second sub-vertex 2. If the statistical value corresponding to the alternative alignment method 1 is 2, the statistical value corresponding to the alternative alignment method 2 is 8, and the statistical value corresponding to the alternative alignment method 3 is 6, then the statistical value corresponding to the alternative alignment method 1 is the smallest. By aligning the first sub-vertices in the first local vertex set with the second sub-vertices in the second local vertex set through the alternative alignment method 1, a first alignment result is obtained, that is, the first sub-vertex 1 is aligned with the second sub-vertex 1, the first sub-vertex 2 is aligned with the second sub-vertex 2, and the first sub-vertex 3 is aligned with the second sub-vertex 3.

[0262] In the embodiments of the present application, by performing alignment within a local range, the shapes and features of the first sub-model and the second sub-model can be more precisely matched, and the error caused by global alignment can be reduced. The amount of data processed by local alignment is small, and the computational complexity is low, which can improve the efficiency of alignment. Through local alignment, it is possible to better adapt to the deformation or pose differences that may exist in different parts of the model, and enhance the adaptability of the model.

[0263] Continue to refer to Figure 3M , in step 408, update the pose parameters of the first sub-model to the pose parameters of the second sub-model corresponding to the first sub-model in the second alignment result.

[0264] In some embodiments, step 408 "update the pose parameters of the first sub-model to the pose parameters of the second sub-model corresponding to the first sub-model in the second alignment result" can be implemented by performing the following processing: for each first sub-vertex in the first local vertex set, update the pose parameters of the first sub-vertex to the pose parameters of the second sub-vertex corresponding to the first sub-vertex in the second alignment result.

[0265] Exemplarily, the pose parameters include position and orientation. Updating the pose parameters means adjusting the position and orientation of the first sub-vertex to be consistent with those of the second sub-vertex after alignment, and the pose parameters of the first sub-vertex can be updated by direct assignment or applying a transformation matrix. Taking direct assignment as an example, for each first sub-vertex in the first local vertex set, such as the first sub-vertex 1, the second sub-vertex corresponding to the first sub-vertex in the second alignment result is the second sub-vertex 1. If the position of the first sub-vertex 1 is P1, the orientation is D1, the position of the second sub-vertex 1 is Q1, and the orientation is E1, then the position of the first sub-vertex 1 is updated to Q1, and the orientation is updated to E1.

[0266] In the embodiments of the present application, by splitting the model into sub-models and performing local alignment, the alignment accuracy can be improved, especially when different parts of the model have different deformations or poses. The preset configuration file allows users to define the splitting positions and orders, so that the solution can be flexibly adjusted according to different models and application requirements. Splitting the model into sub-models can reduce the complexity of overall alignment, making the alignment process of each sub-model more focused on local geometry and improving the calculation efficiency.

[0267] In some embodiments, refer to Figure 3O , Figure 3O which is the fifteenth process schematic diagram of the method for processing a three-dimensional model provided by the embodiments of the present application. Before step 104, perform Figure 3O steps 105 to 108 as follows for specific description.

[0268] In step 105, apply the bone parameters of the third three-dimensional model to the bones of the preset animation model.

[0269] In some embodiments, for each third vertex in the third three-dimensional model, determine the fourth vertex corresponding to the third vertex in the preset animation model, and apply the bone parameters of the third vertex to the fourth vertex corresponding to the third vertex.

[0270] Exemplarily, the bone parameters include bone position and bone weight. Apply the bone position of the third vertex to the fourth vertex corresponding to the third vertex, and apply the bone weight of the third vertex to the fourth vertex corresponding to the third vertex.

[0271] In step 106, set the key frames corresponding to the animation model, and determine the movement trajectories of the bones of the animation model in the key frames.

[0272] In some embodiments, the key frames are the critical moments in the animation, which define the positions and poses of the bones at these moments. The key frames can be user-defined image frames, can also be image frames including the target object, or can be image frames at preset moments.

[0273] Exemplarily, in an animation editing software, a series of key frames are set, which define different postures of the animation model in the walking animation. For example, in the first key frame, the left foot of the animation model steps forward; in the second key frame, the right foot of the animation model steps forward, and so on. Through these key frames, the landing points of the bones of the animation model in the walking animation are fitted into a motion trajectory.

[0274] In step 107, control the bones of the animation model to move according to the motion trajectory to obtain animation data.

[0275] In some embodiments, according to the motion trajectory defined by the key frames, calculate the position and posture of the bones in each frame. Use an interpolation algorithm to calculate the intermediate posture of the bones in each frame as the interpolation result. For each frame, calculate the transformation matrix of each bone through the interpolation result. These matrices describe the position, rotation, and scaling of the bones. Store the calculated transformation matrix as animation data.

[0276] Exemplarily, common interpolation algorithms include Linear Interpolation (LERP) and Spline Interpolation (such as Catmull-Rom or Bezier splines). The animation data can be saved in the form of a skeletal animation file (such as FBX, DAE, or a custom format).

[0277] In step 108, apply the animation data to the third 3D model.

[0278] In some embodiments, first, map the bone structure in the animation data to the bones of the third 3D model to complete bone binding; second, bind the mesh of the third 3D model to the bones so that the mesh can deform according to the movement of the bones to complete skinning; finally, apply the animation data to the third 3D model that has completed bone binding and skinning. According to the bone transformation matrix in the animation data, update the bone posture of the third 3D model, and drive the mesh deformation of the third 3D model through skinning technology to form an animation.

[0279] Exemplarily, common skinning technologies include rigid skinning and soft body skinning, where soft body skinning (such as LBS) allows mesh vertices to deform according to the weights of multiple bones. During the skinning process, assign weights to each third vertex of the third 3D model, indicating the degree to which the third vertex is affected by each bone. The weight assignment can be completed through manual adjustment or an automatic calculation method.

[0280] In the embodiments of the present application, by applying the bone parameters of the third 3D model to a preset animation model, setting key frames, and generating animation data, high-quality character animations can be efficiently produced. Through key frame editing and adjustment of animation data, the animation effects can be flexibly modified and optimized to meet different animation requirements, and it has good compatibility and scalability.

[0281] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0282] When performing bone binding and skinning on a model, in the related art, the method of manually annotating bones by humans is used. Representative methods include the automatic binding method (AccuRIG) or the animation production platform (Mixamo). The main method is to obtain the joint positions through manual annotation, and then automatically skin the model according to the user's annotation. Alternatively, in the related art, for fully automatic bone binding and skinning, the 3D model can be bone-bound and skinned through deep learning. The deep neural network is trained through a large amount of data training to obtain a pre-trained deep neural network. When a 3D model is input into the pre-trained deep neural network, the pre-trained deep neural network can output its bones and skinning weights.

[0283] In the related art, the method of manually annotating bones by humans cannot achieve full automation and is difficult to expand to other categories. In the related art, the fully automatic bone binding and skinning method requires collecting a large amount of training data, and the quality of the training data affects the final effect of the model, and it has poor adaptability to categories other than those included in the training dataset. In addition, both of these methods require the input 3D model to be in a certain or some specific poses, such as T-pose or A-pose.

[0284] The 3D model processing method provided by the embodiments of this application first trains a deep neural network in a self-supervised manner by giving a template model with bone binding and bone animation (i.e., the first model). When a point cloud is input, the trained deep neural network (i.e., the pre-trained neural network model) can output the depth features (i.e., feature vectors) for each point cloud. This feature includes semantic information, such as which body part a certain point belongs to. Then, by matching the depth features of the vertices of the template model (i.e., the first feature vector) and the depth features of the vertices of the user model to be bound (i.e., the second model), a one-to-one semantic correspondence between the mesh vertices of the template model and the mesh vertices of the user model is obtained. Finally, the template model is optimized to align it with the user model, and the bone positions and skinning weights (i.e., bone parameters) of the user asset are obtained based on the optimized template asset. Using the semantic correspondence information of the vertices, the model is optimized by parts. First, the template model is rotated, translated, or scaled as a whole, then the body trunk is optimized, and finally the limbs, tail, ears, etc. The optimization process and order can be configured for different models through a configuration file.

[0285] In the embodiments of this application, the user only needs to select the desired template to achieve fully automatic bone binding and skinning, reducing the user's usage cost. This method has no special requirements for the posture of the input 3D model and can be extended and adapted to different types of models. Through configurable part-by-part optimization and fine-tuning of the configuration file, the final binding result can be optimized without modifying the model itself.

[0286] The 3D model processing method provided by the embodiments of this application can be used for automatic bone binding of 3D characters in games. In a game, when a user uses 3D Artificial Intelligence Generated Content (AIGC) technology to generate a certain 3D character, then this method can be directly used for automatic binding and display the animation, which is applicable to various types of characters.

[0287] See Figure 10 , Figure 10It is a schematic flow diagram of model generation provided by an embodiment of the present application. Using 3D AIGC technology, a static model corresponding to the prompt is generated through the prompt. Through skeleton binding and skinning, the static model is converted into a dynamic model. Technically, it is mainly divided into two parts: the template model processing part and the skeleton binding and skinning part. The template model processing part is carried out in the product deployment stage, and its main purpose is to train a neural network for feature extraction. The skeleton binding and skinning part is carried out in the product usage stage. This part is used to optimize the template model, including global translation and rotation optimization, and part-by-part scaling and pose optimization, to match the template model with the user model, so as to apply the skeleton parameters of the optimized template model to the user model and complete the skeleton binding and skinning process of the user model.

[0288] For the template model processing part, refer to Figure 11 , Figure 11 It is a schematic flow diagram of processing the template model provided by an embodiment of the present application. Each type of model only requires one template and some (10 - 20 segments) of animation resources. In order to improve the adaptability of the neural network to models of different body types, a method of enhanced part-by-part scaling is proposed. Specifically: refer to Figure 12 , Figure 12 It is a schematic diagram of part-by-part scaling of the template model provided by an embodiment of the present application. For a template, randomly scale its different parts, and then use the scaled model as the training data of the neural network.

[0289] Training is carried out through a neural network based on Transformer for processing point clouds. Specifically, for each training model sample, first randomly sample N points uniformly on the model, denoted as x ∈ R N * 3 , input these N points into the neural network, and the depth features corresponding to each point can be obtained, denoted as y ∈ R N * K . Similarly, randomly rotate these N points, and the rotated points are denoted as x′ ∈ R N * 3 , and calculate their depth features, denoted as y′ ∈ R N * K . The method of contrast training is used to train this neural network, and its loss function is defined as the following formula (2):

[0290]

[0291] For the skeleton binding and skinning part, refer to Figure 13 , Figure 13It is a schematic diagram of the process of bone binding and skinning provided by an embodiment of the present application. First, the user model and the template model input by the user are feature-matched to obtain some prior knowledge, such as body part division, etc. The specific method is to respectively extract the feature vectors of each vertex of the template model and the user model through a trained neural network, and denote them as y t ∈R N * K ,y u ∈R M * K 。The feature matching weight W ∈ R N * M is calculated by the following formula (3):

[0292] W[n, m] = y t [n] · y u [m] (3)

[0293] Secondly, global translation and rotation optimization, as well as part-by-part scaling and pose optimization are performed. The final effect of these two steps is to match the template model and the user model to minimize the Chamfer Distance between the two as much as possible. The technical routes of the two optimizations are similar, and the main difference is the content of the optimization. Global translation and rotation optimization is to obtain the global translation and rotation, and part-by-part scaling and pose optimization is to obtain the poses and sizes of the details of each part. The objective functions of both optimizations can be seen in the following formula (4):

[0294] E mean(W * cdist(x t ,x u )) (4)

[0295] Among them, mean() represents taking the mean of all values in the input matrix, cdist() represents calculating the distances between two points in two input point clouds, and x t ,x u respectively represent the point cloud composed of the vertices of the template model and the point cloud composed of the vertices of the user model. For each template model, a corresponding asset configuration file (i.e., configuration file) can be designed, which mainly controls the division of the parts of the template model and the optimization order.

[0296] Since the preprocessing and optimization steps of the template model are separated, application expansion can be achieved at a low cost. For example, assuming that a new template and its corresponding animation data are obtained, a new neural network can be trained to obtain feature extraction for the new template. Then, an optimization configuration file is designed according to the characteristics of the template. And the user only needs to download the trained neural network and the optimization configuration file to complete the update of the product.

[0297] In the embodiment of the present application, by extracting the feature vectors of the first three-dimensional model and the second three-dimensional model, the second vertices corresponding to each first vertex in the first three-dimensional model are determined, thereby realizing the automatic transfer and reuse of bone parameters. Compared with manually setting bone parameters, the setting time and workload are reduced, and the generation efficiency of the third three-dimensional model is improved. The first three-dimensional model and the second three-dimensional model only represent the same object, and the poses or sizes of the first three-dimensional model and the second three-dimensional model are not limited, which improves the flexibility and adaptability of model processing.

[0298] Next, the exemplary structure of the software module of the three-dimensional model processing device 555 provided in the embodiment of the present application will be further described. In some embodiments, as Figure 2 shown, the software modules in the three-dimensional model processing device 555 stored in the memory 550 may include:

[0299] A model acquisition module 5551, configured to acquire a first three-dimensional model and a second three-dimensional model for representing the same object, wherein there is an association relationship between the first vertices in the first three-dimensional model and the preset bone parameters in the bone.

[0300] A feature extraction module 5552, configured to extract the first feature vectors of each first vertex in the first three-dimensional model and the second feature vectors of each second vertex in the second three-dimensional model.

[0301] A vertex matching module 5553, configured to determine the second vertices corresponding to each first vertex based on the first feature vectors and the second feature vectors.

[0302] A model generation module 5554, configured to associate the bone parameters associated with each first vertex in the first three-dimensional model with the second vertices corresponding to the first vertices, and use the second three-dimensional model after association as the third three-dimensional model.

[0303] In some embodiments, the model acquisition module 5551 is further configured to acquire a preset model, where the preset model is used to represent a sample object, and the sample object is an object different from the object represented by the first three-dimensional model; extract sample object features from the sample object; call a generator based on the sample object features to obtain a prediction model; determine the loss between the prediction model and the preset model through a discriminator; update the parameters of the generator and the discriminator based on the loss to obtain a pre-trained generator and a pre-trained discriminator; call the pre-trained generator and the pre-trained discriminator based on the object features of the object to generate the first three-dimensional model.

[0304] In some embodiments, the feature extraction module 5552 is further configured to, based on the coordinates of multiple first vertices in the first 3D model, call a pre-trained neural network model to extract a first feature vector for each first vertex; and based on the coordinates of multiple second vertices in the second 3D model, call a pre-trained neural network model to extract a second feature vector for each second vertex.

[0305] In some embodiments, the feature extraction module 5552 is further configured to obtain a sample 3D model, extract a first global sample vertex set from the sample 3D model; transform the first global sample vertex set to obtain a second global sample vertex set; based on the first global sample vertex set, call an initialized neural network model for feature extraction to obtain a first global feature for each first global sample vertex in the first global sample vertex set, and based on the second global sample vertex set, call an initialized neural network model for feature extraction to obtain a second global feature for each second global sample vertex in the second global sample vertex set; for each first global sample vertex, determine a target second global sample vertex corresponding to the first global sample vertex, construct a global feature pair based on the first global feature of the first global sample vertex and the second global feature of the target second global sample vertex, and based on the global feature pair, call a preset global loss function to obtain a global loss; update the parameters of the initialized neural network model based on the global loss to obtain a pre-trained neural network model.

[0306] In some embodiments, the feature extraction module 5552 is further configured to perform any one of the following processes: for each first global sample vertex in the first global sample vertex set, rotate the first global sample vertex to obtain a second global sample vertex corresponding to the first global sample vertex, and combine the multiple second global sample vertices into a second global sample vertex set; for each first global sample vertex in the first global sample vertex set, translate the first global sample vertex to obtain a second global sample vertex corresponding to the first global sample vertex, and combine the multiple second global sample vertices into a second global sample vertex set.

[0307] In some embodiments, the feature extraction module 5552 is further configured to, for each first global sample vertex, determine the exponent of the product of the first global feature and the second global feature in the global feature pair; determine the sum of the exponents corresponding to the multiple first global features; determine the ratio of the exponent to the sum of the exponents; determine the first sum of the ratios corresponding to the multiple first global features; determine the first ratio of the first sum to the number of the multiple first global sample vertices, and use the first ratio as the global loss.

[0308] In some embodiments, the feature extraction module 5552 is further configured to determine the difference between the first global feature and the second global feature in each global feature pair; determine the second sum of the differences corresponding to the multiple global feature pairs; determine the second ratio of the second sum to the number of the multiple global feature pairs, and use the second ratio as the global loss.

[0309] In some embodiments, the feature extraction module 5552 is further configured to cut the sample three-dimensional model into multiple sub-sample three-dimensional models, extract a first set of local sample vertices from each sub-sample three-dimensional model; perform transformations on each first set of local sample vertices to obtain multiple second sets of local sample vertices; for each first set of local sample vertices, perform the following processing: call a pre-trained neural network model based on the first set of local sample vertices to perform feature extraction to obtain the first local features of each first local sample vertex in the first set of local sample vertices, call a pre-trained neural network model based on the second set of local sample vertices to perform feature extraction to obtain the second local features of each second local sample vertex in the second set of local sample vertices; for each first local sample vertex, determine the target second local sample vertex corresponding to the first local sample vertex, construct a local feature pair based on the first local feature of the first local sample vertex and the second local feature of the target second local sample vertex, call a preset local loss function based on the local feature pair to determine the local loss; update the parameters of the pre-trained neural network model based on the local loss to obtain an updated neural network model.

[0310] In some embodiments, the vertex matching module 5553 is further configured to determine the similarity between the first feature vector of each first vertex in the first three-dimensional model and the second feature vector of each second vertex; find the second vertex corresponding to each first vertex from a preset relationship matrix, where each element in the relationship matrix corresponds to a similarity.

[0311] In some embodiments, the elements in each row of the relationship matrix represent the similarities between the first feature vector of any one first vertex and the second feature vectors of multiple second vertices. The vertex matching module 5553 is further configured to perform the following processing for each row of the relationship matrix: use the largest element in the row as the target element; use the first vertex corresponding to the row as the target first vertex; use the second feature vector corresponding to the target element as the target second feature vector, and determine the second vertex corresponding to the target second feature vector; use the second vertex corresponding to the target second feature vector as the second vertex corresponding to the target first vertex.

[0312] In some embodiments, the model generation module 5554 is further configured to apply the bone position of the first vertex to the second vertex corresponding to the first vertex; apply the bone weight of the first vertex to the second vertex corresponding to the first vertex; and update the second 3D model to a third 3D model based on the updated second vertex.

[0313] In some embodiments, the model generation module 5554 is further configured to sample a first global vertex set from the first 3D model, where the first global vertex set includes a plurality of first vertices obtained by globally sampling the first 3D model; sample a second global vertex set from the second 3D model, where the second global vertex set includes a plurality of second vertices obtained by globally sampling the second 3D model; align the first vertices in the first global vertex set with the second vertices in the second global vertex set to obtain a first alignment result, where the first alignment result indicates that the first vertices and the second vertices correspond one by one; and for each first vertex in the first global vertex set, update the pose parameter of the first vertex to the pose parameter of the second vertex corresponding to the first vertex in the first alignment result.

[0314] In some embodiments, the model generation module 5554 is further configured to generate a plurality of candidate alignment methods, where in each candidate alignment method, the first vertices in the first global vertex set and the second vertices in the second global vertex set correspond one by one; obtain a preset objective function, where the objective function is used to calculate a statistical value of the distances between the first vertices in the first global vertex set and the second vertices in the second global vertex set in any one of the candidate alignment methods; and align the first vertices in the first global vertex set with the second vertices in the second global vertex set by using the candidate alignment method that minimizes the statistical value of the objective function to obtain a first alignment result.

[0315] In some embodiments, the model generation module 5554 is further configured to divide the first 3D model into a plurality of first sub-models according to a preset configuration file, where the configuration file is used to indicate a plurality of division positions and a division order; divide the second 3D model into a plurality of second sub-models at the plurality of division positions according to the division order; and for each first sub-model, perform the following processing: align the first sub-model with the second sub-model corresponding to the first sub-model to obtain a second alignment result, where the second alignment result indicates that the plurality of first sub-models and the plurality of second sub-models correspond one by one; and update the pose parameter of the first sub-model to the pose parameter of the second sub-model corresponding to the first sub-model in the second alignment result.

[0316] In some embodiments, the model generation module 5554 is further configured to sample a first local vertex set from the first sub-model, where the first local vertex set includes a plurality of first sub-vertices sampled from the first sub-model; determine a target second sub-model corresponding to the first sub-model, and sample a second local vertex set from the target second sub-model, where the second local vertex set includes a plurality of second sub-vertices sampled from the target second sub-model; align the first sub-vertices in the first local vertex set with the second sub-vertices in the second local vertex set to obtain a second alignment result.

[0317] In some embodiments, the model generation module 5554 is further configured to, for each first sub-vertex in the first local vertex set, update the pose parameter of the first sub-vertex to the pose parameter of the second sub-vertex corresponding to the first sub-vertex in the second alignment result.

[0318] In some embodiments, the model generation module 5554 is further configured to apply the bone parameters of the third 3D model to the bones of a preset animation model; set key frames corresponding to the animation model, and determine the movement trajectories of the bones of the animation model in the key frames; control the bones of the animation model to move according to the movement trajectories to obtain animation data; and apply the animation data to the third 3D model.

[0319] An embodiment of the present application provides a computer program product, which includes a computer program or computer executable instructions, and the computer program or computer executable instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instructions from the computer-readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the above-mentioned 3D model processing method of the embodiment of the present application.

[0320] An embodiment of the present application provides a computer-readable storage medium storing computer executable instructions, where computer executable instructions or a computer program are stored therein. When the computer executable instructions or the computer program are executed by a processor, the processor will be caused to execute the 3D model processing method provided by the embodiment of the present application. For example, as Figure 3A shown in the 3D model processing method.

[0321] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0322] In some embodiments, the computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as a stand-alone program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0323] As an example, the computer-executable instructions may or may not correspond to a file in a file system, may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program being discussed, or, stored in multiple cooperating files (such as files that store one or more modules, subroutines, or portions of code).

[0324] As an example, the computer-executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or, on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0325] In summary, through the embodiments of the present application, by extracting the feature vectors of the first three-dimensional model and the second three-dimensional model, the second vertices respectively corresponding to each first vertex in the first three-dimensional model are determined, thereby realizing the automatic transfer and reuse of bone parameters, reducing the setting time and workload compared with manually setting bone parameters, and improving the generation efficiency of the third three-dimensional model. The first three-dimensional model and the second three-dimensional model only represent the same object, and the poses or sizes of the first three-dimensional model and the second three-dimensional model are not limited, improving the flexibility and adaptability of model processing.

[0326] The above is only the embodiments of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for processing a three-dimensional model, characterized in that: The method comprises: Acquire a first three-dimensional model and a second three-dimensional model for representing the same object, wherein a first vertex in the first three-dimensional model is associated with a preset bone parameter in the bone; Extracting a first eigenvector of each first vertex in the first three-dimensional model and a second eigenvector of each second vertex in the second three-dimensional model; Determine the second vertex corresponding to each of the first vertices based on the first eigenvector and the second eigenvector; The bone parameters in the first three-dimensional model that have the association relationship with each of the first vertices are associated with the second vertices corresponding to the first vertices, and the associated second three-dimensional model is used as the third three-dimensional model.

2. The method according to claim 1, characterized in that The extracting a first feature vector of each first vertex in the first three-dimensional model and a second feature vector of each second vertex in the second three-dimensional model comprises: Based on the coordinates of the plurality of first vertices in the first three-dimensional model, calling a pre-trained neural network model to extract a first feature vector of each first vertex; Based on the coordinates of the plurality of second vertices in the second three-dimensional model, the pre-trained neural network model is called to extract a second feature vector of each second vertex.

3. The method according to claim 2, characterized in that Before calling a pre-trained neural network model based on the coordinates of the plurality of first vertices in the first three-dimensional model to extract a first feature vector of each first vertex, the method further includes: Acquire a sample three-dimensional model, and extract a first global sample vertex set from the sample three-dimensional model; Transforming the first global sample vertex set to obtain a second global sample vertex set; Based on the first global sample vertex set, the initialized neural network model is called to perform feature extraction to obtain a first global feature of each first global sample vertex in the first global sample vertex set; based on the second global sample vertex set, the initialized neural network model is called to perform feature extraction to obtain a second global feature of each second global sample vertex in the second global sample vertex set; For each of the first global sample vertices, determine a target second global sample vertex corresponding to the first global sample vertex, construct a global feature pair based on the first global feature of the first global sample vertex and the second global feature of the target second global sample vertex, and call a preset global loss function based on the global feature pair to obtain a global loss; The parameters of the initialized neural network model are updated based on the global loss to obtain the pre-trained neural network model.

4. The method according to claim 3, characterized in that The step of transforming the first global sample vertex set to obtain a second global sample vertex set includes: Perform any of the following: For each of the first global sample vertices in the first global sample vertex set, the first global sample vertex is rotated to obtain the second global sample vertex corresponding to the first global sample vertex, and a plurality of the second global sample vertices are combined into a second global sample vertex set; For each of the first global sample vertices in the first global sample vertex set, the first global sample vertex is translated to obtain the second global sample vertex corresponding to the first global sample vertex, and a plurality of the second global sample vertices are combined into a second global sample vertex set.

5. The method according to claim 3, characterized in that: The calling a preset global loss function based on the global feature pair to determine the global loss includes: For each of the first global sample vertices, determining an exponent of a product of the first global feature and the second global feature in the global feature pair; Determine the sum of indices of the indices corresponding to the plurality of first global features respectively; determining a ratio of the index to the sum of the indices; Determine a first sum of the ratios respectively corresponding to a plurality of the first global features; A first ratio of the first sum to the number of the first global sample vertices is determined, and the first ratio is used as a global loss.

6. The method according to claim 3, characterized in that The calling a preset global loss function based on the global feature pair to determine the global loss includes: determining a difference between the first global feature and the second global feature in each of the global feature pairs; Determine a second sum of the differences respectively corresponding to a plurality of the global feature pairs; A second ratio of the second sum to the number of the plurality of global feature pairs is determined, and the second ratio is used as a global loss.

7. The method according to claim 3, characterized in that After updating the parameters of the initialized neural network model based on the global loss to obtain the pre-trained neural network model, the method further includes: Cutting the sample three-dimensional model into a plurality of sub-sample three-dimensional models, and extracting a first local sample vertex set from each of the sub-sample three-dimensional models; Transforming each of the first local sample vertex sets respectively to obtain a plurality of second local sample vertex sets; For each of the first local sample vertex sets, the following processing is performed: Based on the first local sample vertex set, the pre-trained neural network model is called to perform feature extraction to obtain a first local feature of each first local sample vertex in the first local sample vertex set; based on the second local sample vertex set, the pre-trained neural network model is called to perform feature extraction to obtain a second local feature of each second local sample vertex in the second local sample vertex set; For each of the first local sample vertices, determine a target second local sample vertex corresponding to the first local sample vertex, construct a local feature pair based on the first local feature of the first local sample vertex and the second local feature of the target second local sample vertex, and call a preset local loss function based on the local feature pair to determine a local loss; The parameters of the pre-trained neural network model are updated based on the local loss to obtain an updated neural network model.

8. The method according to any one of claims 1 to 7, characterized in that: The determining the second vertex corresponding to each first vertex based on the first eigenvector and the second eigenvector includes: For each of the first vertices in the first three-dimensional model, determining a similarity between the first feature vector of the first vertex and the second feature vector of each of the second vertices; The second vertices corresponding to each of the first vertices are searched from a preset relationship matrix, wherein each of the elements in the relationship matrix corresponds to one similarity.

9. The method according to claim 8, characterized in that The elements of each row of the relationship matrix represent the similarity between the first eigenvector of any one of the first vertices and the second eigenvectors of multiple second vertices; The step of searching the preset relationship matrix for the second vertex corresponding to each of the first vertices includes: For each of the rows of the relationship matrix, perform the following processing: Taking the largest element in the row as the target element; Taking the first vertex corresponding to the row as the target first vertex; Taking the second eigenvector corresponding to the target element as a target second eigenvector, and determining the second vertex corresponding to the target second eigenvector; The second vertex corresponding to the target second eigenvector is used as the second vertex corresponding to the target first vertex.

10. The method according to any one of claims 1 to 7, characterized in that: Before associating the bone parameters in the first three-dimensional model that are associated with each of the first vertices with the second vertices corresponding to the first vertices, and using the associated second three-dimensional model as the third three-dimensional model, the method further includes: Sampling the first three-dimensional model to obtain a first global vertex set, wherein the first global vertex set includes a plurality of first vertices obtained by globally sampling the first three-dimensional model; Sampling the second three-dimensional model to obtain a second global vertex set, wherein the second global vertex set includes a plurality of second vertices obtained by globally sampling the second three-dimensional model; Aligning the first vertex in the first global vertex set with the second vertex in the second global vertex set to obtain a first alignment result, wherein the first alignment result indicates that the first vertex corresponds one-to-one with the second vertex; For each of the first vertices in the first global vertex set, the pose parameters of the first vertex are updated to the pose parameters of the second vertex corresponding to the first vertex in the first alignment result.

11. The method according to claim 10, characterized in that Aligning the first vertex in the first global vertex set with the second vertex in the second global vertex set to obtain a first alignment result, including: Generate a plurality of candidate alignments, wherein in each of the candidate alignments, the first vertices in the first global vertex set correspond one-to-one to the second vertices in the second global vertex set; Obtaining a preset objective function, wherein the objective function is used to calculate a statistical value of a distance between the first vertex in the first global vertex set and the second vertex in the second global vertex set in any one of the candidate alignment modes; The first vertices in the first global vertex set and the second vertices in the second global vertex set are aligned using the candidate alignment method that minimizes the statistical value of the objective function to obtain a first alignment result.

12. The method according to claim 10, characterized in that After updating the pose parameters of the first vertex in the first global vertex set to the pose parameters of the second vertex corresponding to the first vertex in the first alignment result, the method further includes: Segmenting the first three-dimensional model into a plurality of first sub-models according to a preset configuration file, wherein the configuration file is used to indicate a plurality of segmentation positions and a segmentation order; According to the segmentation order, segmenting the second three-dimensional model into a plurality of second sub-models at a plurality of segmentation positions; For each of the first sub-models, the following processing is performed: Aligning the first sub-model with the second sub-model corresponding to the first sub-model to obtain a second alignment result, wherein the second alignment result indicates that a plurality of the first sub-models correspond one-to-one to a plurality of the second sub-models; Update the pose parameters of the first sub-model to the pose parameters of the second sub-model corresponding to the first sub-model in the second alignment result.

13. The method according to claim 12, characterized in that The aligning the first sub-model with the second sub-model corresponding to the first sub-model to obtain a second alignment result includes: Sampling the first sub-model to obtain a first local vertex set, wherein the first local vertex set includes a plurality of first sub-vertices obtained by sampling the first sub-model; Determine a target second sub-model corresponding to the first sub-model, and sample from the target second sub-model to obtain a second local vertex set, wherein the second local vertex set includes a plurality of second sub-vertices obtained by sampling the target second sub-model; Aligning the first sub-vertex in the first local vertex set with the second sub-vertex in the second local vertex set to obtain a second alignment result; The updating of the pose parameters of the first sub-model to the pose parameters of the second sub-model corresponding to the first sub-model in the second alignment result includes: For each of the first sub-vertex in the first local vertex set, the position and posture parameters of the first sub-vertex are updated to the position and posture parameters of the second sub-vertex corresponding to the first sub-vertex in the second alignment result.

14. The method according to any one of claims 1 to 7, characterized in that: Associating the bone parameters in the first three-dimensional model that are associated with each of the first vertices with the second vertices corresponding to the first vertices, and using the associated second three-dimensional model as a third three-dimensional model, including: For each of the first vertices, perform the following processing: Applying the bone position of the first vertex to the second vertex corresponding to the first vertex; Applying the bone weight of the first vertex to the second vertex corresponding to the first vertex; Based on the updated second vertices, the second three-dimensional model is updated to the third three-dimensional model.

15. The method according to any one of claims 1 to 7, characterized in that: After associating the bone parameters in the first three-dimensional model that are associated with each of the first vertices with the second vertices corresponding to the first vertices, and using the associated second three-dimensional model as the third three-dimensional model, the method further includes: Applying the skeleton parameters of the third three-dimensional model to the skeleton of the preset animation model; Setting key frames corresponding to the animation model, and determining the motion trajectory of the skeleton of the animation model in the key frames; Controlling the skeleton of the animation model to move according to the motion trajectory to obtain animation data; The animation data is applied to the third three-dimensional model.

16. The method according to any one of claims 1 to 7, characterized in that: Before acquiring the first three-dimensional model and the second three-dimensional model, the method further includes: Acquire a preset model, wherein the preset model is used to represent a sample object, and the sample object is an object different from the object represented by the first three-dimensional model; extracting sample object features from the sample object; Calling a generator based on the sample object features to obtain a prediction model; Determine the loss between the prediction model and the preset model by a discriminator; Based on the loss, the parameters of the generator and the discriminator are updated to obtain a pre-trained generator and a pre-trained discriminator; The pre-trained generator and the pre-trained discriminator are called based on the object features of the object to generate the first three-dimensional model.

17. A three-dimensional model processing device, characterized in that: The device comprises: A model acquisition module, used to acquire a first three-dimensional model and a second three-dimensional model for representing the same object, wherein a first vertex in the first three-dimensional model is associated with a preset bone parameter in the bone; A feature extraction module, configured to extract a first feature vector of each first vertex in the first three-dimensional model and a second feature vector of each second vertex in the second three-dimensional model; A vertex matching module, configured to determine the second vertex corresponding to each of the first vertices based on the first eigenvector and the second eigenvector; The model generation module is used to associate the bone parameters in the first three-dimensional model that have the association relationship with each of the first vertices with the second vertices corresponding to the first vertices, and use the associated second three-dimensional model as the third three-dimensional model.

18. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; A processor, used to implement the three-dimensional model processing method described in any one of claims 1 to 16 when executing the computer executable instructions or computer program stored in the memory.

19. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the three-dimensional model processing method described in any one of claims 1 to 16 is implemented.

20. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the three-dimensional model processing method described in any one of claims 1 to 16 is implemented.

Images