Virtual model processing method and device, electronic equipment, storage medium and product

Abstract

The application discloses a processing method of a virtual model, and the method comprises the following steps: acquiring a plurality of target images of a target virtual model in a virtual scene and image-related data of each target image; detecting the positions of the skeletal joints of the target virtual model in each target image to obtain a position detection result of the skeletal joints; converting the positions of the skeletal joints in the target images into the positions of the skeletal joints in the virtual scene based on the position detection result and the image-related data to obtain target positions of the skeletal joints; generating a skeleton instance corresponding to the target virtual model based on the target positions; and performing skinning processing on the target virtual model according to the skeleton instance to obtain a skinning processing result of the target virtual model. In this way, the application realizes the automatic binding of the virtual model by acquiring a plurality of target images and camera parameters, detecting two-dimensional joints and converting the two-dimensional joints into three-dimensional coordinates, generating a skeleton instance and automatically performing skinning, and the processing efficiency is effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer graphics technology, specifically to a method for processing virtual models, a device for processing virtual models, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In related technologies, the skeletal binding and skinning of virtual models usually rely on technical artists to manually mark joints, manually build skeletal structures and assign skinning weights, or use automatic binding tools to output general reference skeletons or free skeletons. However, the output skeleton structure is difficult to match with the actual shape of the target virtual model, and the naming, hierarchy and attachment point rules of the skeleton cannot meet the standardized specifications of specific projects. As a result, the generated skinning results cannot be directly reused and still require a lot of manual post-processing and adaptation adjustments. The degree of manual dependence is high and the degree of automation and processing efficiency are low. Summary of the Invention

[0003] This application provides a method for processing virtual models, a device for processing virtual models, a computer-readable storage medium, an electronic device, and a computer program product. By acquiring multiple target images of the target virtual model and camera geometric parameters, the two-dimensional positions of skeletal joints in the images are detected and converted into three-dimensional spatial coordinates. Based on this, a matching skeleton instance is generated and skinning is automatically completed. This achieves automated binding of the virtual model without the need for manual marking of joints and adjustment of skeletal structure, effectively reducing manual dependence and processing costs, and improving processing efficiency and animation reuse compatibility.

[0004] On one hand, embodiments of this application provide a method for processing virtual models, the method comprising: Multiple target images of a target virtual model in a virtual scene and image correlation data for each target image are acquired. Each target image is obtained by a virtual camera that observes the target virtual model in the virtual scene. The observation position of the virtual camera corresponding to each target image is different. The image correlation data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. The positions of the skeletal joints of the target virtual model in each target image are detected to obtain the position detection results of the skeletal joints; Based on the position detection results and the image correlation data, the position of the skeletal joint in the target image is converted into the position of the skeletal joint in the virtual scene to obtain the target position of the skeletal joint. Based on the target location, generate a skeleton instance corresponding to the target virtual model; Based on the skeleton instance, the target virtual model is skinned to obtain the skinning result of the target virtual model.

[0005] On the other hand, embodiments of this application provide a virtual model processing apparatus, the apparatus comprising: An image data acquisition module is used to acquire multiple target images of a target virtual model in a virtual scene and image-related data of each target image. Each target image is obtained by a virtual camera in the virtual scene that observes the target virtual model. The observation position of the virtual camera corresponding to each target image is different. The image-related data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. The joint detection module is used to detect the position of the skeletal joints of the target virtual model in each target image, and obtain the position detection result of the skeletal joints; The position conversion module is used to convert the position of the skeletal joint in the target image into the position of the skeletal joint in the virtual scene based on the position detection result and the image related data, so as to obtain the target position of the skeletal joint. The skeleton generation module is used to generate a skeleton instance corresponding to the target virtual model based on the target location; The skinning module is used to perform skinning processing on the target virtual model based on the skeleton instance, so as to obtain the skinning processing result of the target virtual model.

[0006] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the virtual model processing method as described in any of the above embodiments.

[0007] On the other hand, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the virtual model processing method as described in any of the above embodiments by calling the computer program stored in the memory.

[0008] On the other hand, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the virtual model processing method as described in any of the above embodiments.

[0009] The virtual model processing method, virtual model processing device, computer-readable storage medium, electronic device, and computer program product provided in this application embodiment first acquire multiple target images of the target virtual model in a virtual scene and image-related data corresponding to each target image. The target images are acquired by virtual cameras at different observation positions in the virtual scene. The image-related data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. Then, the positions of the skeletal joints of the target virtual model in each target image are detected to obtain the position detection results of the skeletal joints. Based on the position detection results and the image-related data, the positions of the skeletal joints in the target images are converted into the positions of the skeletal joints in the virtual scene to obtain the target positions of the skeletal joints. Finally, based on the target positions of the skeletal joints, a skeleton instance corresponding to the target virtual model is generated. According to the generated skeleton instance, skinning is performed on the target virtual model. The process yields the skinning results of the target virtual model, thereby achieving automated binding of the virtual model. Compared to the traditional virtual model processing method where technical artists manually mark joints, build skeletons, and assign skinning weights, this method transforms complex 3D joint positioning into simple 2D image detection. By combining the geometric parameters of multi-view target image cameras, 3D coordinate transformation is achieved without manual intervention in joint marking, skeleton adjustment, and other operations. This reduces labor and time costs to a certain extent, improves the automation level and processing efficiency of virtual models, and the automated processing flow can adapt to the needs of large-scale batch processing of virtual models. The generated skeleton instances are highly matched with the target virtual model, and the skinning results have a high degree of standardization, effectively ensuring the compatibility of virtual model animation reuse. To a certain extent, this method solves the problems of high dependence on manual labor, low processing efficiency, and difficulty in animation reuse, meeting the standardized and large-scale production needs of virtual model assets in fields such as games. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating the virtual model processing method provided in the embodiments of this application.

[0012] Figure 2 This is a schematic diagram of the custom skeleton automatic binding process provided in the embodiments of this application.

[0013] Figure 3 This is a schematic diagram of the cross-view consistency filtering and weighted analytical triangulation process provided in the embodiments of this application.

[0014] Figure 4 This is a schematic diagram of the structure of the virtual model processing device provided in the embodiments of this application.

[0015] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] This application provides a method for processing virtual models, a device for processing virtual models, a computer-readable storage medium, an electronic device, and a computer program product. Specifically, the method for processing virtual models in this application can be executed by an electronic device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, smart TV, wearable smart device, smart vehicle terminal, etc. The terminal can also include a client, which can be a browser client, instant messaging client, or mini-program, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0018] It should be noted that, in this embodiment, the execution entity of the virtual model processing method can be a terminal device or a server. The terminal device can be a local terminal device or a client device in a cloud platform. This embodiment does not limit the type of execution entity.

[0019] For example, when the processing method for the virtual model runs on a terminal device, the terminal device may include a display screen and a processor. The display screen is used to present a graphical user interface (GUI) and receive instructions generated by the user interacting with the GUI. The processor is used to store applications, generate the GUI, respond to instructions, and control the display of the GUI on the display screen. When the user operates the GUI through the display screen, the GUI can control the local content of the terminal device in response to the received operation instructions. The terminal device can provide the GUI to the user in various ways, such as rendering it on the terminal device's display screen or presenting the GUI through holographic projection.

[0020] For example, when the processing method for the virtual model runs on a server, this method can be implemented and executed based on a cloud system. The cloud system includes servers and client devices. The application's runtime and the graphical user interface (GUI) presentation are separate. The storage and execution of the virtual model's processing method are completed on the server. The GUI presentation is completed on the client, which is primarily used for data reception, transmission, and GUI presentation. For example, the client can be a display device with data transmission capabilities located close to the user, such as a mobile terminal, television, computer, PDA, personal digital assistant, or head-mounted display. However, the terminal device performing data processing is the server in the cloud. During this process, the user operates the client to send instructions to the server. The server executes the instructions, encodes and compresses the GUI data, returns it to the client via the network, and finally, the client decodes and outputs the GUI.

[0021] It should be noted that, in this embodiment, the execution entity of the virtual model processing method can be a terminal device or a server. The terminal device can be a local terminal device or a client device in the aforementioned cloud system. This embodiment does not limit the type of execution entity.

[0022] The technical solution of this application will be described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0023] In the 3D game content production process, skeletal rigging and skinning are crucial steps in achieving animation-driven 3D characters. Various game projects establish strict skeletal specifications, which include bone naming, hierarchical structure, number of joints, symmetry relationships, attachment bones, and dynamic bones, among other things. These specifications are tightly coupled with animation libraries, inverse kinematics (IK) constraints, weapon attachments, and cloth and hair physics systems.

[0024] As AI-generated 3D character mesh technology continues to develop, the cost of generating character meshes has decreased and the production capacity for mass generation has increased. However, the skeletal binding and skinning processes remain key hurdles for importing characters into game engines, becoming an efficiency bottleneck and a risk point for asset consistency in the process of large-scale production of game content.

[0025] In related technologies, the skeletal binding and skinning of 3D character meshes are mainly implemented through two methods: manual operation and automated binding tools. Manual processing involves experienced technical artists manually binding bones and adjusting skinning parameters on AI-generated character meshes in batches. Automated binding tools primarily employ reference skeleton fitting, retargeting, and free skeleton generation techniques. Some automated binding solutions also introduce 2D joint priors to reduce processing difficulty, while others apply mature multi-view 2D to 3D triangulation techniques from motion capture to the character binding stage. Furthermore, some solutions generate or predict skeleton structures through 3D feature space and combine this with 3D network regression to complete the skeleton output.

[0026] However, manual processing relies heavily on the expertise of senior technical artists and requires repeated adjustments and optimizations, resulting in low efficiency and an inability to keep pace with the production capacity of AI-generated character meshes in batches. In automatic rigging tools, general online rigging solutions based on reference skeleton fitting and redirection can only output standardized reference skeletons, failing to match the custom bone naming, hierarchical structure, and attachment point requirements of specific game projects. Character assets cannot be directly reused, necessitating extensive post-processing and manual correction.

[0027] Furthermore, solutions based on free skeleton generation and 3D feature space prediction, with the goal of adapting to the input model, cannot meet the customized requirements of game projects for fixed skeleton naming, hierarchical structure, attachment bones, and dynamic skeleton sets. Additional skeleton mapping, missing bone completion, and asset consistency verification are required, increasing engineering integration costs. Automatic binding solutions that introduce 2D joint priors rely on 3D network regression algorithms for output and are constrained by fixed humanoid topology, failing to cover the output requirements of arbitrary custom skeleton specifications. Additionally, multi-view 2D to 3D triangulation solutions in the field of source autocapture do not construct deterministic processing links for engineering constraints in game character binding tasks, such as joint semantic verification, instantiation of project skeleton specification constraints, skinning processing, and asset export closed-loop, making them difficult to adapt to the actual needs of industrialized game production. Moreover, some technical solutions lack consistency filtering mechanisms for multi-view detection; occlusion and false detections affect joint localization results, and skinning weight calculations are not optimized in conjunction with semantic constraints, easily leading to poor virtual model deformation effects.

[0028] For the above issues, please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for processing a virtual model provided in an embodiment of this application. It should be noted that the steps shown may be executed in a logical order different from that shown in the flowchart. The method includes: 110: Acquire multiple target images of the target virtual model in the virtual scene and image-related data for each target image. Each target image is obtained through a virtual camera that observes the target virtual model in the virtual scene. The observation position of the virtual camera corresponding to each target image is different. The image-related data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. 120: Detect the position of the skeletal joints of the target virtual model in each target image, and obtain the position detection results of the skeletal joints; 130: Based on the position detection results and image-related data, the position of the skeletal joint in the target image is converted into the position of the skeletal joint in the virtual scene, thus obtaining the target position of the skeletal joint; 140: Based on the target location, generate a skeleton instance corresponding to the target virtual model; 150: Based on the skeleton instance, perform skinning processing on the target virtual model to obtain the skinning processing result of the target virtual model.

[0029] Specifically, in the virtual model processing, skeleton binding and skinning typically rely on manual operation. Technical artists and others need to manually mark joints, adjust the skeleton structure, and set skinning weights. However, manual joint marking is inefficient and prone to errors. 3D joint positioning lacks geometric transformation basis, skeleton generation cannot adapt to custom specifications, and skinning weight calculation lacks a standardized process. This results in virtual models being unable to quickly achieve animation-driven processing, and consistency is difficult to guarantee during batch processing. Furthermore, the automation level of virtual model processing is low, failing to form a complete closed loop from image acquisition to skinning output, thus failing to meet the needs of large-scale virtual model production. Based on the above problems, this application provides a virtual model processing method. It acquires multiple target images of the target virtual model and camera geometric parameters using a virtual camera, detects the positions of skeletal joints using 2D images, converts the 2D joint positions into 3D target positions in the virtual scene using image-related data, generates skeleton instances based on the 3D target positions, and finally completes the skinning process, achieving full automation of the virtual model skeleton generation and skinning process.

[0030] In some embodiments, a virtual scene can be understood as a three-dimensional virtual space constructed by computer graphics technology, possessing complete spatial geometric properties, and serving as a carrier for the existence and display of the target virtual model.

[0031] In some embodiments, the target virtual model can be understood as a three-dimensional mesh model that requires skeleton generation and skinning, typically a game character, virtual image, etc.

[0032] In some embodiments, the target image can be understood as a two-dimensional image of a target virtual model captured by a virtual camera in a virtual scene, which can be used for the detection of skeletal joints.

[0033] In some embodiments, image-related data can be understood as a set of auxiliary data corresponding to the target image, including at least the camera geometry parameters of the virtual camera, which can be used to support the coordinate transformation of the key points.

[0034] In some embodiments, the image-related data may also include a 3D location cache and depth data.

[0035] In some embodiments, a virtual camera can be understood as an observation device in a virtual scene that simulates the imaging principle of a real camera, and can be used to render and photograph virtual models of targets from multiple perspectives to obtain target images from different observation angles.

[0036] In some embodiments, the virtual camera observation position can be understood as the three-dimensional spatial coordinates of the target virtual model observed by the virtual camera in the virtual scene. Different positions can acquire images of the target virtual model from different perspectives. For example, the virtual camera observation position can be multiple observation perspectives such as front, back, left, and right.

[0037] In some embodiments, camera geometric parameters can be understood as parameters describing the imaging geometry of a virtual camera, typically including camera intrinsic parameters, camera extrinsic parameters, or a camera projection matrix determined by camera intrinsic and extrinsic parameters, used for the transformation calculation between two-dimensional and three-dimensional coordinates. For example, the camera projection matrix can be a 3×4 homogeneous matrix. 4×4 homogeneous matrix wait.

[0038] In some embodiments, skeletal joints can be understood as nodes that constitute the skeletal structure of the target virtual model and can be used for skeletal connection and motion control of the target virtual model.

[0039] In some embodiments, the position detection result can be understood as a set of two-dimensional position data of the detected skeletal joints in the target image.

[0040] In some embodiments, the position in the virtual scene can be understood as the coordinate data of the skeletal joints in the three-dimensional space of the virtual scene, i.e., the target position.

[0041] In some embodiments, the target position of a skeletal joint can be understood as the three-dimensional coordinates of the skeletal joint in the virtual scene, which can be used as the basis for generating skeleton instances.

[0042] In some embodiments, a skeleton instance can be understood as a three-dimensional skeletal structure composed of skeletal joints, skeletal connections, skeletal names and geometric topological relationships, which can be used to drive the motion and deformation of the target virtual model.

[0043] In some embodiments, skinning can be understood as binding the vertices of the target virtual model mesh to a skeleton instance and assigning weights to the vertices based on the influence of the skeleton, so that the virtual model can deform reasonably as the skeleton moves.

[0044] In some embodiments, the skinning result can be understood as the virtual model data after skinning is completed, which typically includes the skeleton structure and vertex skinning weights of the virtual model, and can be directly used for animation-driven and project animation library reuse.

[0045] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: In a computer-constructed virtual scene, multiple virtual cameras with different observation positions are deployed. During the offline asset construction phase, the virtual cameras are used to observe and capture images of the target virtual model, resulting in multiple target images covering different perspectives. At the same time, image-related data corresponding to each target image is collected. The image-related data includes at least the camera geometric parameters when the virtual camera observes the target virtual model. These camera geometric parameters are the geometric information of the virtual camera when observing the target virtual model, which is used to support the transformation from two-dimensional coordinates to three-dimensional coordinates, that is, to realize the coordinate transformation of the key points.

[0046] Understandably, acquiring multi-view target images can effectively avoid the occlusion problem caused by single-view observation, and the recording of camera geometric parameters provides a stable geometric basis for position transformation.

[0047] Subsequently, for each acquired target image, the two-dimensional positions of the skeletal joints of the target virtual model in the target image are detected, and the two-dimensional position detection results of each skeletal joint in each target image are obtained, so as to transform the joint localization of the three-dimensional virtual model into a two-dimensional image detection problem and reduce the difficulty of localization.

[0048] Next, the two-dimensional joint positions in the position detection results are combined with image-related data. Through a preset geometric transformation logic, the two-dimensional positions of the skeletal joints in the target image are converted into their positions in the three-dimensional space of the virtual scene. This three-dimensional position is the target position of the skeletal joint, which achieves accurate three-dimensional joint positioning to a certain extent.

[0049] Then, based on the target position of the skeletal joints and combined with the structural features of the target virtual model, a skeleton instance matching the target virtual model is generated. The skeleton instance serves as the skeletal structure that drives the movement of the virtual model, providing a data foundation for subsequent skinning processing.

[0050] Finally, using the generated skeleton instance as the binding basis, the mesh vertices of the target virtual model are associated and bound together to establish the influence relationship between the mesh vertices and the skeleton bones, thus completing the skinning process and obtaining a skinning result that can be directly used for animation-driven processing.

[0051] Thus, in this embodiment, firstly, multiple target images of the target virtual model in the virtual scene and image-related data corresponding to each target image are acquired. The target images are acquired by virtual cameras at different observation positions in the virtual scene. The image-related data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. Subsequently, the positions of the skeletal joints of the target virtual model in each target image are detected to obtain the position detection results of the skeletal joints. Based on the position detection results and the image-related data, the positions of the skeletal joints in the target images are converted into the positions of the skeletal joints in the virtual scene to obtain the target positions of the skeletal joints. Finally, based on the target positions of the skeletal joints, a skeleton instance corresponding to the target virtual model is generated. According to the generated skeleton instance, skinning processing is performed on the target virtual model to obtain the skinning processing result of the target virtual model, thereby realizing the virtual model skinning process. The automated binding of virtual models, compared to the traditional method where technical artists manually mark joints, build skeletons, and assign skinning weights, transforms complex 3D joint positioning into simple 2D image detection. By combining the geometric parameters of multi-view target image cameras to achieve 3D coordinate transformation, it eliminates the need for manual intervention in joint marking and skeleton adjustment. This reduces labor and time costs to a certain extent, improves the automation and processing efficiency of virtual models, and the automated processing flow can adapt to the needs of large-scale batch processing of virtual models. The generated skeleton instances are highly matched with the target virtual model, and the skinning processing results have a high degree of standardization, effectively ensuring the compatibility of virtual model animation reuse. To a certain extent, it solves the problems of high dependence on manual labor, low processing efficiency, and difficulty in animation reuse, meeting the standardized and large-scale production needs of virtual model assets in fields such as games.

[0052] In some embodiments provided in this application, step 120 includes: The target image is input into a pre-trained joint detection model, which then detects the positions of the skeletal joints of the target virtual model in the target image, thus obtaining the position detection results.

[0053] Specifically, in the offline asset construction phase, joint detection of virtual model skeletons largely relies on manual labeling of joint positions on images. However, this manual labeling is inefficient and its accuracy depends on the operator's experience, leading to labeling deviations during batch processing. Furthermore, some detection methods determine joints based on image edge features, failing to accurately identify the semantic positions of joints in the virtual model. The detected joint positions deviate significantly from their actual positions, making it difficult to meet the requirements of subsequent 3D coordinate transformation and skeleton generation. Based on these issues, in some embodiments provided in this application, the target image is input into a pre-trained joint detection model. This pre-trained model automatically detects the joint positions of the skeleton and outputs standardized position detection results, i.e., the positions of the target virtual model's skeleton joints in the target image.

[0054] In some embodiments, the joint detection model can be understood as a visual detection model built based on deep learning algorithms, which can be used to identify the skeletal joint positions of a target virtual model in an image, and has the ability to extract features, recognize semantics, and locate pixel coordinates.

[0055] In some embodiments, the joint detection model can also output the detection confidence score for each detection position during inference, as the probability that the joint is correctly detected from that viewpoint, for use as a reliability measure during cross-viewpoint weighted fusion. The detection confidence score typically ranges from [0,1].

[0056] In some embodiments, pre-training completion can be understood as the knuckle detection model having completed deep learning training and parameter optimization using a large number of virtual model image samples before performing the virtual model knuckle detection task, and the model reaching a convergent state, ready for direct inference.

[0057] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: During the offline asset construction phase, when performing skeletal joint location detection, the target images captured by the virtual camera are directly input into a pre-trained joint detection model. Before being put into use, this joint detection model has undergone deep learning training using a large number of virtual model image samples. The network weight parameters of the model have been fully optimized, enabling it to accurately extract the limb features and joint semantic features of the target virtual model.

[0058] After receiving the target image, the pre-trained joint detection model performs a global feature scan of the target image, locates the two-dimensional pixel coordinates of the skeletal joints of the target virtual model in the target image, removes interference information from irrelevant background areas, and integrates the two-dimensional coordinates of all joints to form a complete position detection result.

[0059] In some implementations, the joint detection model also outputs the detection confidence score for each joint, which is used to weight the reliability of observations from different perspectives.

[0060] Thus, in this embodiment, the target image acquired from the virtual scene is input into a pre-trained joint detection model. The joint detection model automatically identifies and locates the positions of the skeletal joints of the target virtual model in the target image, directly obtaining the position detection results of the skeletal joints. This achieves automated detection of the skeletal joint positions of the virtual model. Compared to the method of technical artists manually marking joint positions on the image or using simple image feature algorithms to identify joint positions, the use of a pre-trained joint detection model to achieve automated detection of skeletal joints replaces manual marking, improving the efficiency of joint detection to a certain extent and reducing the time and labor costs caused by manual operation. Moreover, the model trained by deep learning has semantic recognition capabilities, and the detection accuracy is stable and consistent, unaffected by differences in human experience, effectively reducing joint position detection errors and providing highly reliable input data for subsequent 2D coordinate to 3D coordinate conversion.

[0061] In some embodiments provided in this application, the position detection result includes the predicted position of the skeletal joints in each target image, and the camera geometric parameters include camera intrinsic parameters and camera extrinsic parameters, or include a camera projection matrix determined by the camera intrinsic parameters and camera extrinsic parameters. Step 130 includes: Based on the predicted position of the skeletal joints in a target image and the camera geometric parameters of the target image, a linear equation related to the position of the skeletal joints in the virtual scene is constructed to obtain the target linear equation corresponding to the target image. The linear equation of the target image is solved to obtain the candidate positions of the skeletal joints in the virtual scene corresponding to the target image. The target location is determined based on the candidate locations corresponding to each target image.

[0062] Specifically, in the virtual model processing during the offline asset construction phase, most methods for converting 2D coordinates to 3D coordinates do not incorporate the geometric parameters of the virtual camera. They typically rely on empirical formulas or rough estimations, resulting in low conversion accuracy. Furthermore, some conversion methods lack standardized mathematical calculation logic and unified logical construction and solution rules, leading to arbitrary calculation processes and significant deviations in the 3D position results of skeletal joints. In addition, most conversion methods cannot adapt to changes in camera parameters across different virtual scenes, resulting in poor stability of 3D positioning results. This makes it difficult to meet the accuracy requirements of subsequent skeleton instance generation and skinning processing, impacting the animation-driven effect of the virtual model. Based on these problems, some embodiments provided in this application use camera intrinsic parameters, camera extrinsic parameters, or the camera projection matrix as geometric basis, combined with the predicted positions of skeletal joints to construct linear equations. These equations are then solved to obtain candidate positions from a single viewpoint. Finally, multi-view candidate positions are fused to determine the final 3D target position of the skeletal joints in the virtual scene, achieving standardized coordinate transformation.

[0063] In some embodiments, the predicted location can be understood as the estimated two-dimensional pixel position of the skeletal joints in the target image from various viewpoints, obtained from the joint detection step. For example... .

[0064] In some embodiments, camera intrinsic parameters can be understood as parameters used to describe the internal imaging characteristics of a virtual camera, typically including information that determines the basic imaging properties of the camera, such as focal length and principal point.

[0065] In some embodiments, camera extrinsic parameters can be understood as parameters used to describe the spatial position and orientation of a virtual camera in a virtual scene, typically including information such as rotation and translation that determine the camera's observation state.

[0066] In some embodiments, the camera projection matrix can be understood as a matrix obtained by geometric operations on camera intrinsic and extrinsic parameters, which can be used to characterize the correspondence between three-dimensional spatial points projected onto a two-dimensional image plane.

[0067] In some embodiments, the linear equation can be understood as a mathematical equation constructed based on the principle of analytic geometric projection, combined with the predicted positions of skeletal joints and camera geometric parameters, which can be used to solve for the three-dimensional spatial positions of the joints. For example, based on the principle of direct linear transformation (DLT) in analytic geometry, combined with the predicted positions of skeletal joints... A weighted linear equation constructed with camera geometry parameters.

[0068] In some embodiments, the target linear equation can be understood as a linear equation constructed for a single target image and a single skeletal joint, used to solve for the three-dimensional candidate location.

[0069] In some embodiments, the candidate position can be understood as the estimated three-dimensional coordinates of the skeletal joints in the virtual scene obtained by solving the target linear equation corresponding to a single target image.

[0070] In some embodiments, the solution process can be understood as the operation of performing numerical operations on the constructed target linear equation using methods such as Singular Value Decomposition (SVD) to obtain the corresponding solution of the equation.

[0071] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When performing coordinate transformation of skeletal joints during the offline asset construction phase, the camera geometric parameters, including camera intrinsic and extrinsic parameters, are first obtained from the image-related data, or the camera projection matrix is ​​calculated from the camera intrinsic and extrinsic parameters. This is used to characterize the imaging geometry of the virtual camera.

[0072] Subsequently, the predicted positions of the skeletal joints in each target image are extracted from the position detection results. These predicted positions are the two-dimensional pixel coordinates of the joints in the target image. For each skeletal joint, the predicted position of each joint in a single target image is combined with the camera's geometric parameters. Based on the analytical geometric projection constraints in 3D space, a DLT equation, also known as the target linear equation, is constructed to describe the mapping relationship between the 2D position of the skeletal joint and the 3D position of the virtual scene. The 3D analytical geometric projection constraints can be:

[0073] in, The coordinates of the skeletal joints are three-dimensional homogeneous coordinates. For projection depth factor, For the first A camera projection matrix with multiple viewpoints.

[0074] After the target linear equation is constructed, stable numerical calculation methods such as SVD are used to solve the target linear equation. The solution of the equation is obtained through mathematical operations. The solution is the three-dimensional coordinate of the skeletal joint in the virtual scene under the current target image view, which is also the candidate position of the skeletal joint under the current target image view.

[0075] Subsequently, following the same processing logic, all target images are traversed, generating multiple candidate positions for each skeletal joint from different viewing angles. After obtaining the candidate positions for each target image, the candidate positions are fused and calculated to finally obtain the target position of each skeletal joint in the virtual scene.

[0076] Thus, in this embodiment, the camera geometric parameters in the image-related data include camera intrinsic and extrinsic parameters, or a camera projection matrix determined by the camera intrinsic and extrinsic parameters. Based on the predicted position of the skeletal joints in the target image and the camera geometric parameters corresponding to the target image, a target linear equation related to the position of the skeletal joints in the virtual scene is constructed, and the target linear equation is solved to obtain candidate positions of the skeletal joints corresponding to a single target image. Then, the target position of the skeletal joints is determined based on the candidate positions corresponding to each target image, thereby achieving a standardized conversion of the skeletal joints from the two-dimensional position in the target image to the three-dimensional position in the virtual scene. Compared to... The scheme that relies on experience-based judgment or rough calculation to complete the coordinate transformation of skeletal joints achieves coordinate transformation by constructing a standardized target linear equation. By relying on camera geometric parameters and abandoning experience-based estimation, it improves the positioning accuracy of 2D to 3D skeletal joints to a certain extent, provides reliable data support for subsequent skeleton instance generation, effectively improves the matching degree between the skeleton and the virtual model, and thus optimizes the skinning effect to a certain extent. It ensures the smoothness and realism of the virtual model's animation drive. Moreover, the logic of equation construction and solution is unified and reproducible, and it can adapt to different virtual scenes and virtual cameras with different parameters, thus ensuring the stability and consistency of coordinate transformation results to a certain extent.

[0077] Furthermore, by solving for single-view candidate positions and then fusing multi-view candidate positions to obtain the target position, the effective data from different observation perspectives can be fully utilized. This reduces the positional error caused by occlusion or imaging deviation in a single perspective to a certain extent, thereby enhancing the stability and accuracy of determining the three-dimensional position of skeletal joints.

[0078] In some embodiments provided in this application, the step of determining the target location based on the candidate locations corresponding to each target image includes: The candidate positions of the skeletal joints in each target image are reprojected onto the corresponding target image to obtain the reprojection position for each target image. Determine the reprojection error between the reprojection position corresponding to the target image and the predicted position in the position detection result corresponding to the target image; Based on the reprojection error corresponding to each target image, the weighting weight of the candidate position corresponding to each target image is determined; Based on the weighted weights of the candidate positions corresponding to each target image, the weighted solution is performed on all candidate positions of each skeletal joint to obtain the target position.

[0079] Specifically, in the coordinate transformation stage of virtual model processing, the candidate positions of skeletal joints obtained by solving the linear equation of the target from a single viewpoint are prone to varying degrees of deviation due to factors such as viewpoint differences and observation occlusion. Simply fusing candidate positions from multiple viewpoints directly cannot accurately identify and eliminate observations with large errors, resulting in insufficient precision in the final determined target positions of skeletal joints. This affects the generation quality of the skeleton instance, reduces the matching degree between the skeleton instance and the target virtual model, leads to poor skinning effects, and prevents the virtual model from achieving smooth animation. Based on the above problems, in some embodiments provided in this application, the reprojection error is calculated by reprojecting the candidate positions back to the target image. Weights are then assigned to the candidate positions from each viewpoint based on the reprojection error, and the multi-view candidate positions are fused through weighted solving to finally obtain stable target positions of skeletal joints.

[0080] In some embodiments, reprojection can be understood as the calculation process of remapping the three-dimensional candidate positions of skeletal joints in a virtual scene, combined with camera geometric parameters, onto the corresponding two-dimensional plane of the target image.

[0081] In some embodiments, the reprojection position can be understood as the pixel coordinates formed on the two-dimensional plane of the target image after the candidate positions of the skeletal joints have been reprojected.

[0082] In some embodiments, the reprojection error can be understood as the numerical difference between the reprojected position and the predicted position in the position detection result, and is used to measure the accuracy of the candidate position.

[0083] In some embodiments, the weighting weight can be understood as a numerical coefficient determined based on the magnitude of the reprojection error, used to adjust the contribution ratio of candidate positions from different viewpoints in the final target position calculation.

[0084] In some embodiments, the weighted solution process can be understood as combining the weights corresponding to each candidate position and performing numerical calculations on the multi-view candidate positions to obtain the final coordinates. For example, weighted least squares or weighted SVD.

[0085] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: After calculating the candidate locations, the candidate locations of the skeletal joints in each target image are determined by combining the camera geometry parameters of each icon image. The image is then reprojected onto the corresponding target image from which the candidate location was generated, resulting in the reprojected location for each target image. .

[0086] Subsequently, the numerical difference between the reprojection position corresponding to the target image and the predicted position in the target image position detection result is determined. This numerical difference is the reprojection error corresponding to the candidate position of the current viewpoint. Among them, reprojection error The calculation formula is:

[0087] in, For reprojection error, This represents the predicted location of skeletal joints in each target image. The reprojection position is the pixel coordinate obtained after the 3D candidate position is remapped onto the image plane through the camera projection matrix.

[0088] Next, based on the reprojection error The magnitude of the error is used to assign a corresponding weight to the candidate position corresponding to each target image. In some embodiments, the magnitude of the error is negatively correlated with the weight, i.e., the reprojection error... The larger the value, the smaller the effective weight of this perspective in the weighted fusion.

[0089] Finally, based on the assigned weights, a weighted solution is performed on the candidate positions of all views corresponding to a single skeletal joint. By integrating the candidate position information from multiple views through weighted operations, the target position of the skeletal joint is finally obtained.

[0090] Thus, in this embodiment, the candidate positions of the skeletal joints in each target image are reprojected onto the corresponding target image to obtain the reprojected positions. The reprojection error between the reprojected positions in the target image and the predicted positions in the position detection results is determined. Based on the reprojection error of each target image, a weighted weight is determined for the candidate positions. The target positions of each skeletal joint are then calculated using this weighted weight, thereby achieving the conversion of the two-dimensional positions of the skeletal joints in the virtual model to the three-dimensional target positions. Compared to the simple fusion of candidate positions from multiple perspectives, reprojecting the candidate positions onto the corresponding target images... Image reprojection location acquisition allows for intuitive verification of the accuracy of candidate locations from multiple perspectives. Calculating the reprojection error between the reprojection location and the predicted location effectively quantifies the deviation between the candidate location and the actual detected location. Furthermore, the weighting of candidate locations is determined based on the reprojection error, and weighted calculation is performed on all candidate locations based on this weighting. This effectively eliminates interference from candidate locations with excessive errors and integrates highly accurate candidate location data from multiple perspectives. This, to a certain extent, improves the accuracy of calculating the target location of skeletal joints in the virtual scene, enhances the stability of the target location calculation results, and provides accurate joint location data for skeleton instance generation.

[0091] In some embodiments provided in this application, the step of determining the weighted weight of the candidate position corresponding to each target image when determining the target position based on the reprojection error includes: When the reprojection error exceeds the error threshold, reduce the weight of the corresponding candidate position; or, When the reprojection error exceeds the error threshold, the corresponding candidate position is deleted.

[0092] Specifically, in the process of determining the weighting of candidate positions based on reprojection error, some candidate positions contain abnormal data due to limb occlusion, detection bias, etc. The reprojection error of these candidate positions with abnormal data is relatively large. If they are directly included in the weighting calculation, it will affect the calculation accuracy of the target position, causing deviations in the 3D coordinates of the skeletal joints. This results in insufficient matching between the skeleton instance and the target virtual model, and the skinning effect does not meet the requirements of animation-driven processing, affecting the processing quality of the virtual model. Based on the above problems, in some embodiments provided in this application, a fixed error threshold is set as the judgment criterion. Candidate positions exceeding the threshold range are subject to reduced weighting or direct deletion, thereby optimizing the weighting allocation rules, eliminating abnormal data interference, and improving the calculation accuracy of the target position.

[0093] In some embodiments, the error threshold can be understood as a pre-set numerical standard used to determine whether the reprojection error of a candidate position is within a reasonable range, and can be used to distinguish between valid and abnormal candidate positions. For example, the error threshold can be arbitrarily set between 3 and 8 pixels. In some embodiments, the error threshold can be adaptively set by the configuration file according to the rendering resolution or character scale.

[0094] In some embodiments, reducing the weighting can be achieved by multiplying the effective weight of the viewpoint by a decay coefficient. ,in, 0, when This is equivalent to eliminating candidate positions from that observation perspective.

[0095] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: The reprojection error corresponding to each candidate position is calculated. Then, the pre-set function for determining reprojection error is invoked. Is the error threshold reasonable? The reprojection error of the candidate position corresponding to each target image With this error threshold Compare the numerical values.

[0096] When the comparison results show reprojection error at the candidate position Greater than the error threshold When this happens, you can choose to perform either of the following two processing methods: The first approach is to reduce the reprojection error. The corresponding weighted weight of the candidate position, for example, multiplying the effective weight of the viewpoint by an attenuation coefficient to reduce the contribution ratio of the candidate position in the final target position weighted solution process; The second approach is to directly address the reprojection error. The corresponding candidate positions are deleted so that they are not included in the weighted solution process.

[0097] Understandably, through the above targeted processing, a scientific allocation of weights for all candidate positions was achieved, providing a reliable data foundation for obtaining accurate target positions through weighted solutions.

[0098] Thus, in this embodiment, when the reprojection error exceeds the error threshold, the weighting of the corresponding candidate position is reduced or the corresponding candidate position is deleted. This achieves candidate position selection and weight optimization based on the reprojection error. Compared with the scheme of directly using candidate positions to participate in the weighted solution, the quantitative judgment of abnormal candidate positions is achieved by using the error threshold. This can effectively filter abnormal candidate data with excessive reprojection error, weaken the computational interference caused by single-view observation bias or false detection of occlusion, and thus improve the calculation accuracy of the three-dimensional target position of the skeletal joint to a certain extent. This makes the target position more consistent with the real spatial position of the skeletal joint in the virtual scene, and ensures the stability and reliability of the coordinate transformation results of the skeletal joint.

[0099] In some embodiments provided in this application, the image-related data further includes a three-dimensional position cache, which includes the three-dimensional position of each pixel in the target image in the virtual scene. Step 130 includes: Based on the position detection results, the 3D position that matches the position detection results is read from the 3D position cache to obtain the target position of the skeletal joint.

[0100] Specifically, the coordinate transformation of skeletal joints in the virtual model processing flow requires constructing complex mathematical equations and iteratively solving them. This computational process is cumbersome and lengthy, consuming significant computing resources and placing high demands on processing equipment performance. Furthermore, errors arising from numerical calculations can reduce the accuracy of 3D positioning of skeletal joints to some extent. In addition, real-time geometric calculations are time-consuming and cannot meet the batch processing needs of large-scale virtual models, slowing down the production progress of virtual model assets and failing to meet the efficiency requirements of industrial production in fields such as gaming. Based on the above problems, in some embodiments provided in this application, a 3D position cache is added to the image-related data. By matching the position detection results with the 3D position cache lookup table, the 3D target position of the skeletal joints in the virtual scene is directly obtained, simplifying the coordinate transformation process to some extent and improving the efficiency and accuracy of coordinate transformation.

[0101] In some embodiments, the three-dimensional position cache can be understood as three-dimensional position buffer cache data generated synchronously when rendering the target image, which stores the three-dimensional spatial position information of each pixel in the target image in the virtual scene.

[0102] In some embodiments, the foreground pixel caches the world coordinates of the three-dimensional surface point it hits.

[0103] In some embodiments, matching can be understood as the process of finding the corresponding pixel's three-dimensional position data in the three-dimensional position cache by using the two-dimensional pixel coordinates of the skeletal joints in the target image as the retrieval index.

[0104] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When performing the conversion from the two-dimensional position of skeletal joints to the three-dimensional position of the virtual scene, the image-related data includes an additional three-dimensional position cache in addition to the camera geometric parameters. The three-dimensional position cache is cached data generated synchronously during the rendering of the target image in the virtual scene. It is output along with the regular red, green, blue (RGB) target image and stores the three-dimensional spatial position information of each pixel in the target image in the virtual scene.

[0105] During coordinate transformation, the position detection results of the skeletal joints are first extracted, that is, the two-dimensional pixel coordinates of the skeletal joints in the target image.

[0106] Subsequently, using the two-dimensional pixel coordinates as the retrieval index, a corresponding search is performed in the three-dimensional position cache. Without constructing any mathematical equations or projection constraints, the three-dimensional position data that matches the two-dimensional pixel coordinates is found.

[0107] Finally, the successfully matched 3D position data is read and used to determine the target position of the skeletal joint in the virtual scene.

[0108] Thus, in this embodiment, image-related data corresponding to the target virtual model in the virtual scene is first obtained. The image-related data also includes a three-dimensional position cache in addition to the original parameters. The three-dimensional position cache can completely store the three-dimensional spatial position of each pixel in the target image in the virtual scene. Then, based on the position detection results of the skeletal joints, the three-dimensional position data matching the position detection results is directly read from the three-dimensional position cache to determine the target position of the skeletal joints in the virtual scene. This realizes the conversion of the skeletal joints from the two-dimensional position of the target image to the three-dimensional position of the virtual scene. Compared with the coordinate transformation scheme that requires the construction of complex mathematical equations and the execution of multiple solution operations, the coordinate transformation is realized by the lookup table matching method of the three-dimensional position cache. This eliminates the complex equation construction and solution process, simplifies the operation steps of coordinate transformation to a certain extent, reduces the performance dependence of computing devices, improves the processing speed of coordinate transformation, and can adapt to the batch processing scenario of large-scale virtual models. In this way, it improves the automation level and industrial production efficiency of virtual model processing to a certain extent, and provides strong support for the rapid production of virtual model assets in fields such as games.

[0109] In some embodiments provided in this application, the image-related data further includes depth data, which includes the depth of each pixel in the target image. Step 130 includes: Based on the position detection results and depth data, the positions of the skeletal joints in the target image are back-projected into the virtual scene to obtain the target positions of the skeletal joints.

[0110] Specifically, virtual model processing typically relies on solving complex mathematical equations to achieve coordinate transformation of skeletal joints. However, this is cumbersome, consumes significant computational resources, and is inefficient. Furthermore, some solutions use pre-stored 3D position caches for coordinate transformation, but this only adapts to specific rendering configurations, resulting in poor compatibility and versatility. Additionally, some solutions calculate the 3D position of skeletal joints using planar coordinates, lacking depth data support. This leads to inaccuracies in the depth positioning of skeletal joints in the virtual scene, resulting in insufficient 3D position accuracy. This fails to meet the precision requirements of subsequent skeleton instance generation and skinning, making it difficult to adapt to diverse rendering pipelines and virtual model processing scenarios. To address these issues, some embodiments provided in this application add depth data to the image-related data. Combined with the position detection results of skeletal joints, a back-projection geometric calculation method is used to convert the 2D position of skeletal joints in the target image into a 3D target position in the virtual scene, improving the accuracy, versatility, and processing efficiency of coordinate transformation to a certain extent.

[0111] In some embodiments, depth data can be understood as numerical data generated synchronously when rendering the target image, used to record the depth distance of each pixel in the target image in the virtual scene.

[0112] In some embodiments, back projection can be understood as a geometric calculation process that restores points on the image plane to the three-dimensional space of the virtual scene based on linear depth values, combined with two-dimensional pixel coordinate depth data and camera geometric relationships.

[0113] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When converting the two-dimensional position of skeletal joints to the three-dimensional position of the virtual scene, the image-related data includes depth data in addition to the original camera geometric parameters. The depth data is a linear depth value of the depth buffer generated synchronously during the rendering of the target image in the virtual scene, recording the depth distance of each pixel in the target image within the virtual scene.

[0114] During the coordinate transformation process, the two-dimensional position detection results of the skeletal joints are first extracted. These two-dimensional position detection results are the two-dimensional pixel coordinates of the skeletal joints in the target image.

[0115] Subsequently, the depth data corresponding to the two-dimensional pixel coordinates is retrieved, and combined with the imaging geometry of the virtual camera, following the principle of three-dimensional spatial back projection, the skeletal joints are projected from the two-dimensional plane of the target image into the three-dimensional space of the virtual scene.

[0116] The aforementioned back-projection calculation process obtains the three-dimensional coordinates of the skeletal joints in the virtual scene through rigorous geometric calculations, which is the target position of the skeletal joints. To a certain extent, it integrates two-dimensional pixel coordinates and depth data, making up for the missing depth dimension position information of the two-dimensional coordinates and improving the accuracy of the three-dimensional positioning of the skeletal joints.

[0117] Thus, in this embodiment, image-related data corresponding to the target virtual model in the virtual scene is first obtained. The image-related data includes depth data in addition to the original parameters. The depth data records the depth value of each pixel in the target image in the virtual scene. Then, based on the position detection results of the skeletal joints and the depth data, the position of the skeletal joints in the target image is back-projected into the virtual scene to determine the target position of the skeletal joints in the virtual scene. This realizes the transformation of the skeletal joints from the two-dimensional position of the target image to the three-dimensional position of the virtual scene. Compared with coordinate transformation schemes that lack depth dimension support or rely on solving complex equations, the coordinate transformation achieved by combining depth data with back-projection effectively supplements the depth dimension information of the two-dimensional coordinates, ensuring the accuracy of the three-dimensional positioning of the skeletal joints to a certain extent, reducing the performance requirements of computing devices, improving the processing speed of coordinate transformation, and adapting to various rendering scenarios that only output depth data. This, in turn, improves the versatility and compatibility of the virtual model processing method to a certain extent, provides reliable data support for subsequent skeleton instance generation, optimizes the skinning effect, and ensures the smoothness and realism of the virtual model animation drive.

[0118] In some embodiments provided in this application, the position detection result includes the predicted position of the skeletal joint in each target image and the predicted probability of the predicted position. The skeletal joint includes multiple joints. Step 130 includes: Based on the predicted probability of each skeletal joint in each target image, determine the target weight corresponding to each skeletal joint in each target image; Based on the predicted positions of the skeletal joints in each target image and the image-related data corresponding to each target image, the predicted positions of the skeletal joints in each target image are converted into the positions of the skeletal joints in the virtual scene, thus obtaining the predicted converted positions of the skeletal joints in each target image. Based on the target weight of each skeletal joint, the predicted transformation position of each skeletal joint is weighted and solved to obtain the target position of each skeletal joint.

[0119] Specifically, the 3D localization of skeletal joints in virtual models typically relies on single-view images for detection and conversion. However, single-view observation is susceptible to limb occlusion and viewing angle deviations, leading to inaccuracies in joint detection results and insufficient 3D positioning accuracy. Furthermore, single-view data cannot cover the entire orientation of the target virtual model, resulting in blind spots and reducing the reliability of the localization results. In addition, multi-view detection results lack an effective fusion mechanism, failing to integrate effective information from different perspectives and unable to mitigate the impact of single-view errors. This results in poor robustness of 3D skeletal joint localization, making it difficult to meet the accuracy requirements of subsequent skeleton generation and skinning processing. Based on these problems, some embodiments provided in this application employ multiple target images from multiple observation positions to acquire multi-dimensional detection data. Target weights are assigned according to prediction probabilities, and the predicted positions from each perspective are converted into predicted transformed positions before weighted solving. The multi-view information is then fused to obtain the target position of the skeletal joint.

[0120] In some embodiments, the target image can be understood as multiple two-dimensional images generated by virtual cameras at different observation positions capturing images of the target virtual model, used to provide multi-directional keypoint detection baseline data. For example, the target virtual model can be captured at multiple observation positions (front / back / left / right) to cover the main limb areas of the target virtual model, reducing occlusion and blind spots.

[0121] In some embodiments, the predicted location can be understood as the estimated two-dimensional pixel position of the skeletal joints in the target image from various viewpoints, obtained from the joint detection step. For example... .

[0122] In some embodiments, the prediction probability can be understood as a numerical value used to measure the reliability of the predicted location detection results for skeletal joints; a higher value indicates a more reliable detection result. For example, the prediction probability is... The range of values ​​is .

[0123] In some embodiments, the target weight can be understood as a numerical coefficient assigned based on the prediction probability, used to adjust the contribution of each viewpoint's predicted transformation position in the weighted calculation. For example, the target weight is... ,in, , To truncate linear functions or softmax normalization functions, Visibility coefficient, It can take any value between 0 and 1 or a continuous visibility value.

[0124] In some embodiments, the predicted transformation position can be understood as the virtual scene's three-dimensional position obtained by transforming the predicted positions of the skeletal joints from each viewpoint.

[0125] In some embodiments, the weighted solution process can be understood as a mathematical operation process that uses the target weight as a coefficient to perform a fusion calculation on the multi-view predicted transformation position to obtain the final target position.

[0126] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When performing coordinate transformation of skeletal joints, the target image is generated by virtual cameras taken from multiple different observation positions. For example, in a scene where realistic game characters are mass-bound, six preset perspectives can be used: front / back / left / right / left front 45° / right front 45°.

[0127] Multiple images are used to cover the target virtual model from different angles, effectively reducing occlusion and blind spots. The location detection results include the predicted location of each skeletal joint in each target image. and the corresponding predicted probability The predicted probability is used to characterize the reliability of the joint detection results from the corresponding viewpoint.

[0128] During the coordinate transformation process, firstly, for each skeletal joint, a corresponding target weight is assigned according to the predicted probability value of that skeletal joint in each target image. The higher the predicted probability of a detection result, the greater the target weight is assigned, so as to ensure that the high-confidence detection result dominates the calculation.

[0129] Next, for each skeletal joint in each target image, the predicted position is converted into a three-dimensional position in the virtual scene by combining the image-related data corresponding to that target image, thus obtaining the predicted converted position from the corresponding viewpoint.

[0130] Finally, for each skeletal joint, the predicted transformation position of all viewpoints is weighted and calculated using the assigned target weight as the calculation coefficient to obtain the target position of each skeletal joint in the virtual scene. By fusing the multi-view detection results through mathematical operations, single-view bias and interference information are effectively eliminated.

[0131] Thus, in this embodiment, firstly, multiple target images with different observation positions of the virtual camera in the virtual scene are acquired, and the predicted positions and corresponding predicted probabilities of multiple skeletal joints in each target image are obtained. Based on the predicted probability of each skeletal joint in each target image, the target weight corresponding to each skeletal joint in each target image is determined. Then, based on the predicted positions of the skeletal joints in each target image and the image-related data corresponding to each target image, the predicted positions of the skeletal joints in each target image are converted into predicted transformed positions of the skeletal joints in the virtual scene. Finally, the predicted transformed positions of each skeletal joint are weighted and calculated according to the target weight of each skeletal joint to obtain the predicted position of each skeletal joint in the virtual scene. The target location in the scene is thus calculated by fusing the 3D position of the skeletal joints based on multi-view image data. Compared with the solution that relies on a single view image to complete the transformation of the skeletal joint position, the multi-view image acquisition method fully covers the observation orientation of the target virtual model, which reduces the detection error caused by occlusion and blind spots to a certain extent. Moreover, the weighted solution processing effectively fuses the effective information from multiple views, which improves the accuracy and robustness of the 3D positioning of the skeletal joints to a certain extent. This provides reliable data support for the subsequent skeleton instance generation, improves the matching degree between the skeleton and the target virtual model, optimizes the skinning effect, ensures the smoothness and realism of the virtual model animation, and adapts to the processing scenarios of virtual models with complex postures and complex structures.

[0132] In some embodiments provided in this application, before the step of determining the target weight corresponding to each skeletal joint in each target image based on the predicted probability corresponding to each skeletal joint in each target image, the method further includes: If the predicted probability of a skeletal joint in the target image is less than a preset probability threshold in the position detection results, the position detection results are deleted.

[0133] Specifically, in the process of obtaining the target position of skeletal joints through weighted solving of multi-view detection results, multi-view detection generates a large number of position detection results. Among these, some position detection results have low prediction probabilities and are considered low-reliability data with no reference value. If such invalid data directly enters the target weight allocation and weighted solving stage, it will interfere with the logic of weighted fusion calculation, reduce the accuracy of 3D positioning of skeletal joints, and cause the target position to deviate from the actual spatial position. Moreover, such invalid data will increase unnecessary computation, consume processing resources, slow down the overall efficiency of coordinate transformation, and thus affect the quality and progress of subsequent skeleton generation and skinning processing. Based on the above problems, in some embodiments provided in this application, a position detection result screening process is added to directly delete low-reliability data with prediction probabilities lower than a preset probability threshold, and only retain valid data to participate in subsequent calculations, thereby improving positioning accuracy and processing efficiency.

[0134] In some embodiments, the preset probability threshold can be understood as a pre-set probability threshold used to determine whether a location detection result can be interpreted as valid, that is, a probability threshold for distinguishing between high-confidence and low-confidence location detection results. For example, the preset probability threshold is... The value ranges from 0.2 to 0.5 and can be set based on engineering experience or adjusted adaptively by the configuration file according to the rendering resolution and character scale.

[0135] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: Before performing the operation of determining the target weight based on the predicted probability, the position detection results of all skeletal joints in each target image are first verified one by one.

[0136] When verifying the location detection results, the predicted probability value corresponding to each location detection result can be extracted and compared with the preset probability threshold.

[0137] When the predicted probability of a detection result at a certain location is less than a preset probability threshold, the detection result at that location is determined to be low-confidence invalid data and is directly removed from the dataset, preventing it from participating in subsequent target weight allocation and weighted solution processing. For example, in a scenario of batch binding of realistic game characters, the preset probability threshold can be set to 0.3, and keypoint detection results with a predicted probability lower than 0.3 in all viewpoints can be directly discarded.

[0138] Finally, after the screening is completed, based on the high-confidence location detection results that meet the predicted probability criteria, the subsequent target weight allocation and weighted solution processing are performed.

[0139] Thus, in this embodiment, before determining the target weight of each skeletal joint in each target image based on the predicted probability of each skeletal joint in each target image, it is determined whether the predicted probability of the skeletal joint in the position detection result of a single skeletal joint in the target image is less than a preset probability threshold. If the determination result is yes, the position detection result is directly deleted. This achieves automatic identification and elimination of low-confidence position detection results. Compared with the scheme of directly assigning target weights without filtering low-prediction probability detection results, the pre-screening and elimination of low-confidence data effectively eliminates the interference of invalid information on subsequent calculations, improves the calculation accuracy of the three-dimensional target position of skeletal joints to a certain extent, makes the positioning result more consistent with the real spatial position of the joints in the virtual scene, and at the same time reduces the extra calculations caused by invalid data to a certain extent, reduces resource consumption, and improves the processing speed of coordinate transformation.

[0140] In some embodiments provided in this application, before the step of determining the target weight corresponding to each skeletal joint in each target image based on the predicted probability corresponding to each skeletal joint in each target image, the method further includes: If the pixel at the predicted position in the target image does not hit the surface of the foreground object in the target image, the target weight of the skeletal joint corresponding to the predicted position in the target image is set to zero, or the position detection result corresponding to the skeletal joint corresponding to the predicted position in the target image is deleted.

[0141] Specifically, in the process of obtaining the target position of skeletal joints through weighted solving of multi-view detection results, multi-view detection generates a large number of position detection results. Some of the predicted positions obtained by the detection deviate from the actual area of ​​the target virtual model, with the corresponding pixels falling in the background area of ​​the target image or the surface of a non-foreground object. Such position detection results are invalid data without practical significance. However, if such invalid data directly enters the target weight allocation and weighted solving stage, it will interfere with the logic of weighted fusion calculation, reduce the accuracy of 3D positioning of skeletal joints, and cause the target position to deviate from the real spatial position. Moreover, such invalid data will increase unnecessary computation, occupy processing resources, slow down the overall efficiency of coordinate transformation, and thus affect the quality of subsequent skeleton generation and skinning processing, making it impossible to guarantee the stability and reliability of virtual model processing. Based on the above problems, in some embodiments provided in this application, a foreground surface judgment process is added. For invalid detection results where the predicted position pixels do not hit the foreground object surface, the target weight is reset to zero or directly deleted, and only valid data is retained to participate in subsequent calculations, thereby improving positioning accuracy and processing efficiency.

[0142] In some embodiments, the foreground object can be understood as a virtual model of the target in the target image that serves as the subject of observation.

[0143] In some embodiments, the foreground object surface can be understood as the set of pixels corresponding to the visible area of ​​the target virtual model presented in the target image.

[0144] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: Before performing the operation of determining the target weight based on the predicted probability, the pixel attribution of each skeletal joint in each target image is first checked to determine whether the pixel corresponding to the predicted position is within the surface range of the foreground object in the target image. The foreground object is the target virtual model that needs to be processed.

[0145] If the determination result is that the pixel at the predicted position does not hit the surface of the foreground object, that is, the pixel falls in the background area or a non-model surface area, the position detection result corresponding to the predicted position is determined to be invalid data.

[0146] Subsequently, for the aforementioned invalid data, one of the following two standardization methods will be performed: The first approach is to directly set the target weight corresponding to the skeletal joint in the current target image to zero, so that the detection result at that location does not contribute any data in the subsequent weighted solution process.

[0147] The second approach is to completely remove the location detection result corresponding to the skeletal joint from the dataset, so that the location detection result does not participate in the subsequent target weight allocation and weighted solution calculation.

[0148] Finally, after completing the foreground surface determination and invalid data processing, the subsequent target weight determination and weighted solution operations are performed based on the remaining valid location detection results.

[0149] Thus, in this embodiment, before determining the target weight of each skeletal joint in each target image based on the predicted probability of each skeletal joint in each target image, it is determined whether the pixel corresponding to the predicted position of the skeletal joint in the target image hits the surface of the foreground object in the target image. When the determination result is negative, the target weight of the skeletal joint corresponding to the predicted position in the target image is set to zero, or the position detection result corresponding to the skeletal joint in the target image is deleted. This achieves automatic identification and processing of invalid detection results outside the surface of the foreground object, compared to not processing the predicted position. The scheme of directly assigning target weights by judging the foreground surface effectively filters out invalid detection results, eliminates interference from background areas and non-model surface data to a certain extent, improves the calculation accuracy of the 3D target position of the skeletal joints, and thus provides solid support for subsequent skeleton instance generation. It also improves the matching degree between the skeleton and the target virtual model, optimizes the skinning effect, and ensures the smoothness and realism of the virtual model's animation drive. Furthermore, the method of resetting the target weights to zero and deleting them is simple and efficient, reducing resource consumption caused by invalid calculations and improving the processing speed of coordinate transformation.

[0150] In some embodiments provided in this application, step 140 includes: Based on the target locations of the skeletal joints and the pre-determined skeleton configuration data, a skeleton instance is generated. The skeleton configuration data is used to indicate the bones contained in the skeleton instance, the correspondence between the skeletal joints and the bones in the skeleton instance, the generation rules for generating the corresponding bones in the skeleton instance based on the target locations of the skeletal joints, the name of each bone in the skeleton instance, and the geometric topological relationship of the bones in the skeleton instance.

[0151] Specifically, the skeleton generation schemes for related virtual models cannot adapt to the project's custom skeleton specifications. The generated skeletons lack unified standards in bone composition, joint-bone correspondence, bone generation logic, bone naming, and geometric topology, failing to directly meet the actual needs of animation reuse in projects. Furthermore, some skeleton generation schemes can only generate free skeletons without specification constraints, requiring manual modification of bone names, adjustment of hierarchical topology, and completion of missing bones in post-processing. However, this manual operation is costly, and the consistency of skeletons cannot be guaranteed during batch processing. In addition, some skeleton generation schemes rely on fixed template skeletons, failing to meet the custom bone configuration requirements of different projects, increasing the difficulty of animation library reuse and slowing down the industrial production efficiency of virtual model assets. Based on the above problems, in some embodiments provided in this application, skeleton instances are generated according to the specification requirements of the configuration data, based on the target positions of bone joints, to ensure that the generated skeleton instances conform to the project's custom bone standards.

[0152] In some embodiments, the pre-determined skeleton configuration data can be understood as project specification data that has been set in advance and stored in formats such as YAML or JSON. It typically includes project specification data such as skeleton composition, joint-skeleton correspondence, skeleton generation rules, skeleton names, and skeleton geometric topology.

[0153] In some embodiments, a skeleton instance can be understood as a virtual model skeleton structure that conforms to project specifications and includes main kinematic bones, attachment bones, and dynamic bones, generated based on the target positions of skeletal joints and skeleton configuration data.

[0154] In some embodiments, the generation rules can be understood as the specific execution standards and logic of the corresponding bones in the skeleton instance constructed based on the target positions of the skeletal joints. These rules typically include four categories: interpolation, extrapolation, offset, and chain splitting. Examples include interpolate, extend, offset, and chain_split.

[0155] In some embodiments, geometric topological relationships can be understood as the spatial connection methods, hierarchical affiliations, and positional relationships between bones in a skeleton instance. For example, parent-child hierarchical affiliation, symmetrical bone pair association, and spatial structure of bone connection chains.

[0156] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When generating a skeleton instance, the system first retrieves the pre-defined skeleton configuration data. This skeleton configuration data is a pre-defined specification document that clearly defines the types and number of bones that the skeleton instance should include, clearly defines the matching relationship between each bone joint and the corresponding bone in the skeleton instance, sets the specific execution rules for generating the corresponding bone based on the target position of the bone joint, and uniformly specifies the name of each bone in the skeleton instance, as well as the spatial connection method and hierarchical relationship between all bones.

[0157] Subsequently, based on the target positions of the skeletal joints in the virtual scene, the three-dimensional target positions of the skeletal joints are semantically mapped to the skeleton configuration data. According to the correspondence in the configuration data, each joint is associated with the corresponding bone specified in the configuration.

[0158] Next, following the generation rules in the configuration data, each bone is constructed by combining the spatial coordinates of the target position of the joint. At the same time, each bone is given a specified name according to the configuration data. Based on the geometric topology, the hierarchical construction and spatial connection of all bones are completed, and finally a skeleton instance that conforms to the skeleton configuration data specification is generated.

[0159] Thus, in this embodiment, a skeleton instance corresponding to the target virtual model is generated based on the target position of the skeletal joints and the pre-determined skeleton configuration data. The skeleton configuration data indicates all the bones included in the skeleton instance, the correspondence between skeletal joints and bones in the skeleton instance, the generation rules for the corresponding bones in the skeleton instance generated by the target position of the skeletal joints, the name of each bone in the skeleton instance, and the geometric topological relationship of the bones in the skeleton instance. This achieves automated processing of generating standardized skeleton instances according to predetermined specifications. Compared with the scheme of arbitrarily building skeleton structures based solely on the position of skeletal joints, generating skeleton instances through hard constraints of skeleton configuration data ensures to a certain extent that the skeleton's skeletal composition, joint correspondence, generation rules, bone names, and geometric topology conform to the project's custom specifications, eliminating the need for manual modification and adjustment in the later stages, reducing manual operation costs. Furthermore, the standardized skeleton generation method effectively ensures the consistency of the skeleton structure when processing batch virtual models. The generated skeleton can be directly reused from the project's animation library, improving the reuse efficiency of animation assets to a certain extent.

[0160] In addition, the custom skeleton configuration data can be flexibly adapted to the skeleton requirements of different projects, which to some extent broadens the scope of application of virtual model processing methods and meets the industrial production standard requirements of diverse virtual model assets.

[0161] In some embodiments provided in this application, the skeleton instance includes multiple bones, and step 150 includes: Based on the distance of each model vertex of the target virtual model relative to each bone, determine the first weight of each bone relative to each model vertex; The skinning weights of each model vertex are determined based on the first weight of each bone relative to each model vertex. The skinning result is obtained based on the skinning weight of each model vertex.

[0162] Specifically, in the skinning process of virtual models, the allocation of skinning weights mostly relies on manual drawing. However, this manual operation is inefficient, and the rationality of the weight allocation depends on the operator's experience level. The consistency of skinning effects cannot be guaranteed when batch processing virtual models. Furthermore, some automatic skinning solutions lack standardized weight calculation logic and do not consider the spatial distance relationship between model vertices and bones when allocating weights. This results in weight allocation that does not conform to the physical deformation laws of the virtual model, easily leading to problems such as limb distortion, deformation, and stiffness during animation. Based on the above problems, in some embodiments provided in this application, the spatial distance between the model vertices of the target virtual model and each bone in the skeleton instance is used as the basis. First, the first weight corresponding to the model vertex of each bone is calculated, and then the final skinning weight is determined by the first weight, completing the standardized skinning process and obtaining a natural and reasonable skinning result.

[0163] In some embodiments, model vertices can be understood as the basic unit points that constitute the three-dimensional mesh of the target virtual model, and are the smallest unit for skin weight allocation.

[0164] In some embodiments, spatial distance can be understood as the shortest straight-line distance between model vertices and bones, and can be used to calculate the first weight.

[0165] In some embodiments, the first weight can be understood as an initial influence weight assigned based on the spatial distance between the vertex and the skeleton, which can be used to initially characterize the degree of influence of the skeleton on the model vertex.

[0166] In some embodiments, skin weights can be understood as weight values ​​that, after standardization, ultimately characterize the degree to which model vertices are affected by each bone.

[0167] In some embodiments, the skinning result can be understood as virtual model binding data including the skinning weights of all model vertices, which can be directly used to drive virtual model animation deformation.

[0168] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When performing skinning operations, a skeleton instance consists of multiple bones connected according to a preset relationship. First, each vertex of the target virtual model is traversed. For each individual vertex, the spatial distance between that vertex and each bone in the skeleton instance is calculated. This spatial distance is the shortest straight-line distance from the model vertex to the bone. .

[0169] Subsequently, the first weight is assigned based on the spatial distance. The closer the spatial distance between a model vertex and a certain bone, the larger the first weight value of that bone, which means that the bone has a greater influence on the deformation of the model vertex. The farther the spatial distance, the smaller the first weight value.

[0170] For example, the first weight is calculated using a Gaussian decay function:

[0171] in, These are parameters based on engineering experience or given by a configuration file.

[0172] After calculating the first weights of all bones corresponding to a single vertex, the first weights are integrated and standardized to convert the initial first weights into skin weights that conform to the animation engine's usage specifications, ensuring that the sum of all skin weight values ​​corresponding to a single model vertex remains stable.

[0173] Finally, following the above process, all model vertices of the target virtual model are processed sequentially, and the skinning weights of all model vertices are summarized and integrated to form a complete skinning data system, ultimately generating the skinning processing result of the target virtual model.

[0174] Thus, in this embodiment, the skeleton instance is first defined as including multiple bones. Based on the distance of each model vertex of the target virtual model relative to each bone, the first weight of each bone relative to each model vertex is determined. Based on the first weight of each bone relative to each model vertex, the skinning weight of each model vertex is determined. Based on the skinning weight of each model vertex, the skinning processing result of the target virtual model is obtained. This realizes automated skinning processing of virtual models based on spatial distance calculation. Compared with the solution of technical artists manually drawing and adjusting skinning weights, the standardized logic calculation of skinning weights based on spatial distance makes the weight allocation conform to the physical deformation law of the virtual model. To a certain extent, it ensures the natural and smooth deformation of the limbs of the virtual model when driven by animation, avoiding problems such as distortion and stiffness of the virtual model. Moreover, the unified calculation rules effectively ensure the consistency of the skinning effect of batch virtual models, which to a certain extent meets the needs of large-scale industrial production of virtual model assets. The generated skinning processing result can be directly connected to the animation engine, thereby improving the usability and reusability of virtual model assets to a certain extent.

[0175] In some embodiments provided in this application, the skeleton example includes multiple bones and multiple joints, and step 150 includes: Identify each bone or joint as a heat source; The second weight of the heat source is obtained by weight diffusion based on the grid geodesic distance or the graph Laplacian operator corresponding to the heat source. The skinning process results are obtained based on the second weight of each heat source.

[0176] Specifically, in the skinning process of virtual models, some solutions allocate weights based on the linear spatial distance between model vertices and bones, without considering the model's own mesh topology. This leads to drastic changes in weight distribution and insufficient smoothness of the skinning effect, resulting in stiffness and distortion of limbs in animated virtual models. Other solutions rely on manual adjustment of weight smoothness, which is costly and cannot guarantee consistency during batch processing. Furthermore, some solutions that automatically calculate skinning weights lack a weight diffusion mechanism that conforms to the model surface. In areas with complex topologies or in the bending and wrinkling areas of stylized virtual models, the weight allocation is unreasonable, easily leading to clipping, deformation, and distortion, failing to meet the requirements of high-quality animation. Based on these problems, in some embodiments provided in this application, bones or joints in the skeleton instance are set as heat sources. The skinning weights are smoothly diffused along the model mesh surface using the geodesic distance or the graph Laplacian operator, obtaining a standardized second weight, ultimately generating a smooth and natural skinning result.

[0177] In some embodiments, a joint is a node in a skeleton instance that connects adjacent bones and can be used to enable rotational and bending movements of the bones.

[0178] In some embodiments, the heat source is a bone or joint node set as the starting point for skin weight propagation, which can be used to initiate a smooth propagation process of skin weight.

[0179] In some embodiments, the geodesic distance is the shortest path distance from the model vertex to the heat source calculated along the grid surface of the target virtual model, and the skin weight exhibits a monotonically decreasing distribution as the geodesic distance increases.

[0180] In some embodiments, the graph Laplacian operator is a mathematical computational tool built upon the mesh adjacency graph structure of the target virtual model to achieve smooth diffusion of skin weights.

[0181] In some embodiments, weight diffusion is a process in which skin weights are smoothly and continuously transferred from the heat source along the mesh structure of the target virtual model. The diffusion process follows the mesh topology constraints and has no cross-regional abrupt changes.

[0182] In some embodiments, the second weight is a weight value that characterizes the degree of influence of bones or joints on the smoothing of model vertices, obtained after weight diffusion calculation.

[0183] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: During the skinning process, a skeleton instance includes multiple interconnected bones and multiple joints between the bones. First, each bone or joint in the skeleton instance is set as a heat source for skinning weight propagation. The heat source serves as the starting node for skinning weight propagation, representing the starting position where the corresponding bone or joint affects the surrounding model vertices.

[0184] Then, select one of the following two skin weight diffusion methods to perform skin weight calculation: The first calculation method uses the geodesic distance of the grid to perform skinning weight diffusion. The geodesic distance is the shortest path distance from a vertex to the heat source calculated along the grid surface of the target virtual model. The skinning weight is diffused smoothly in a decaying manner from near to far according to this geodesic distance. For example, in the elbow flexion area of ​​a realistic game character, the weight gradually decreases as the distance from the grid surface to the joint heat source increases.

[0185] The second calculation method uses the graph Laplacian operator to carry out skin weight diffusion. The graph Laplacian operator is constructed based on the mesh adjacency graph structure of the target virtual model, and achieves smooth and continuous propagation of skin weight between mesh vertices through mathematical operations.

[0186] Both of the above diffusion methods allow the skin weights to be naturally transmitted from the heat source along the model's own mesh structure, thus avoiding cross-regional abrupt changes in skin weights to a certain extent.

[0187] After the skin weight diffusion is completed, the second weight corresponding to each heat source is obtained. The second weight can characterize the degree of smoothing influence of each bone or joint on each vertex of the model.

[0188] Finally, the second weights of all heat sources are summarized and integrated to form a complete and smooth skin weight distribution. Based on this skin weight distribution, the skin processing result of the target virtual model is obtained.

[0189] Thus, in this embodiment, the skeleton instance is first defined as including multiple bones and multiple joints. Each joint in the skeleton instance is defined as a heat source. Based on the geodesic distance of the grid or the graph Laplacian operator, the weights are smoothly diffused from the heat source along the grid surface to the surrounding vertices to obtain the second weight of each heat source. Based on the second weight of each heat source, the skinning result of the target virtual model is obtained. This realizes the automated skinning of the virtual model based on heat source diffusion and grid topology calculation. Compared with the solution of technical artists manually drawing and adjusting the skinning weights, the weight diffusion method with heat source setting and topology awareness effectively solves the problem of weight jump caused by straight distance calculation. To a certain extent, the skinning weights can conform to the grid surface distribution of the target virtual model, ensuring the continuous and smooth distribution of skinning weights. This, in turn, avoids problems such as virtual model clipping and deformation to a certain extent, and improves the quality of skinning and animation display effect.

[0190] In some embodiments provided in this application, step 150 includes: Based on the semantic attribution information of each model vertex of the target virtual model, determine the relevant vertex set corresponding to each bone. The relevant vertex set is the set of vertices in the mesh vertices of the target virtual model that are affected by the corresponding bone and do not overlap with each other. Based on the relevant vertex sets, topological isolation constraints are set on the mesh of the target virtual model, whereby the topological isolation constraints are used to prevent the spread of vertex weights between the relevant vertex sets of different bones; Based on the topological isolation constraints and the relevant vertex set, the skinning weight of each model vertex is determined by solving the harmonic field, and the skinning processing result is obtained.

[0191] Specifically, in the skinning process of virtual models, some solutions allocate weights based on the linear spatial distance between model vertices and bones, without considering the semantic attribution information of model vertices to divide control areas. The influencing vertices corresponding to different bones overlap, causing skinning weights to spread across regions during calculation. This leads to problems such as deformation and distortion of the virtual model at semantic boundaries. Furthermore, the lack of an effective weight diffusion isolation mechanism fails to guarantee the independence of weights in different bone regions, resulting in low skinning accuracy and failing to meet the high-precision deformation requirements of virtual models in fields such as games and animation. Based on these problems, some embodiments provided in this application determine a set of exclusive and non-overlapping related vertices for each bone based on the semantic attribution information of model vertices. Topological isolation constraints are set to prevent weight diffusion across regions, and a smooth skinning weight is obtained by combining harmonic field solutions, ultimately generating a high-quality skinning result.

[0192] In some embodiments, semantic attribution information can be understood as semantic classification information used to characterize the body part to which each model vertex of the target virtual model belongs and the corresponding controlled skeleton.

[0193] In some embodiments, the relevant vertex set can be understood as a set of vertices consisting of the vertices of the target virtual model mesh, which is affected by a single bone and does not overlap with the vertex sets of other bones.

[0194] In some embodiments, topological isolation constraints can be understood as mesh structure constraints set on the target virtual model mesh to prevent skinning weights from spreading between relevant vertex sets of different bones.

[0195] In some embodiments, the harmonic field can be understood as a mathematical field used to solve for the smooth skin weights.

[0196] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: During the skinning process, the semantic attribution information of each vertex of the target virtual model is first extracted. This semantic attribution information clarifies the body part and control skeleton corresponding to each vertex.

[0197] Subsequently, based on the semantic attribution information of the model vertices, a corresponding set of related vertices is divided for each bone in the skeleton instance. All vertices in the set of related vertices are only affected by the current corresponding bone, and there are no overlapping model vertices between the sets of related vertices of different bones.

[0198] After completing the division of the relevant vertex set, a topological isolation constraint is set on the mesh structure of the target virtual model based on the relevant vertex set. This topological isolation constraint can block the diffusion of skin weights between relevant vertex sets of different bones, thus avoiding cross-diffusion interference of skin weights at the mesh structure level.

[0199] Subsequently, using topological isolation constraints and related vertex sets as the basis for calculation, the skinning weight of each vertex of the target virtual model is calculated by solving the harmonic field mathematically. The harmonic field solution process ensures that the weights are smoothly distributed within the constraint range.

[0200] Finally, the skinning weights of all model vertices are integrated to form a complete skinning data system, resulting in the skinning processing results of the target virtual model.

[0201] Thus, in this embodiment, based on the semantic attribution information of each model vertex of the target virtual model, a set of related vertices corresponding to each bone is determined. This set of related vertices is the set of vertices in the mesh of the target virtual model that are influenced by the corresponding bone and do not overlap with each other. Based on the set of related vertices, topological isolation constraints are set on the mesh of the target virtual model. These constraints prevent the spread of vertex weights between the related vertices of different bones. Finally, based on the topological isolation constraints and the set of related vertices, the skinning weights of each model vertex are determined by solving the harmonic field, resulting in the skinning processing result of the target virtual model. This enables automated skinning of virtual models by combining semantic attribution information and topological isolation constraints. Compared to the approach where technical artists manually draw and adjust skinning weights without isolating and restricting weight diffusion, this approach effectively solves the problem of cross-regional diffusion of skinning weights by dividing relevant vertex sets through semantic attribution information and combining topological isolation constraints. To a certain extent, this makes the deformation of the virtual model at semantic boundaries natural and smooth, without errors, distortions, or other anomalies. Furthermore, the harmonic field solution method effectively ensures the smoothness and accuracy of skinning weight distribution, thereby improving the precision and quality of skinning processing and meeting the needs of high-precision animation-driven systems.

[0202] In some embodiments provided in this application, the step of determining the set of related vertices corresponding to each bone based on the semantic attribution information of each model vertex of the target virtual model includes: Semantic segmentation of body parts is performed on the target image to obtain a semantic segmentation mask; Based on the semantic segmentation mask, the three-dimensional vertices corresponding to the semantic segmentation mask are read from the cache information generated when rendering the target image to obtain the initial set of related vertices; A morphological erosion operation is performed on the initial set of related vertices to shrink the region boundary, and mutual exclusion is performed on the eroded sets of related vertices corresponding to different bones so that the sets of related vertices of different bones do not contain the same model vertices, thus obtaining the set of related vertices.

[0203] Specifically, when determining the set of relevant vertices corresponding to bones, relying solely on simple semantic information segmentation cannot achieve accurate region delineation. The initially obtained vertex set has blurred boundaries and contains a large number of redundant vertices, leading to the failure of topological isolation constraints, cross-diffusion of skinning weights across different bone regions, and deformation, misalignment, and distortion of the virtual model at body connection points. Based on these problems, in some embodiments provided in this application, a semantic segmentation mask is obtained by performing semantic segmentation on the target image to obtain a set of initial relevant vertices. Morphological erosion operations are then used to optimize the boundaries, and duplicate vertices are removed through mutual exclusion processing, ultimately resulting in a set of relevant vertices with clear boundaries and no overlap.

[0204] In some embodiments, semantic segmentation of body parts can be understood as an image processing process that semantically divides body regions in a target image.

[0205] In some embodiments, a semantic segmentation mask can be understood as image data generated after semantic segmentation processing, used to characterize the range of different body parts.

[0206] In some embodiments, the cache information can be understood as an identifier buffer (IDBuffer) or a 3D position buffer generated synchronously when rendering the target image, used to record the storage data of the correspondence between pixels and 3D vertices. In some embodiments, the identifier buffer stores the mesh vertex index corresponding to each pixel, and the 3D position buffer stores the world coordinates of the 3D surface point corresponding to each pixel.

[0207] In some embodiments, the initial set of related vertices can be understood as the original set of vertices obtained by mapping from cached information based on a semantic segmentation mask.

[0208] In some embodiments, the morphological erosion operation can be understood as an image processing operation that is executed iteratively multiple times to shrink the boundary of the vertex set region inward and remove redundant vertices from the boundary of the set region.

[0209] In some embodiments, mutual exclusion processing can be understood as a process of comparing different vertex sets and removing duplicate model vertices.

[0210] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When determining the relevant vertex set, the target image is first subjected to semantic segmentation of body parts. Semantic segmentation algorithms are used to identify the region range of different body parts in the target image and generate a semantic segmentation mask.

[0211] Subsequently, using the semantic segmentation mask as a reference, the three-dimensional vertices corresponding to the mask region are extracted from the identifier buffer or three-dimensional position cache generated synchronously when rendering the target image, forming an initial set of related vertices.

[0212] Next, a morphological erosion operation is performed on the initial set of related vertices through multiple iterations. The erosion process causes the region boundaries of the vertex set to shrink uniformly inward, effectively eliminating redundant and blurry vertices at the region boundaries and optimizing the regularity and accuracy of the set region boundaries. For example, semantic boundary regions such as the shoulders and fingers of realistic-style game characters.

[0213] After the erosion operation is completed, mutual exclusion processing is performed on the relevant vertex sets corresponding to different bones. The model vertices in all relevant vertex sets are compared one by one, and the model vertices that appear repeatedly between different relevant vertex sets are found and removed to ensure that there are no identical model vertices in the relevant vertex sets of any two bones. Finally, the relevant vertex sets that meet the requirements are obtained.

[0214] Thus, in this embodiment, semantic segmentation of the target image is performed on the body parts to obtain a semantic segmentation mask. Based on the semantic segmentation mask, the 3D vertices corresponding to the semantic segmentation mask are read from the cache information generated when rendering the target image to obtain an initial set of related vertices. Morphological erosion is then performed on the initial set of related vertices to shrink the region boundaries. Mutual exclusion processing is then performed on the eroded set of related vertices corresponding to different bones to ensure that the set of related vertices of different bones does not contain the same model vertices. This results in the set of related vertices corresponding to each bone. This achieves automated and accurate determination of the set of related vertices based on semantic segmentation and mesh topology processing. Compared with the scheme that relies on manual experience to divide the vertex regions corresponding to bones without boundary optimization and duplicate vertex removal, the semantic segmentation mask achieves accurate region mapping, which improves the initial accuracy of the set of related vertices to a certain extent. Furthermore, the morphological erosion operation effectively optimizes the boundary of the vertex set and removes redundant and blurry vertices, making the boundary of the set more consistent with the actual control range of the bones.

[0215] Furthermore, mutual exclusion effectively solves the problem of vertex overlap between different sets, ensuring to a certain extent that each model vertex belongs to only one set of related vertices of a bone. This provides a stable and reliable foundation for subsequent topological isolation constraints, avoids the problem of skin weight cross-diffusion caused by vertex overlap, improves the accuracy of skin weight calculation, and makes the deformation effect of the virtual model more natural and smooth when driven by animation, effectively improving the quality of virtual model processing.

[0216] In some embodiments provided in this application, the step of setting topological isolation constraints on the mesh of the target virtual model based on the relevant vertex set includes: On the mesh adjacency graph of the target virtual model, the mesh edges between the relevant vertex sets corresponding to different bones are marked as blocking edges. The blocking edges are used to prevent the spread of vertex weights between the relevant vertex sets of different bones. Construct a graph Laplacian operator by blocking edges.

[0217] Specifically, when setting topological isolation constraints, the diffusion range of skin weights is limited at the logical level. However, this lacks a concrete implementation method that fits the actual mesh structure of the virtual model, making it impossible to block the cross-region diffusion of skin weights from the mesh topology. The actual effect of the isolation constraints is poor, and there is a lack of mathematical operator construction methods that match the topological isolation constraints. Conventional graph Laplacian operators do not consider isolation restrictions, and the mathematical logic of skin weight calculation does not match the isolation requirements, resulting in poor stability and low accuracy of the skin weight solution. This makes the virtual model prone to limb deformation and misalignment during animation. Based on the above problems, in some embodiments provided in this application, blocking edges are marked on the mesh adjacency graph of the target virtual model. A graph Laplacian operator is constructed through these blocking edges, grounding the topological isolation constraints into a dual constraint of mesh structure and mathematical operators, effectively blocking the cross-region diffusion of skin weights.

[0218] In some embodiments, a mesh adjacency graph can be understood as a topological diagram used to describe the connection relationships between mesh vertices and the distribution state of mesh edges in a target virtual model.

[0219] In some embodiments, barrier edges can be understood as mesh connection edges marked on the mesh adjacency graph, located between mutually exclusive sets of vertices of different bones, used to cut off the cross-region diffusion path of vertex weights.

[0220] In some embodiments, the graph Laplacian operator can be understood as a mathematical computational operator based on blocking edges, used for solving skinned weighted harmonic fields, which excludes the contribution of blocking edges during construction and preserves the positive semidefiniteness and row sum properties of the matrix.

[0221] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When setting topological isolation constraints, the mesh adjacency graph of the target virtual model is retrieved first. This mesh adjacency graph records the connection relationships of all mesh vertices and the distribution of mesh edges in the model.

[0222] Subsequently, in the mesh adjacency graph, the mesh edges connecting the relevant vertex sets corresponding to different bones are identified, and these interconnected mesh edges are uniformly marked as blocking edges. Blocking edges cut off the path of vertex weight diffusion across regions at the mesh physical connection level. For example, cross-bone connection edges at semantic boundaries such as the neck, fingers, and shoulder seams of realistic-style game characters are all marked as blocking edges.

[0223] After marking the blocking edges, the graph Laplacian operator is reconstructed with the blocking edges as constraints. The mathematical contribution of the blocking edges is excluded during the construction of the graph Laplacian operator, so that the computational logic of the graph Laplacian operator meets the topological isolation constraint requirements, effectively preventing the cross-diffusion of skin weights and forming a stable topological isolation constraint.

[0224] Thus, in this embodiment, topological isolation constraints are set on the mesh of the target virtual model based on the relevant vertex sets. On the mesh adjacency graph of the target virtual model, the mesh edges between the relevant vertex sets corresponding to different bones are marked as blocking edges. The blocking edges can be used to prevent the spread of vertex weights between the relevant vertex sets of different bones. A graph Laplacian operator is constructed through the blocking edges, thereby realizing the automated setting of topological isolation constraints based on the mesh adjacency graph and the blocking edge markings. Compared with the scheme of manually dividing the weight spread region without combining the constraints to construct the graph Laplacian operator, the topological isolation constraints are transformed into a concrete mesh structure restriction by marking the blocking edges on the mesh adjacency graph. The spread of vertex weights between the relevant vertex sets of different bones is blocked from the physical connection level, effectively avoiding the cross-penetration interference of weights. Moreover, the graph Laplacian operator constructed based on the blocking edges provides a mathematical basis for the subsequent skinning weight solution to fit the constraints, which improves the stability and accuracy of weight calculation to a certain extent. This makes the limb deformation of the virtual model natural and smooth during the animation-driven process, without distortion or other abnormalities, and can adapt to virtual models with various topological structures, thereby improving the versatility and reliability of the virtual model processing method to a certain extent.

[0225] In some embodiments provided in this application, the step of determining the skinning weights of each model vertex by solving the harmonic field based on topological isolation constraints and the relevant vertex set includes: Using the relevant vertex set as the Dirichlet boundary condition, and based on the constructed graph Laplacian operator, the multi-source harmonic field is solved simultaneously for the remaining vertices in the mesh of the target virtual model other than the relevant vertex set, to obtain the skinning weight of each model vertex.

[0226] Specifically, in the calculation of skinning weights for virtual models, there is a lack of standardized solution methods for model vertices in transition regions. Conventional calculation methods do not incorporate topological isolation constraints and fixed boundary conditions, leading to abrupt changes in skinning weight distribution and abrupt, unnatural weight transitions between different skeleton control regions. Based on these issues, in some embodiments provided in this application, the relevant vertex set is used as the Dirichlet boundary condition. Relying on the pre-constructed graph Laplacian operator with topological isolation constraints, the multi-source harmonic field is solved simultaneously for the remaining vertices in the target virtual model that do not belong to the relevant vertex set, ultimately obtaining continuous, smooth, and constraint-compliant skinning weights.

[0227] In some embodiments, the Dirichlet boundary condition can be understood as a boundary constraint used to fix the skin weight values ​​of the relevant vertex set in order to ensure the stability of the weights in the core region. For example, the Dirichlet boundary condition is to fix the skin weight of the corresponding bone of all model vertices within the relevant vertex set to 1, and the weight of the remaining bones to 0.

[0228] In some embodiments, the multi-source harmonic field can be understood as a mathematical field that simultaneously considers the influence of multiple bone heat sources and solves for the smooth skin weights under topological isolation constraints.

[0229] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: When determining the skin weights, the relevant vertex set that has already been divided is first set as the Dirichlet boundary condition. This Dirichlet boundary condition can fix the skin weight values ​​of all model vertices within the relevant vertex set, so that the weights of the core region remain stable and do not change.

[0230] Subsequently, the graph Laplacian operator constructed using blocking edges is invoked. This graph Laplacian operator incorporates topological isolation constraints, ensuring that the weights will not spread to other skeletal regions across the blocking edges during the computation process.

[0231] Simultaneously, from all the mesh vertices of the target virtual model, the remaining vertices that do not belong to the relevant vertex set are selected; these are the transition vertices between different bone control regions. For example, semantic boundary regions such as the shoulders, finger gaps, and shoulder seams of realistic-style game characters.

[0232] Next, based on the fixed Dirichlet boundary conditions, and with the help of the graph Laplace operator adapted to topological isolation constraints, mathematical operations are performed on all remaining vertices to solve for the multi-source harmonic field.

[0233] The above solution process follows the topological isolation constraint, and the weights can only spread smoothly within the control area of ​​a single bone. They will not break through the restriction of the blocking edge and spread to other areas. In addition, the multi-source harmonic field will consider the influence of multiple bone heat sources at the same time, and calculate the skin weight of each remaining vertex according to the principle of minimizing energy, which to a certain extent ensures the continuous smoothness of the weight distribution.

[0234] Finally, the fixed weights of the relevant vertex set are integrated with the solution weights of the remaining vertices to obtain the skinning weights of all vertices of the target virtual model.

[0235] Thus, in this embodiment, the skin weight of each model vertex is determined by solving the harmonic field based on topological isolation constraints and the relevant vertex set. The relevant vertex set is used as the Dirichlet boundary condition. Based on the constructed Graph Laplacian operator, the multi-source harmonic field is solved simultaneously for the remaining vertices in the mesh of the target virtual model, excluding the relevant vertex set, thereby obtaining the skin weight of each model vertex. This achieves automated solution of multi-source harmonic field skin weights by combining Dirichlet boundary conditions and Graph Laplacian operator. Compared to the solution where technical artists manually adjust the vertex weights in the transition region without combining boundary conditions and topological constraints, using the relevant vertex set as the Dirichlet boundary condition effectively fixes the skin weights of the skeletal core control region, ensuring the stability and accuracy of the semantic core region weights to a certain extent. Furthermore, by relying on the Graph Laplacian operator with topological isolation constraints to solve the multi-source harmonic field, a continuous and smooth distribution of skin weights in the transition region is achieved to a certain extent, effectively solving the problem of sudden weight jumps.

[0236] Furthermore, strict topological constraints effectively limit the cross-regional diffusion of weights, ensuring that the deformation of limb joints in the virtual model is natural and smooth during animation-driven processes, without any abnormalities such as distortion or misalignment. The method of solving multi-source harmonic fields simultaneously takes into account the influence of multiple bone heat sources, effectively improving the accuracy and rationality of skinning weights. Moreover, the standardized solution process effectively ensures the consistency of skinning effects when processing batches of virtual models, thereby meeting the production requirements of high-precision deformation in game animations to a certain extent.

[0237] In some embodiments provided in this application, step 150 further includes: After obtaining the skinning results, non-negative truncation is performed on the skinning weights; Select the K bones with the highest weights from the weights corresponding to each bone as the influencing bones; The weights of the selected bones are normalized. To ensure that the processed skin weights meet preset constraints, where K is a preset positive integer.

[0238] Specifically, during the skinning process, there are instances where the initial skinning weights are negative. Such negative weights can cause abnormal deformations in the virtual model during animation, failing to meet basic graphics computation requirements. Furthermore, individual model vertices are affected by an excessive number of bones, increasing the computational load on the game engine and reducing animation smoothness. Additionally, the total value of the initial skinning weights is not fixed, and the weight distribution lacks a unified standard, easily leading to issues like limb stretching and distortion when directly imported into the game engine. Moreover, the initial skinning weights are not standardized and cannot match the game engine's preset constraints, requiring manual secondary correction, which reduces the efficiency of virtual model processing and makes it difficult to ensure consistency in batch processing. To address these issues, in some embodiments provided in this application, after obtaining the initial results from the skinning process, non-negative truncation, front-K bone selection, and normalization are performed on the skinning weights to correct weight anomalies, ensuring that the final skinning weights fully conform to the game engine's preset constraints.

[0239] In some embodiments, nonnegative truncation can be understood as correcting the negative skinning weight values ​​of model vertices to zero, thus eliminating abnormal weight values.

[0240] In some embodiments, the first K bones can be understood as the bones selected from the top K positions for each model vertex after sorting them by weight value from high to low.

[0241] In some embodiments, influencing bones can be understood as having an actual effect on the deformation of model vertices, and the bones that are retained after the first K-bone screening and meet the engine's number constraints.

[0242] In some embodiments, normalization can be understood as adjusting the weight values ​​of the selected bones so that the sum of the weights of all bones affecting a single model vertex reaches a fixed standard. For example, according to the formula... This operation adjusts the weight values ​​of the selected bone so that the sum of the weights of all bones affecting a single model vertex reaches a fixed value of 1.

[0243] In some embodiments, preset constraints can be understood as the specifications set by the game engine for the range of skin weight values, the number of affected bones, and the total weight.

[0244] In some embodiments, K can be understood as a pre-defined positive integer used to limit the maximum number of influential bones that a single model vertex can correspond to. For example, K can be 4.

[0245] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: After obtaining the initial skinning result, a non-negative truncation operation is performed on the skinning weights of each vertex of the target virtual model. All skinning weight values ​​of each vertex are checked one by one, and all negative weight values ​​less than zero are directly corrected to zero, effectively eliminating the calculation anomalies caused by negative weights.

[0246] In addition, according to a pre-set positive integer K, the top K bones with the highest weight values ​​can be selected for each model vertex. These top K bones are then identified as the influential bones of the model vertex. At the same time, invalid bones with too low weight values ​​are removed, effectively reducing the computational burden caused by invalid bones.

[0247] In addition, the weights of the first K affected bones selected for each model vertex can be normalized. By adjusting the values, the sum of the weights of all affected bones corresponding to a single model vertex can be kept at a fixed value, so that the weight values ​​conform to the calculation specifications of the game engine.

[0248] Understandably, after non-negative truncation, K-selection, and normalization, the final skinning weights satisfy the preset constraints and can be directly imported into the game engine.

[0249] Thus, in this embodiment, after obtaining the skinning result of the target virtual model, a non-negative truncation process is performed on the skinning weight of each model vertex. The top K bones with the highest weights for each model vertex are selected as influencing bones, and the weights of the selected bones are normalized to ensure that the processed skinning weights meet preset constraints. This achieves standardized and automated correction of skinning weights. Compared to the approach of technical artists manually checking for negative weights, manually selecting influencing bones, and manually adjusting weight values, the non-negative truncation operation effectively eliminates negative weight anomalies, avoiding deformation problems during virtual model animation to a certain extent. Furthermore, the selection of the top K bones effectively controls the number of influencing bones for a single vertex, reducing the computational pressure on the game engine and improving the smoothness of animation. In addition, the normalization process standardizes the weight values, ensuring that the weight distribution conforms to the engine's computational logic to a certain extent, avoiding problems such as limb stretching and distortion. This, in turn, improves the efficiency of virtual model skinning processing, ensures the consistency of skinning results during batch processing, and allows the processed skinning weights to be directly adapted to various game engines, improving the usability and reusability of virtual model assets.

[0250] In some embodiments provided in this application, the method further includes: After generating skeleton instances and / or obtaining skinning results, the skeletal geometric parameters of the skeleton instances and / or the model deformation parameters corresponding to the skinning results are verified. When the verification result exceeds the preset threshold, a check mark is output and the check mark is associated with the skinning processing result.

[0251] Specifically, the processing flow of related virtual models lacks a quality verification step. After the skeleton instance is generated, problems such as abnormal bone length, asymmetry between left and right bones, and incorrect bone topology connections may occur. Furthermore, after skinning, deformation defects such as excessive stretching of model vertices, overlapping of model faces, and abnormal changes in model volume may occur. These defects cannot be identified in time during the processing stage, leading to animation distortion, deformation, clipping, and other runtime anomalies after the virtual model is imported into the game engine. To address these issues, some embodiments provided in this application add an automated verification step after skeleton instance generation and skinning. This step detects bone geometric parameters and model deformation parameters. When the verification result exceeds a preset threshold, a verification mark is generated, and the verification mark is correlated with the skinning result for output, thereby achieving quality control of virtual model processing.

[0252] In some embodiments, bone geometry parameters can be understood as geometric attribute parameters used to describe bone length, bone symmetry, and bone topological connectivity in a skeleton instance.

[0253] In some embodiments, model deformation parameters can be understood as parameters used to describe the deformation state of the virtual model after skinning, such as the vertex side length stretching ratio, the number of self-interpenetrations of the facets, the volume change rate, and the reprojection error.

[0254] In some embodiments, the preset threshold can be understood as a pre-set critical value for parameter qualification, combining virtual model creation specifications and game engine requirements. For example, the reference bone length in the skeleton specification template. Pre-set the acceptable threshold value for parameters as a benchmark, and define the bone length threshold range. The threshold for the difference in bone length between the left and right sides is any value between 10% and 15%, and the reprojection error threshold is any pixel in the range of 3 to 8.

[0255] In some embodiments, the "to be reviewed" marker can be understood as a marker information used to indicate that the virtual model processing result is abnormal and requires manual review.

[0256] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: After completing the skeleton instance generation operation and skinning process to obtain the final skinning result, the bone geometric parameters and model deformation parameters are verified. The bone geometric parameters include the actual length of the bones, the degree of left-right symmetry of the bones, and the topological connection relationship between the bones. The model deformation parameters include the side length stretching ratio of the virtual model vertices, the self-interpenetration state of the model patches, the overall volume change rate of the model, and the reprojection error.

[0257] Next, the actual parameter values ​​obtained from each verification are compared with the bone length referenced by the skeleton specification template. The model is compared one by one with a pre-set threshold, which is a qualified critical value determined by combining virtual model production specifications and game engine operation requirements. For example, bone length must meet the following requirements. The difference in length between the left and right symmetrical bones is less than or equal to 15%, and the reprojection error is less than or equal to 5 pixels.

[0258] If the value of any verification result exceeds the preset threshold, the processing result of the virtual model is determined to have quality abnormalities.

[0259] A check mark is then generated, indicating that the processing result is abnormal and needs to be manually checked. At the same time, the check mark is bound to the skin processing result, so that the check mark is output synchronously with the skin processing result, so that technicians can identify abnormal models and carry out targeted review and correction work.

[0260] Thus, in this embodiment, after generating the skeleton instance and obtaining the skinning result, the skeletal geometric parameters of the skeleton instance and the model deformation parameters corresponding to the skinning result are verified. When the verification result exceeds a preset threshold, a verification mark is output, and the verification mark is associated with the skinning result. This achieves automated quality verification and anomaly identification in the skeleton generation and skinning process of virtual model processing. Compared with the solution of technical artists manually checking the skeleton parameters and skinning effect item by item and then manually marking abnormal problems, the automated verification realizes the quality control of virtual model processing. It can identify various defects in skeleton instances and skinning results in a timely manner, and to a certain extent avoids abnormal virtual model assets from flowing into the subsequent use process. Moreover, the verification mark can locate abnormal models, replacing inefficient manual sampling inspection, and reducing the cost of quality inspection and the problem omission rate to a certain extent.

[0261] Furthermore, the batch automated verification method can be adapted to the industrial production scenarios of large-scale virtual models, which to a certain extent ensures the consistency and reliability of batch processing results, effectively reduces the repair cost of animation-driven anomalies in the later stage, and improves the delivery quality and production efficiency of virtual model assets.

[0262] In some embodiments provided in this application, the method further includes: Export the skeleton instance and skinning results as a 3D asset exchange format file.

[0263] Specifically, after skeleton generation and skinning are completed, the resulting skeleton instances and skinning results are mostly in tool-specific data formats, lacking a unified and standardized export method. This makes them incompatible with different game engine 3D production software and animation editing tools, requiring cumbersome format conversion before asset use. However, this conversion process is prone to data corruption issues such as incorrect bone hierarchy, missing bone names, and skinning weight offsets, increasing post-processing time and manpower costs. Furthermore, the non-standardized data format hinders the storage, transmission, and batch reuse of virtual model assets, failing to meet the rapid asset circulation needs of industrial game production and limiting the practical implementation and widespread application of virtual model processing solutions. Based on these issues, some embodiments provided in this application encapsulate and export the generated skeleton instances and skinning results into an industry-standard 3D asset exchange format file, achieving seamless cross-platform and cross-software compatibility for virtual model assets.

[0264] In some embodiments, the 3D asset exchange format can be understood as an industry-standardized 3D model asset storage format that supports cross-platform and cross-software reading, parsing, and use. For example, specific formats include FBX (Filmbox) and GLTF (GL Transmission Format).

[0265] In some embodiments, the 3D asset exchange format file can be understood as a virtual model asset carrier that encapsulates the structural data and skinning weight data of a skeleton instance, and can be used to realize the storage, transmission and cross-platform reuse of virtual model assets.

[0266] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please refer to the following exemplary description: After the skeleton instance is generated and the skinning process yields its final result, the structural data of the skeleton instance is extracted first, along with the skinning weight data obtained from the skinning process. The structural data of the skeleton instance includes geometric and logical information such as the spatial structure of the bones, bone names, hierarchical topology, and bone connection methods. Simultaneously, the skinning weight data includes animation-driven information such as the skinning weights corresponding to each model vertex and the binding relationships between vertices and bones.

[0267] Subsequently, the extracted structural data and skinning weight data of the skeleton instances are integrated, and data encoding and structural encapsulation are performed according to the unified industry standards of FBX and GLTF 3D asset exchange formats. This constructs a file data structure that conforms to general specifications, ultimately generating a complete 3D asset exchange format file. This 3D asset exchange format file requires no additional format adjustments or data conversions and can be directly imported into various mainstream game engine 3D modeling tools and animation production platforms. It retains all skeleton and skinning-related data, preventing information loss or data corruption. For example, it can be used to batch bind realistic game characters to scenes and export FBX format files, and automatically bind UGC (User Generated Content) character customization to scenes and export GLTF format files.

[0268] Thus, in this embodiment, after generating the skeleton instance and obtaining the skinning result, the skeleton instance and the skinning result are exported as a 3D asset exchange format file. This achieves standardized and automated export of virtual model processing results. Compared to the solution where technical artists manually adjust the data format and adapt it to different 3D software and game engines one by one, by unifying the output specifications of virtual model assets, the format compatibility problem between different platforms and software is effectively solved. To a certain extent, the cumbersome format conversion process is eliminated, the post-processing cost of virtual model assets is reduced, and the processing and delivery efficiency of virtual model assets can be improved to a certain extent. Moreover, the exported file retains the structural data of the skeleton instance and the skinning weight data of the skinning result, which to a certain extent avoids data loss, disorder and other problems, and ensures the integrity and usability of virtual model assets.

[0269] In addition, standardized file formats facilitate centralized storage, rapid transmission, and batch reuse of virtual model assets, which to some extent meets the asset circulation needs of large-scale production in the game industry, improves the practicality and versatility of virtual model processing solutions, and broadens the application scenarios and promotional value of virtual model processing solutions.

[0270] To more clearly illustrate the virtual model processing method provided in the embodiments of this application, please also refer to... Figure 2 and Figure 3 And the following exemplary description, Figure 2 This is a schematic diagram of the custom skeleton automatic binding process provided in the embodiments of this application. Figure 3 A schematic diagram of the cross-view consistency filtering and weighted analytical triangulation process provided in this application embodiment is as follows: In the process of processing the virtual model, the target image and image-related data of the target virtual model in the virtual scene are first acquired. The target image is generated by the virtual camera that observes the target virtual model in the virtual scene. The virtual camera can be set to multiple different observation positions to form multi-view target images, such as front, back, left, right, left front 45°, and right front 45° viewpoints.

[0271] Image-related data includes at least the camera geometric parameters of the virtual model of the target observed by the virtual camera. These camera geometric parameters specifically include camera intrinsic parameters, camera extrinsic parameters, or a camera projection matrix determined by geometric operations using the camera intrinsic and extrinsic parameters. Image-related data may also include auxiliary rendering channel data such as 3D position cache, depth data, visibility buffer, and identifier buffer. The 3D position cache stores the 3D position of each pixel in the target image in the virtual scene, and the depth data records the depth distance value of each pixel in the target image.

[0272] Subsequently, the target image is input into a pre-trained joint detection model. The model automatically detects the positions of the skeletal joints of the target virtual model in the target image and outputs the joint position detection results. In multi-view scenes, the position detection results include the predicted positions of the skeletal joints in the target image, i.e., the two-dimensional pixel coordinates. and the predicted probability of the predicted location This joint detection model is a general pre-trained visual detection model that does not require specific training for the skeleton specifications or character styles of the target project and can directly output the joint position and reliability metric.

[0273] Next, coordinate transformation processing is performed to convert the 2D position of the skeletal joints to the 3D target position. The coordinate transformation processing includes three processing methods and a multi-view weighted fusion mechanism: The first method is analytical triangulation, which relies on the analytical geometric projection constraints in three-dimensional space. Based on the position detection results and camera geometric parameters, a target linear equation related to the three-dimensional position of the skeletal joints is constructed. The target linear equation is then solved by singular value decomposition to obtain the three-dimensional target position of the skeletal joints.

[0274] The second method is the three-dimensional position cache lookup table method. Based on the position detection result, the three-dimensional position that matches the position detection result is directly read from the three-dimensional position cache to quickly obtain the three-dimensional target position of the bone joint.

[0275] The third method is depth backprojection. Based on the position detection results and depth data, combined with the camera's geometric parameters, the positions of the skeletal joints in the target image are backprojected into the virtual scene according to the backprojection formula, thus obtaining the 3D target positions of the skeletal joints. The backprojection formula is as follows:

[0276] in, For depth value, , As the camera's main point, , (This refers to the camera's focal length).

[0277] In multi-view scenarios, weighted fusion and data filtering processing are also required: first, based on a preset probability threshold... Filter the data; if the predicted probability of a certain skeletal joint is less than the preset probability threshold, delete the detection result for that location directly.

[0278] Subsequently, it is determined whether the pixel at the predicted position hits the surface of the foreground object, i.e., the target virtual model, in the target image. If it does not hit, the target weight of the corresponding skeletal joint is reset to zero or the detection result at that position is deleted.

[0279] Next, based on the predicted probability of each skeletal joint in each target image... The target weights are calculated by truncating a linear function or a softmax normalization function. ,in, Visibility coefficient, It can take any value between 0 and 1 or a continuous visibility value.

[0280] Then, the predicted positions of the skeletal joints in each target image are converted into predicted transformed positions in the virtual scene.

[0281] Finally, the predicted transformation positions are weighted according to the target weights, and the reprojection error is calculated simultaneously. If the reprojection error exceeds a preset threshold, outlier views are removed, and the three-dimensional target position of each skeletal joint is finally obtained.

[0282] Next, a skeleton instance is generated based on the three-dimensional target positions of the skeletal joints and the pre-determined skeleton configuration data. The skeleton configuration data indicates the bones included in the skeleton instance, the correspondence between skeletal joints and bones in the skeleton instance, the generation rules for generating corresponding bones based on the target positions of the skeletal joints, the name of each bone in the skeleton instance, and the geometric topological relationships of the bones in the skeleton instance.

[0283] Furthermore, the skeleton instance generation process can be executed according to semantic mapping, hierarchical organization, missing bone completion, and constraint verification. It can also perform interchange detection and correction on symmetrical joints to ensure that the generated skeleton instances fully comply with the project's custom skeleton specifications. For example, based on the central axis of the target virtual model or the symmetrical bone pairs defined in the skeleton configuration data, it can determine whether the 3D spatial positions of detected symmetrical joints are interchanged. When an interchange is detected, the target positions of the corresponding joints are swapped, and the confidence weight or target weight of that viewpoint observation in subsequent weighted fusion is reduced. Alternatively, the local coordinate system of each bone can be determined through orthogonalization based on the bone segment direction and reference vector. For the torso chain, the reference vector is taken as the world-up direction; for the limb chain, the reference vector is taken as the plane normal defined by the three points of the joint chain; for Twist-type bones, their rotation angle is spherically interpolated between parent and child bones according to preset weights to maintain the deformation continuity of adjacent bones, and the local coordinate system is output along with the skeleton instance.

[0284] Subsequently, skinning is performed, which includes three processing methods and incorporates topological constraints and normalization: The first processing method involves a skeleton instance with multiple bones. First, it calculates the shortest distance between each vertex of the target virtual model and each bone. Using the Gaussian decay formula Determine the first weight of each bone relative to each model vertex, then select the K bones with the highest weights based on the first weights, and apply a normalization formula. Determine the skinning weights for each model vertex and summarize the results to obtain the skinning process.

[0285] The second processing method involves a skeleton instance with multiple bones and joints. Each bone or joint is identified as a heat source. Weight diffusion is performed based on the geodesic distance of the grid corresponding to the heat source or the graph Laplacian operator to obtain the second weight of the heat source. The skinning processing result is obtained based on the second weight of all heat sources.

[0286] The third processing method first determines the relevant vertex set corresponding to each bone based on the semantic attribution information of each model vertex in the target virtual model. This is achieved by performing semantic segmentation on the body parts of the target image to obtain a semantic segmentation mask, reading the corresponding 3D vertices from the rendering cache to obtain an initial relevant vertex set, performing morphological erosion on the initial relevant vertex set to shrink the region boundaries, and then performing mutual exclusion processing on the vertex sets of different bones to obtain a non-overlapping relevant vertex set. Subsequently, based on the relevant vertex set, the mesh edges between relevant vertex sets of different bones are marked as blocking edges on the mesh adjacency graph of the target virtual model. A graph Laplacian operator is constructed using these blocking edges to complete the topological isolation constraint setting. Finally, using the relevant vertex set as the Dirichlet boundary condition, the multi-source harmonic field, i.e., the Laplacian smoothing target, is solved simultaneously based on the constructed graph Laplacian operator. Determine the skinning weights for each model vertex.

[0287] In addition, after all skinning schemes have been processed, non-negative truncation of the skinning weights is required, the top K bones with the highest weights are selected as the influencing bones, and the weights of the selected bones are normalized so that the processed skinning weights meet the preset constraints.

[0288] After generating skeleton instances and / or obtaining skinning results, the skeletal geometry parameters of the skeleton instances and / or the model deformation parameters corresponding to the skinning results are verified.

[0289] When the verification result exceeds the preset threshold, a check mark is output, and the check mark is associated with the skinning processing result and output. At the same time, the conservative completion rule is triggered to fall back, so as to realize the quality control of virtual model processing.

[0290] Finally, the generated skeleton instance and skinning results are exported as a 3D asset exchange format file. The file encapsulates the skeleton hierarchy, bone transformation matrix, and vertex skinning weights, which can be directly imported into various game engines, 3D modeling tools, and animation production platforms to achieve cross-platform storage, transmission, and batch reuse of virtual model assets.

[0291] To facilitate better implementation of the virtual model processing method of this application embodiment, this application embodiment also provides a virtual model processing apparatus. Please refer to... Figure 4 , Figure 4 This is a schematic diagram of the structure of a virtual model processing device provided in an embodiment of this application. The virtual model processing device 200 may include: The image data acquisition module 210 is used to acquire multiple target images of the target virtual model in the virtual scene and image-related data of each target image. Each target image is obtained by a virtual camera that observes the target virtual model in the virtual scene. The observation position of the virtual camera corresponding to each target image is different. The image-related data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. The joint detection module 220 is used to detect the position of the skeletal joints of the target virtual model in each target image and obtain the position detection results of the skeletal joints. The position conversion module 230 is used to convert the position of the skeletal joint in the target image into the position of the skeletal joint in the virtual scene based on the position detection result and image related data, so as to obtain the target position of the skeletal joint. The skeleton generation module 240 is used to generate a skeleton instance corresponding to the target virtual model based on the target location; The skinning module 250 is used to perform skinning processing on the target virtual model based on the skeleton instance, and obtain the skinning processing result of the target virtual model.

[0292] In some embodiments, the joint detection module 220 can also be used to input the target image into a pre-trained joint detection model, so that the joint detection model can detect the position of the skeletal joints of the target virtual model in the target image and obtain the position detection result.

[0293] In some embodiments, the position detection result includes the predicted position of the skeletal joint in each target image, and the camera geometric parameters include camera intrinsic and camera extrinsic parameters, or include a camera projection matrix determined by the camera intrinsic and camera extrinsic parameters. The position conversion module 230 can also be used to construct a linear equation related to the position of the skeletal joint in the virtual scene based on the predicted position of the skeletal joint in a target image and the camera geometric parameters of the target image, to obtain a target linear equation corresponding to the target image, to solve the target linear equation, to obtain candidate positions of the skeletal joint in the virtual scene corresponding to the target image, and to determine the target position based on the candidate positions corresponding to each target image.

[0294] In some embodiments, the position conversion module 230 can also be used to reproject the candidate positions of the skeletal joints in each target image to the corresponding target image, obtain the reprojected position of each target image, determine the reprojection error between the reprojected position of the target image and the predicted position in the position detection result of the target image, determine the weighting weight of the candidate positions of each target image based on the reprojection error of each target image, and perform weighted solving on all candidate positions of each skeletal joint based on the weighting weight of the candidate positions of each target image to obtain the target position.

[0295] In some embodiments, the position conversion module 230 can also be used to reduce the weight of the corresponding candidate position when the reprojection error is greater than the error threshold; or to delete the corresponding candidate position when the reprojection error is greater than the error threshold.

[0296] In some embodiments, the image-related data further includes a three-dimensional position cache, which includes the three-dimensional position of each pixel in the target image in the virtual scene. The position conversion module 230 can also be used to read the three-dimensional position that matches the position detection result from the three-dimensional position cache based on the position detection result to obtain the target position of the skeletal joint.

[0297] In some embodiments, the image-related data further includes depth data, which includes the depth of each pixel in the target image. The position conversion module 230 can also be used to back-project the position of the skeletal joint in the target image to the virtual scene based on the position detection result and the depth data to obtain the target position of the skeletal joint.

[0298] In some embodiments, the position detection result includes the predicted position of the skeletal joint in each target image and the predicted probability of the predicted position. The skeletal joint includes multiple joints. The position conversion module 230 can also be used to determine the target weight of each skeletal joint in each target image based on the predicted probability of each skeletal joint in each target image, convert the predicted position of the skeletal joint in each target image into the position of the skeletal joint in the virtual scene based on the predicted position of the skeletal joint in each target image and the image correlation data corresponding to each target image, obtain the predicted converted position of the skeletal joint in each target image, and perform weighted solving on the predicted converted position of each skeletal joint based on the target weight of each skeletal joint to obtain the target position of each skeletal joint.

[0299] In some embodiments, the position conversion module 230 can also be used to delete the position detection result if the predicted probability of the skeletal joint in the position detection result corresponding to the skeletal joint in the target image is less than a preset probability threshold.

[0300] In some embodiments, the position conversion module 230 may also be used to set the target weight of the skeletal joint corresponding to the predicted position in the target image to zero, or to delete the position detection result corresponding to the skeletal joint in the target image if the pixel of the predicted position in the target image does not hit the surface of the foreground object in the target image.

[0301] In some embodiments, the skeleton generation module 240 can also be used to generate a skeleton instance based on the target position of the bone joints and the pre-determined skeleton configuration data, wherein the skeleton configuration data is used to indicate the bones contained in the skeleton instance, the correspondence between the bone joints and the bones in the skeleton instance, the generation rules for generating the corresponding bones in the corresponding skeleton instance based on the target position of the bone joints, the name of each bone in the skeleton instance, and the geometric topological relationship of the bones in the skeleton instance.

[0302] In some embodiments, the skinning processing module 250 can also be used to determine the first weight of each bone relative to each model vertex based on the distance of each model vertex of the target virtual model relative to each bone, determine the skinning weight of each model vertex based on the first weight of each bone relative to each model vertex, and obtain the skinning processing result based on the skinning weight of each model vertex.

[0303] In some embodiments, the skeleton instance includes multiple bones and multiple joints. The skinning module 250 can also be used to identify each bone or each joint as a heat source, perform weight diffusion based on the grid geodesic distance or graph Laplacian operator corresponding to the heat source to obtain a second weight of the heat source, and obtain the skinning result based on the second weight of each heat source.

[0304] In some embodiments, the skinning processing module 250 can also be used to determine the relevant vertex set corresponding to each bone based on the semantic attribution information of each model vertex of the target virtual model, set topological isolation constraints on the mesh of the target virtual model based on the relevant vertex set, and determine the skinning weight of each model vertex by solving the harmonic field according to the topological isolation constraints and the relevant vertex set to obtain the skinning processing result. The relevant vertex set is the set of vertices in the mesh of the target virtual model that are determined to be affected by the corresponding bone and do not overlap with each other. The topological isolation constraints are used to prevent the vertex weights from spreading between the relevant vertex sets of different bones.

[0305] In some embodiments, the skinning processing module 250 can also be used to perform semantic segmentation of body parts on the target image to obtain a semantic segmentation mask. Based on the semantic segmentation mask, the module reads the three-dimensional vertices corresponding to the semantic segmentation mask from the cache information generated when rendering the target image to obtain an initial set of related vertices. The module then performs a morphological erosion operation on the initial set of related vertices to shrink the region boundaries and performs mutual exclusion processing on the eroded sets of related vertices corresponding to different bones so that the sets of related vertices of different bones do not contain the same model vertices, thus obtaining a set of related vertices.

[0306] In some embodiments, the skinning processing module 250 can also be used to mark the mesh edges between the relevant vertex sets corresponding to different bones as blocking edges on the mesh adjacency graph of the target virtual model, and construct a graph Laplacian operator through the blocking edges, wherein the blocking edges are used to prevent the vertex weights from spreading between the relevant vertex sets of different bones.

[0307] In some embodiments, the skinning processing module 250 can also be used to solve the multi-source harmonic field simultaneously for the remaining vertices in the mesh of the target virtual model other than the relevant vertex set, based on the constructed graph Laplacian operator, using the relevant vertex set as the Dirichlet boundary condition, to obtain the skinning weight of each model vertex.

[0308] In some embodiments, the skinning processing module 250 may also be used to perform non-negative truncation on the skinning weights of each model vertex after obtaining the skinning processing result; select the top K bones with the highest weights as the influencing bones; and normalize the weights of the selected bones so that the processed skinning weights meet preset constraints, where K is a preset positive integer.

[0309] In some embodiments, the skinning module 250 can also be used to verify the skeletal geometric parameters of the skeleton instance and / or the model deformation parameters corresponding to the skinning result after generating the skeleton instance and / or obtaining the skinning result, and when the verification result exceeds a preset threshold, output a verification mark and associate the verification mark with the skinning result.

[0310] In some embodiments, the skinning processing module 250 can also be used to export the skeleton instance and the skinning processing result as a 3D asset exchange format file.

[0311] Each unit in the aforementioned virtual model processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in an electronic device, or stored in the memory of an electronic device in software form, so that the processor can call and execute the operations corresponding to each unit.

[0312] The virtual model processing device 200 can be integrated into a terminal or server that has memory and a processor and thus computing power, or the virtual model processing device 200 can be the terminal or server.

[0313] Optionally, this application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0314] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be a terminal or a server. Figure 5 As shown, the electronic device 300 includes a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, and a computer program stored in the memory 302 and executable on the processor. The processor 301 and the memory 302 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figures does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0315] The processor 301 is the control center of the electronic device 300. It connects various parts of the electronic device 300 through various interfaces and lines. By running or loading software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, it executes various functions of the electronic device 300 and processes data, thereby performing overall processing of the electronic device 300.

[0316] In this embodiment, the processor 301 in the electronic device 300 loads the instructions corresponding to the processes of one or more computer programs into the memory 302 according to the following steps, and the processor 301 runs the computer programs stored in the memory 302 to realize various functions: Acquire multiple target images of the target virtual model in the virtual scene and image-related data for each target image. Each target image is obtained through a virtual camera that observes the target virtual model in the virtual scene. The observation position of the virtual camera corresponding to each target image is different. The image-related data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. The positions of the skeletal joints of the target virtual model are detected in each target image, and the position detection results of the skeletal joints are obtained. Based on the position detection results and image-related data, the position of the skeletal joint in the target image is converted into the position of the skeletal joint in the virtual scene, thus obtaining the target position of the skeletal joint. Based on the target location, generate a skeleton instance corresponding to the target virtual model; Based on the skeleton instance, skinning processing is performed on the target virtual model to obtain the skinning processing result of the target virtual model.

[0317] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0318] Optional, such as Figure 5 As shown, the electronic device 300 also includes: a display screen 303, a radio frequency circuit 304, an audio circuit 305, an input unit 306, and a power supply 307. The processor 301 is electrically connected to the display screen 303, the radio frequency circuit 304, the audio circuit 305, the input unit 306, and the power supply 307. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0319] The display screen 303 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The display screen 303 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 301, and can receive and execute commands from the processor 301. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 301 to determine the type of touch event. Subsequently, the processor 301 provides corresponding visual output on the display panel according to the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the display screen 303 to achieve input and output functions. However, in some embodiments, the touch panel and the display screen 303 can be implemented as two independent components to achieve input and output functions. That is, the display screen 303 can also be used as part of the input unit 306 to achieve input functions.

[0320] The radio frequency circuit 304 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices, and to transmit and receive signals with network devices or other electronic devices.

[0321] Audio circuitry 305 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuitry 305 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 305, converted back into audio data, and then processed by processor 301 before being transmitted via radio frequency circuitry 304 to, for example, another electronic device, or output to memory 302 for further processing. Audio circuitry 305 may also include an earphone jack to facilitate communication between peripheral headphones and electronic devices.

[0322] The input unit 306 can be used to receive input numbers, characters, or object feature information (such as fingerprints, irises, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0323] Power supply 307 is used to supply power to various components of electronic device 300. Optionally, power supply 307 can be logically connected to processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 307 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0324] although Figure 5 As not shown in the diagram, the electronic device 300 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0325] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the virtual model processing method of the embodiments of this application; for the sake of brevity, further details are omitted here.

[0326] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding flow in the virtual model processing method described in the embodiments of this application. For brevity, further details are omitted here.

[0327] This application also provides a computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding flow in the processing method of the virtual model in this application. For brevity, further details are omitted here.

[0328] It should be understood that the processor in this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0329] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0330] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0331] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0332] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0333] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0334] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0335] In addition, the functional units in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0336] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0337] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing virtual models, characterized in that, The method includes: Multiple target images of a target virtual model in a virtual scene and image correlation data for each target image are acquired. Each target image is obtained by a virtual camera that observes the target virtual model in the virtual scene. The observation position of the virtual camera corresponding to each target image is different. The image correlation data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. The positions of the skeletal joints of the target virtual model in each target image are detected to obtain the position detection results of the skeletal joints; Based on the position detection results and the image correlation data, the position of the skeletal joint in the target image is converted into the position of the skeletal joint in the virtual scene to obtain the target position of the skeletal joint. Based on the target location, generate a skeleton instance corresponding to the target virtual model; Based on the skeleton instance, the target virtual model is skinned to obtain the skinning result of the target virtual model.

2. The method for processing virtual models according to claim 1, characterized in that, The step of detecting the position of the skeletal joints of the target virtual model in each target image to obtain the position detection result of the skeletal joints includes: The target image is input into a pre-trained joint detection model, which detects the positions of the skeletal joints of the target virtual model in the target image, thereby obtaining the position detection result.

3. The method for processing virtual models according to claim 1, characterized in that, The position detection result includes the predicted position of the skeletal joint in each of the target images. The camera geometric parameters include camera intrinsic and camera extrinsic parameters, or include a camera projection matrix determined by the camera intrinsic and extrinsic parameters. The step of converting the position of the skeletal joint in the target image to the position of the skeletal joint in the virtual scene based on the position detection result and the image correlation data, to obtain the target position of the skeletal joint, includes: Based on the predicted position of the skeletal joint in the target image and the camera geometric parameters of the target image, a linear equation related to the position of the skeletal joint in the virtual scene is constructed to obtain the target linear equation corresponding to the target image. The target linear equation is solved to obtain candidate positions of the skeletal joints in the virtual scene corresponding to the target image; The target location is determined based on the candidate locations corresponding to each target image.

4. The virtual model processing method according to claim 3, characterized in that, Determining the target location based on the candidate locations corresponding to each target image includes: The candidate positions of the skeletal joints in each target image are reprojected onto the corresponding target image to obtain the reprojection position for each target image. Determine the reprojection error between the reprojection position corresponding to the target image and the predicted position in the position detection result corresponding to the target image; Based on the reprojection error corresponding to each target image, determine the weighting weight of the candidate position corresponding to each target image; Based on the weighted weights of the candidate positions corresponding to each target image, a weighted solution is performed on all candidate positions of each skeletal joint to obtain the target position.

5. The method for processing virtual models according to claim 4, characterized in that, The step of determining the weighted weight of the candidate position corresponding to each target image when determining the target position based on the reprojection error includes: When the reprojection error is greater than the error threshold, the weighting of the corresponding candidate position is reduced; or, When the reprojection error is greater than the error threshold, the corresponding candidate position is deleted.

6. The method for processing virtual models according to claim 1, characterized in that, The image-related data further includes a 3D position cache, which includes the 3D position of each pixel in the target image in the virtual scene. The step of converting the position of the skeletal joint in the target image to the position of the skeletal joint in the virtual scene based on the position detection result and the image-related data, to obtain the target position of the skeletal joint, includes: Based on the position detection result, the three-dimensional position matching the position detection result is read from the three-dimensional position cache to obtain the target position of the skeletal joint.

7. The method for processing virtual models according to claim 1, characterized in that, The image-related data also includes depth data, which includes the depth of each pixel in the target image. The step of converting the position of the skeletal joint in the target image to the position of the skeletal joint in the virtual scene based on the position detection result and the image-related data, to obtain the target position of the skeletal joint, includes: Based on the position detection results and the depth data, the positions of the skeletal joints in the target image are back-projected into the virtual scene to obtain the target positions of the skeletal joints.

8. The method for processing virtual models according to claim 1, characterized in that, The position detection result includes the predicted position of the skeletal joint in each target image and the predicted probability of the predicted position. The skeletal joint includes multiple joints. Based on the position detection result and the image correlation data, the position of the skeletal joint in the target image is converted into the position of the skeletal joint in the virtual scene to obtain the target position of the skeletal joint, including: Based on the predicted probability of each skeletal joint in each target image, the target weight corresponding to each skeletal joint in each target image is determined; Based on the predicted position of the skeletal joint in each target image and the image-related data corresponding to each target image, the predicted position of the skeletal joint in each target image is converted into the position of the skeletal joint in the virtual scene, thus obtaining the predicted converted position of the skeletal joint in each target image; Based on the target weight of each skeletal joint, the predicted transformation position of each skeletal joint is weighted and solved to obtain the target position of each skeletal joint.

9. The method for processing virtual models according to claim 8, characterized in that, Before determining the target weight corresponding to each skeletal joint in each target image based on the predicted probability corresponding to each skeletal joint in each target image, the method further includes: If the predicted probability of a skeletal joint in the target image is less than a preset probability threshold in the location detection results, the location detection result is deleted.

10. The method for processing virtual models according to claim 8, characterized in that, Before determining the target weight corresponding to each skeletal joint in each target image based on the predicted probability corresponding to each skeletal joint in each target image, the method further includes: If the pixel at the predicted position in the target image does not hit the surface of the foreground object in the target image, the target weight of the skeletal joint corresponding to the predicted position in the target image is set to zero, or the position detection result corresponding to the skeletal joint corresponding to the predicted position in the target image is deleted.

11. The method for processing virtual models according to claim 1, characterized in that, The step of generating a skeleton instance corresponding to the target virtual model based on the target location includes: Based on the target positions of the skeletal joints and the pre-determined skeleton configuration data, a skeleton instance is generated. The skeleton configuration data is used to indicate the bones contained in the skeleton instance, the correspondence between the skeletal joints and the bones in the skeleton instance, the generation rules for generating the corresponding bones in the skeleton instance based on the target positions of the skeletal joints, the name of each bone in the skeleton instance, and the geometric topological relationship of the bones in the skeleton instance.

12. The method for processing virtual models according to claim 1, characterized in that, The skeleton instance includes multiple bones. The step of performing skinning processing on the target virtual model based on the skeleton instance to obtain the skinning processing result of the target virtual model includes: Based on the distance of each model vertex of the target virtual model relative to each bone, a first weight of each bone relative to each model vertex is determined; The skinning weight of each model vertex is determined based on a first weight of each bone relative to each model vertex; The skinning result is obtained based on the skinning weight of each vertex of the model.

13. The method for processing virtual models according to claim 1, characterized in that, The skeleton instance includes multiple bones and multiple joints. The step of performing skinning processing on the target virtual model based on the skeleton instance to obtain the skinning processing result of the target virtual model includes: Each of the aforementioned bones or each of the aforementioned joints is identified as a heat source; The second weight of the heat source is obtained by weight diffusion based on the grid geodesic distance or the graph Laplacian operator corresponding to the heat source. The skin treatment result is obtained based on the second weight of each heat source.

14. The method for processing virtual models according to claim 1, characterized in that, The step of performing skinning processing on the target virtual model based on the skeleton instance to obtain the skinning processing result of the target virtual model includes: Based on the semantic attribution information of each model vertex of the target virtual model, determine the relevant vertex set corresponding to each bone, wherein the relevant vertex set is the set of vertices in the mesh vertices of the target virtual model that are affected by the corresponding bone and do not overlap with each other. Based on the relevant vertex set, a topological isolation constraint is set on the mesh of the target virtual model, wherein the topological isolation constraint is used to prevent the spread of vertex weights between relevant vertex sets of different bones; Based on the topological isolation constraints and the relevant vertex set, the skinning weight of each model vertex is determined by solving the harmonic field, and the skinning processing result is obtained.

15. The method for processing virtual models according to claim 14, characterized in that, The step of determining the set of relevant vertices corresponding to each bone based on the semantic attribution information of each model vertex of the target virtual model includes: Perform semantic segmentation of body parts on the target image to obtain a semantic segmentation mask; Based on the semantic segmentation mask, the three-dimensional vertices corresponding to the semantic segmentation mask are read from the cache information generated when rendering the target image to obtain an initial set of related vertices; A morphological erosion operation is performed on the initial set of related vertices to shrink the region boundary, and mutual exclusion processing is performed on the eroded sets of related vertices corresponding to different bones so that the sets of related vertices of different bones do not contain the same model vertices, thus obtaining the set of related vertices.

16. The method for processing virtual models according to claim 14, characterized in that, The step of performing skinning processing on the target virtual model based on the skeleton instance to obtain the skinning processing result of the target virtual model further includes: After obtaining the skinning process result, a non-negative truncation is performed on the skinning weights; Select the K bones with the highest weights from the weights corresponding to each bone as the influencing bones; The weights of the selected bones are normalized. To ensure that the processed skin weights meet preset constraints, where K is a preset positive integer.

17. A processing apparatus for a virtual model, characterized in that, The device includes: The image data acquisition module is used to acquire multiple target images of a target virtual model in a virtual scene and image-related data of each target image. Each target image is obtained by a virtual camera in the virtual scene that observes the target virtual model. The observation position of the virtual camera corresponding to each target image is different. The image-related data includes at least the camera geometric parameters of the virtual camera observing the target virtual model. The joint detection module is used to detect the position of the skeletal joints of the target virtual model in each target image, and obtain the position detection result of the skeletal joints; The position conversion module is used to convert the position of the skeletal joint in the target image into the position of the skeletal joint in the virtual scene based on the position detection result and the image related data, so as to obtain the target position of the skeletal joint. The skeleton generation module is used to generate a skeleton instance corresponding to the target virtual model based on the target location; The skinning module is used to perform skinning processing on the target virtual model based on the skeleton instance, and obtain the skinning processing result of the target virtual model.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to execute the virtual model processing method according to any one of claims 1-16.

19. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing a computer program, and the processor executing the virtual model processing method according to any one of claims 1-16 by calling the computer program stored in the memory.

Images