Method and system for realizing automatic identification of skeleton points of digital human

Abstract

The invention relates to the field of human body measurement automation, and discloses an automatic skeleton point recognition method and system for a digital human, and the method comprises the steps: firstly recognizing an RGB lens array corresponding to the digital human in a target scene, collecting an omnibearing image, and carrying out the three-dimensional reconstruction to generate image point cloud data; then noise points are detected, filtering parameters are analyzed, noise is reduced, and an optimized point cloud model is constructed; then multi-angle human body images are collected, and joint points are recognized to generate two-dimensional skeleton data; then determining a mapping relation of the two-dimensional skeleton in a three-dimensional space, converting the two-dimensional skeleton into a three-dimensional joint coordinate set, positioning an attitude registration point of the SMPL model and matching a skeleton topological structure; and finally, determining a size measurement value based on the topological structure, and generating an automatic identification result after calibration. According to the invention, the recognition accuracy, efficiency and automation degree of the digital human skeleton points can be improved.

Description

Technical Field

[0001] This invention relates to a method and system for automatic skeletal point recognition of digital humans, belonging to the field of automated human body measurement. Background Technology

[0002] Automatic skeletal point recognition in digital humans refers to the process of automatically detecting and locating key points (such as joint positions) of the human skeleton in a digital human model using computer vision and artificial intelligence technologies. This technology is widely used in fields such as AI digital humans, animation production, and automated anthropometry.

[0003] In existing technologies, skeletal point recognition mainly relies on point cloud output devices to generate high-precision point cloud data through multi-view 3D reconstruction. While this can meet the needs of static display, it has significant shortcomings: First, the point cloud data is massive and difficult to simplify, resulting in huge computational pressure and making it unsuitable for real-time display under multiple conditions; second, the point cloud data is discrete and unrelated to each other, making it difficult to accurately measure skeletal points (such as eye distance) under different human postures; third, operating on the original point cloud is inefficient and requires high computing resources; furthermore, in applications such as clothing customization, manual measurement is still required, and contactless automation cannot be achieved. Therefore, a method for automatic skeletal point recognition of digital humans is needed to improve the accuracy, efficiency, and automation of skeletal point recognition in digital humans. Summary of the Invention

[0004] This invention provides a method and system for automatic recognition of skeletal points in digital humans, the main purpose of which is to improve the accuracy, efficiency and automation of skeletal point recognition in digital humans.

[0005] To achieve the above objectives, the present invention provides a method for automatic skeletal point recognition in digital humans, comprising:

[0006] Identify the RGB lens array corresponding to the digital human in the target scene, and acquire the omnidirectional images corresponding to the RGB lens array. Perform three-dimensional reconstruction on the omnidirectional images to generate image point cloud data corresponding to the omnidirectional images.

[0007] The noise distribution area in the image point cloud data is detected, and the filtering parameters corresponding to the noise distribution area are analyzed. A noise reduction and smoothing operation is performed on the filtering parameters to construct an optimized point cloud model corresponding to the image point cloud data.

[0008] Collect multi-angle human images corresponding to the optimized point cloud model, identify human joints in the multi-angle human images, and generate two-dimensional skeleton data corresponding to the multi-angle human images based on the human joints.

[0009] Determine the position mapping relationship of the two-dimensional skeleton data in three-dimensional space, and transform the three-dimensional joint coordinate set corresponding to the position mapping relationship. Based on the three-dimensional joint coordinate set, locate the preset SMPL model's pose registration point in the optimized point cloud model, and match the skeleton topology structure corresponding to the pose registration point.

[0010] Based on the skeletal topology, the size measurement values ​​corresponding to the skeletal points of the digital human in the target scene are determined, and the size measurement values ​​are calibrated to generate automatic recognition results corresponding to the skeletal points of the digital human in the target scene.

[0011] Optionally, the step of performing three-dimensional reconstruction on the omnidirectional captured image to generate image point cloud data corresponding to the omnidirectional captured image includes:

[0012] Query the feature descriptors in the omnidirectional captured image;

[0013] Based on the feature descriptor, the inter-image matching degree corresponding to the omnidirectional captured image is calculated using the following formula.

[0014]

[0015] Among them, S I,J N represents the image matching degree between the omnidirectional images I and J. I N represents the total number of feature points in image I. J Let J represent the total number of feature points in image J, L represent the number of valid matching point pairs verified by geometry, and k represent the index of the number of feature point pairs. Let represent the descriptor vector of the k-th matching point pair in image I. Let σ represent the descriptor vector of the k-th matching point pair in image J, and σ represent the similarity bandwidth parameter.

[0016] Analyze the positional mapping relationship corresponding to the inter-graph matching degree;

[0017] Based on the position mapping relationship, the three-dimensional image point set corresponding to the all-around captured image is divided;

[0018] Mark the key feature points in the three-dimensional image point set;

[0019] Based on the key feature points, image point cloud data corresponding to the omnidirectional captured image is generated.

[0020] Optionally, performing a noise reduction and smoothing operation on the filtering parameters to construct an optimized point cloud model corresponding to the image point cloud data includes:

[0021] Identify the noise characteristic level corresponding to the filtering parameters;

[0022] Based on the noise characteristic level, high-density noise blocks in the noise distribution area are selected;

[0023] Activate the adaptive filter corresponding to the high-density noise block;

[0024] Based on the adaptive filter, noise reduction and smoothing are performed on the filtering parameters to obtain a set of residual noise points;

[0025] Based on the residual noise set, an optimized point cloud model corresponding to the image point cloud data is constructed.

[0026] Optionally, the step of filtering high-density noise blocks in the noise distribution area based on the noise characteristic level includes:

[0027] Set the density determination threshold corresponding to the noise feature level;

[0028] Based on the density determination threshold, the noise distribution area is divided into regional grid units;

[0029] Statistically calculate the noise density value within the grid cells of the region;

[0030] Generate a noise density map corresponding to the noise density value;

[0031] Based on the noise density map, high-density noise blocks are selected in the noise distribution area.

[0032] Optionally, generating two-dimensional skeleton data corresponding to the multi-angle human body image based on the human body joints includes:

[0033] Query the set of location coordinates corresponding to the human body joint points;

[0034] Based on the set of position coordinates, analyze the skeleton connection relationships in the multi-angle human body images;

[0035] Based on the skeleton connection relationship, key skeleton nodes in the multi-angle human body image are located;

[0036] Parse the connection identifiers in the key skeleton nodes;

[0037] Based on the connection identifier, two-dimensional skeleton data corresponding to the multi-angle human body image is generated.

[0038] Optionally, locating key skeleton nodes in the multi-angle human body image based on the skeleton connection relationship includes:

[0039] Analyze the node connectivity corresponding to the skeleton connection relationship;

[0040] Based on the node connectivity, highly connected points in the multi-angle human body image are determined;

[0041] Based on the highly connected points, identify the multi-branch structure in the multi-angle human body image;

[0042] Query the physiological priority corresponding to the multi-branch structure;

[0043] Based on the physiological priority, key skeletal nodes in the multi-angle human body image are located.

[0044] Optionally, locating the preset pose registration points of the SMPL model in the optimized point cloud model based on the three-dimensional joint coordinate set includes:

[0045] The joint position values ​​corresponding to the three-dimensional joint coordinate set are calculated using the following formula:

[0046]

[0047] Where Gp represents the joint position value corresponding to the three-dimensional joint coordinate set, M represents the total number of joints, j represents the joint index, and x represents the joint position value. j Let x and y represent the x-coordinate of the j-th joint in three-dimensional space. j Let y be the y-coordinate of the j-th joint in three-dimensional space, and z be the z-coordinate of the j-th joint. j Let μ represent the z-coordinate of the j-th joint in three-dimensional space. x μ represents the average x-coordinate of all joints. y μ represents the average y-coordinate of all joints. z This represents the average z-coordinate of all joints;

[0048] Query the standard coordinate data corresponding to the preset SMPL model;

[0049] Based on the joint position values ​​and standard coordinate data, identify the candidate point set near each joint position in the optimized point cloud model;

[0050] Analyze the attitude deviation index corresponding to the candidate point set;

[0051] Based on the attitude deviation index, the preset attitude registration point of the SMPL model in the optimized point cloud model is located.

[0052] Optionally, the matching of the skeletal topology corresponding to the pose registration point includes:

[0053] Analyze the spatial position information corresponding to the attitude registration point;

[0054] Identify the geometric intervals corresponding to key locations in the location information;

[0055] Filter the valid connection portions in the geometric interval;

[0056] Mark the stable reference position corresponding to the effective connection portion;

[0057] Based on the stable reference position, the skeletal topology corresponding to the attitude registration point is matched.

[0058] Optionally, determining the size measurements corresponding to the skeletal points of the digital human in the target scene based on the skeletal topology includes:

[0059] Analyze the joint connection relationships corresponding to the skeletal topology;

[0060] Based on the joint connection relationship, the bone segment region of the digital human skeleton point in three-dimensional space is determined;

[0061] Locate the joint coordinate points in the bone segment region;

[0062] Query the three-dimensional distance value corresponding to the joint coordinate point;

[0063] Based on the three-dimensional distance value, the size measurement value corresponding to the digital human skeleton point in the target scene is determined.

[0064] To address the above problems, the present invention also provides a system for automatic skeletal point recognition in digital humans, the system comprising:

[0065] The data generation module is used to identify the RGB lens array corresponding to the digital human in the target scene, and to collect the omnidirectional shooting images corresponding to the RGB lens array, and to perform three-dimensional reconstruction on the omnidirectional shooting images to generate image point cloud data corresponding to the omnidirectional shooting images.

[0066] The model building module is used to detect the noise distribution area in the image point cloud data, analyze the filtering parameters corresponding to the noise distribution area, and perform noise reduction and smoothing operations on the filtering parameters to build an optimized point cloud model corresponding to the image point cloud data.

[0067] The skeleton data module is used to collect multi-angle human images corresponding to the optimized point cloud model, identify human joints in the multi-angle human images, and generate two-dimensional skeleton data corresponding to the multi-angle human images based on the human joints.

[0068] The structure matching module is used to determine the position mapping relationship of the two-dimensional skeleton data in three-dimensional space, and convert the three-dimensional joint coordinate set corresponding to the position mapping relationship. Based on the three-dimensional joint coordinate set, the module locates the preset SMPL model's pose registration point in the optimized point cloud model and matches the skeleton topology structure corresponding to the pose registration point.

[0069] The result generation module is used to determine the size measurement values ​​corresponding to the skeletal points of the digital human in the target scene based on the skeletal topology, and to perform data calibration on the size measurement values ​​in order to generate automatic recognition results corresponding to the skeletal points of the digital human in the target scene.

[0070] Compared to the problems described in the background technology, this invention reduces reliance on dedicated point cloud equipment and lowers hardware costs and usage barriers by identifying the RGB lens array corresponding to the digital human in the target scene and acquiring omnidirectional images corresponding to the RGB lens array. Simultaneously, the omnidirectional images provide rich multi-angle visual information for subsequent 3D reconstruction, improving the integrity and accuracy of point cloud data and laying an efficient foundation for the entire skeletal point recognition process. This invention also accurately locates interfering data by detecting noise distribution areas in the image point cloud data and analyzing the corresponding filtering parameters, providing a targeted basis for noise reduction and avoiding indiscriminate processing that damages effective information. By performing operations with adapted filtering parameters, the quality of the point cloud model can be effectively optimized, facilitating the efficient construction of accurate skeletal topology. Furthermore, this invention, by acquiring multi-angle human images corresponding to the optimized point cloud model and identifying human joints in the multi-angle human images, can capture data from multiple perspectives. This invention captures key limb information of a digital human, providing rich feature anchors for skeletal structure analysis. By accurately locating joints, it establishes a correlation between two-dimensional image information and three-dimensional point cloud data, laying the foundation for subsequent skeleton data generation. Furthermore, by determining the positional mapping relationship of the two-dimensional skeleton data in three-dimensional space and transforming the corresponding three-dimensional joint coordinate set, the planar skeleton information can be upgraded to three-dimensional space. This provides accurate three-dimensional references for skeletal point measurement under different postures and complete spatial data support for automatic skeletal point recognition. Finally, based on the skeletal topology, this invention determines the dimensional measurement values ​​corresponding to the skeletal points of the digital human in the target scene. By leveraging standardized skeletal connection relationships, it provides clear structural references for dimensional measurement, ensuring that the measured values ​​are consistent with the anatomical laws of the real human body. Simultaneously, it provides a quantitative data foundation for applications such as digital human posture analysis and motion simulation, improving the accuracy and practicality of digital human technology in scenario applications. Therefore, the automatic skeletal point recognition method and system provided by this invention can improve the accuracy, efficiency, and automation of digital human skeletal point recognition. Attached Figure Description

[0071] Figure 1 This is a flowchart illustrating a method for automatically recognizing skeletal points in a digital human, as provided in an embodiment of the present invention.

[0072] Figure 2 This is a schematic diagram illustrating a multi-view 3D reconstruction method for automatically recognizing skeletal points in a digital human, provided in an embodiment of the present invention.

[0073] Figure 3 This invention provides a 2D image human joint point reasoning graph in a method for automatic skeletal point recognition of digital humans, as provided in an embodiment of the present invention.

[0074] Figure 4 This invention provides an anthropometric map in a method for automatically recognizing skeletal points in a digital human, as an embodiment of the present invention.

[0075] Figure 5 This is a schematic diagram of a module for implementing an automatic skeletal point recognition system for a digital human, provided as an embodiment of the present invention.

[0076] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0077] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0078] This application provides a method for automatically recognizing skeletal points in a digital human. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method for automatically recognizing skeletal points in a digital human can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0079] Reference Figure 1 The diagram shown is a flowchart illustrating a method for automatically recognizing skeletal points in a digital human according to an embodiment of the present invention. In this embodiment, the method for automatically recognizing skeletal points in a digital human includes:

[0080] S1. Identify the RGB lens array corresponding to the digital human in the target scene, and acquire the omnidirectional shooting image corresponding to the RGB lens array. Perform three-dimensional reconstruction on the omnidirectional shooting image to generate image point cloud data corresponding to the omnidirectional shooting image.

[0081] This invention reduces reliance on dedicated point cloud equipment and lowers hardware costs and barriers to entry by identifying the RGB lens array corresponding to the digital human in the target scene and acquiring omnidirectional images of the RGB lens array. At the same time, the omnidirectional images can provide rich multi-angle visual information for subsequent 3D reconstruction, improve the integrity and accuracy of point cloud data, and lay an efficient foundation for the entire skeleton point recognition process.

[0082] The target scene refers to a specific environment capable of skeletal point recognition of a digital human. This environment must meet the lighting and spatial conditions required for RGB lens array shooting to ensure the accuracy of image acquisition and subsequent processing. For example, in a clothing customization target scene, the lighting must be uniform, the space must be large enough to accommodate a shooting area with a diameter of at least 3 meters, and objects must be avoided to prevent obstruction. This allows the RGB lens array to clearly capture images of the digital human from various angles, providing a suitable basic environment for skeletal point recognition. The digital human refers to a virtual image constructed using computer technology that possesses human-like physical characteristics and certain behavioral expressions. It can simulate human postures and movements and is widely used in various fields. For example, a digital human used for clothing display on an e-commerce platform has a body proportion similar to a real human and can perform actions such as walking and turning. The figure is set to a height of 175 cm and a shoulder width of 45 cm. It can accurately display the effect of clothing in different postures through skeletal point recognition. The RGB lens array refers to a shooting device composed of multiple RGB lenses arranged in a specific manner. These lenses can simultaneously capture images from different angles, thereby obtaining comprehensive visual information of the subject. For example, in an animation production scene, a circular array of 12 RGB lenses, each 30 cm apart, distributed on a circular track with a diameter of 2 meters, can simultaneously capture images of the digital human from 12 angles, including the front, side, and back, providing multi-view data for subsequent 3D reconstruction. The comprehensive captured images refer to the image set obtained after capturing the digital human from multiple angles using the RGB lens array, which can fully present the digital human's appearance and posture from all directions. For example, in a virtual anchor live streaming scenario, an RGB lens array captures images from eight angles: 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°. The resulting eight images form a 360-degree panoramic image, clearly showing the digital human's facial expressions, body lines, and clothing details. Optionally, the RGB lens array corresponding to the digital human in the target scene can be identified using a camera calibration algorithm, such as using the checkerboard calibration method from the OpenCV library combined with scene space coordinate mapping to calculate the intrinsic and extrinsic parameters of each lens, thereby obtaining the spatial pose parameter set of the RGB lens array. The acquisition of the 360-degree panoramic image corresponding to the RGB lens array can be achieved using multi-camera synchronous triggering technology, such as using the Multicam SDK tool to control all lenses in the array to perform exposure and phase locking at a preset timestamp, thereby obtaining a seamless stitched panoramic image frame sequence covering 360 degrees.

[0083] Furthermore, by performing three-dimensional reconstruction on the omnidirectional captured images to generate image point cloud data corresponding to the omnidirectional captured images, the present invention can transform two-dimensional image information into a three-dimensional spatial structure, providing a three-dimensional foundation for digital human skeleton point recognition; through multi-view image fusion, it can make up for the information loss of single-angle shooting, laying an efficient and reliable data foundation for subsequent skeleton point processing.

[0084] The image point cloud data refers to a dataset formed by integrating the three-dimensional coordinate information of all points in a three-dimensional image point set with key feature points as the core. It fully presents the morphological details of a digital human in three-dimensional space in the form of a large number of discrete points. For example, the generated digital human full-body image point cloud data contains 100,000 points, each with precise three-dimensional coordinates, which can meticulously show the three-dimensional features of the digital human, such as body shape and limb posture.

[0085] As an embodiment of the present invention, the step of performing three-dimensional reconstruction on the omnidirectional captured image to generate image point cloud data corresponding to the omnidirectional captured image includes: querying feature descriptors in the omnidirectional captured image; calculating the inter-image matching degree corresponding to the omnidirectional captured image based on the feature descriptors; analyzing the positional mapping relationship corresponding to the inter-image matching degree; dividing the three-dimensional image point set corresponding to the omnidirectional captured image based on the positional mapping relationship; marking key feature points in the three-dimensional image point set; and generating image point cloud data corresponding to the omnidirectional captured image based on the key feature points.

[0086] The feature descriptor refers to a numerical vector or data structure extracted from omnidirectional images to characterize local features of an image. It uniquely identifies key regions in the image, such as edges and corners, facilitating feature matching between different images. For example, in an image of the shoulder area of ​​a digital human, a feature descriptor consisting of 128 numerical values ​​can be extracted. This descriptor accurately depicts the texture and curvature of the shoulder contour, providing a basis for subsequent matching of images from different angles. The inter-image matching degree refers to a quantitative indicator calculated using the above formula, used to measure the tightness of feature matching and the degree of similarity between omnidirectional images I and J. The larger the value, the better the matching degree of corresponding features between the two images, and the more reliable the correlation basis for subsequent 3D reconstruction. For example, the S-value of image I (frontal view of the digital human) and image J (side view of the digital human) can be calculated. I,J=0.8 indicates a high degree of feature matching between the two, which can be used to collaboratively construct the 3D form of a digital human. The position mapping relationship refers to the coordinate transformation relationship of corresponding feature points in space in different omnidirectional images determined based on the inter-image matching degree. It can clarify the positional correspondence rules of the same feature under different viewpoints. For example, the coordinates of a feature point in the frontal image are (100, 200), and the corresponding coordinates in the side image are (150, 180) after position mapping calculation. This relationship can be used to integrate 2D image features into 3D space. The 3D image point set refers to the points formed after transforming the 2D feature points in the omnidirectional images into 3D space based on the position mapping relationship. These points initially form the morphological outline of the digital human in three-dimensional space. For example, after transformation, the head region of the digital human forms a three-dimensional image point set containing 500 points. The X, Y, and Z coordinates of these points are distributed within a specific range, roughly presenting the three-dimensional shape of the head. The key feature points refer to points in the three-dimensional image point set that are of great significance in characterizing the morphology and skeletal structure of the digital human. They usually correspond to key parts of the human body such as joints and contour turning points. For example, in the three-dimensional image point set, the point corresponding to the elbow joint of the digital human is marked with coordinates (300, 450, 200). This point is key to connecting the upper arm and forearm and is crucial for the subsequent construction of the skeletal topology.

[0087] Furthermore, the querying of feature descriptors in the omnidirectional image can be achieved through local feature extraction algorithms, such as using the SIFT operator to calculate the scale-invariant feature vectors of key points in the image, thereby obtaining a 128-dimensional feature descriptor set; the analysis of the positional mapping relationship corresponding to the matching degree between images can be achieved through feature matching optimization methods, such as using the RANSAC algorithm to remove mismatched points and fitting the optimal homography transformation matrix, thereby obtaining the precise positional mapping relationship between images; the partitioning of the three-dimensional image point set corresponding to the omnidirectional image can be achieved through multi-view stereo vision methods, such as performing sparse reconstruction based on the COLMAP framework to generate a set of three-dimensional point coordinates of the scene, thereby obtaining a three-dimensional image point set with texture information; the marking of key feature points in the three-dimensional image point set can be achieved through saliency detection algorithms, such as using the Harris corner detector to screen geometric feature points with significant curvature changes in the point cloud, thereby obtaining a set of key feature point markers for registration; the generation of image point cloud data corresponding to the omnidirectional image can be achieved through dense reconstruction techniques, such as using OpenMVS to perform depth map fusion on multi-view images to generate a dense point cloud, thereby obtaining a color point cloud data model with millimeter-level precision.

[0088] As another embodiment of the present invention, calculating the inter-image matching degree corresponding to the omnidirectional captured image based on the feature descriptor includes:

[0089]

[0090] Among them, S I,J N represents the image matching degree between the omnidirectional images I and J. I N represents the total number of feature points in image I. J Let J represent the total number of feature points in image J, L represent the number of valid matching point pairs verified by geometry, and k represent the index of the number of feature point pairs. Let represent the descriptor vector of the k-th matching point pair in image I. Let σ represent the descriptor vector of the k-th matching point pair in image J, and σ represent the similarity bandwidth parameter.

[0091] Specifically, the total number of feature points refers to the total number of points extracted from image I used to characterize local image features (such as edges, corners, regions with unique textures, etc.); N J Similarly, consider the number of such points in image J. For example, extracting feature points from a frontal image I of a digital human's upper body, and obtaining N through an algorithm. I =200, covering feature points in areas such as shoulder contours and neckline edges; N extracted from the side image J. J = 180, including feature points such as the side waist and the side of the arm; the effective matching point pair number refers to the number of real and reliable matching point pairs after the initial matching feature point pairs are geometrically verified (such as checking whether the feature point pairs meet spatial geometric constraints, whether they are within reasonable epipolar geometric relationships, etc.). Assuming that 300 point pairs are initially matched for images I and J, after geometric verification (such as verifying whether the point pairs conform to the spatial laws of digital human limb movement), incorrect matches (such as point pairs mismatched due to background interference) are eliminated, and finally L = 250, that is, the effective matching point pair number is 250; the descriptor vector refers to the digital representation of the local features corresponding to the k-th matching point pair in the image, which is a vector that encodes the gray level, texture, gradient and other information around the feature point. For example, the descriptor vector of the 5th matching point pair in image I (corresponding to the feature point of the digital human elbow) is... (The vector length depends on the feature extraction algorithm, such as 128 dimensions in the SIFT algorithm), the descriptor vector corresponding to the 5th matching point pair in image J. This parameter is used to calculate the similarity between point pairs. The similarity bandwidth parameter is a parameter used to control the range of influence of descriptor vector differences on the matching degree when calculating the matching degree. The larger the value of σ, the more gradual the decay effect of descriptor vector differences on the matching degree. The smaller the value of σ, the more significant the influence of the differences. For example, when σ = 0.1, small differences between descriptor vectors will cause the matching degree to decrease rapidly. If σ = 0.3, the matching degree decreases relatively slowly under the same difference. In practical applications, it is necessary to adjust according to the complexity of image features and matching requirements. For example, when processing fine facial feature matching of digital humans, σ = 0.2 can be set.

[0092] Specifically, for a more intuitive understanding of the execution logic and data flow relationships in the 3D reconstruction process of this solution, please refer to [link / reference]. Figure 2 .Should Figure 2 As an illustration of a multi-view 3D reconstruction system for digital human skeletal point recognition, this diagram clearly presents the key link from multi-view image acquisition to 3D point cloud construction: The image layer focuses on an RGB lens array to capture the digital human from all angles (the red pyramid represents the lens and shooting angle), serving as the basic data source for 3D reconstruction; the processing layer transforms 2D image information into 3D spatial data through a step-by-step logic of "extracting feature descriptors → calculating inter-image matching degree → determining positional mapping relationship → constructing a 3D image point set → marking key feature points"; the output layer uses "image point cloud data (the digital human and surrounding discrete points in the image)" as the final result. It should be noted that the relationships between the elements in the diagram are essentially an abstract refinement of the 3D reconstruction logic for digital humans. In real-world scenarios, the complexity of feature matching (such as precise alignment of feature points in multi-view images) and the diversity of point cloud construction (the impact of different reconstruction algorithms on point cloud density and accuracy) far exceed what is shown in the diagram. This architecture only provides a concise demonstration of the core logic, offering an intuitive reference for understanding the systematic approach to 3D reconstruction in digital human skeletal point recognition.

[0093] S2. Detect the noise distribution area in the image point cloud data, analyze the filtering parameters corresponding to the noise distribution area, and perform noise reduction and smoothing operation on the filtering parameters to construct an optimized point cloud model corresponding to the image point cloud data.

[0094] This invention can accurately locate interfering data by detecting the noise distribution area in the image point cloud data and analyzing the corresponding filtering parameters of the noise distribution area, providing a targeted basis for noise reduction and avoiding indiscriminate processing that damages effective information; by performing operations with adapted filtering parameters, it can effectively optimize the quality of the point cloud model and help to efficiently construct accurate skeletal topology structures.

[0095] The noise distribution area refers to an abnormal point cluster in the image point cloud data that deviates from the true form of the digital human due to interference from the shooting environment (such as light reflection, equipment error) or defects in the reconstruction algorithm. These points do not reflect the true structure of the digital human and will interfere with subsequent skeletal point recognition. For example, in the point cloud data of a digital human hand, due to ambient light refraction during shooting, an area containing 200 discrete abnormal points is formed at the edge of the finger. The coordinates of these points deviate from the true finger contour by more than 5mm, which constitutes the noise distribution area. The filtering parameters refer to the key values ​​in the algorithm that control the filtering effect when filtering noise from the point cloud data. Different types of filtering (such as Gaussian filtering and bilateral filtering) correspond to different parameters, such as the standard deviation of Gaussian filtering, and the spatial distance and gray-level difference threshold of bilateral filtering. For noise filtering on the shoulders of a digital human, when selecting Gaussian filtering, setting the standard deviation parameter to 1.2 allows points within a 1.2-standard-deviation range from the center to participate in the weighted calculation, achieving noise smoothing while preserving key skeletal contour features. Optionally, the detection of noise distribution regions in the image point cloud data can be achieved through density clustering algorithms, such as using the DBSCAN method to analyze the spatial distribution density of the point cloud and mark outlier clusters, thereby obtaining the set of spatial coordinates of the noise distribution regions. The analysis of the filtering parameters corresponding to the noise distribution regions can be achieved through adaptive threshold calculation methods, such as dynamically calculating the neighborhood standard deviation threshold based on the StatisticalOutlierRemoval module of the PCL library, thereby obtaining the optimal combination of filtering radius and intensity parameters suitable for the current point cloud.

[0096] Furthermore, by performing noise reduction and smoothing operations on the filtering parameters to construct an optimized point cloud model corresponding to the image point cloud data, the present invention can remove noise interference in the point cloud data, improve data purity, and provide a more reliable foundation for subsequent skeletal point recognition; it can enhance the continuity and correlation of point cloud data, helping to accurately measure skeletal points under different postures; it can also reduce computational complexity, reduce resource consumption, and improve overall process efficiency.

[0097] The optimized point cloud model refers to a point cloud model based on the original image point cloud data, which, after noise reduction, smoothing, and residual noise processing, significantly reduces noise and improves the accuracy and integrity of the point cloud, thus more accurately reflecting the three-dimensional form of the digital human. It provides high-quality data support for skeletal point recognition and posture registration. For example, in the optimized full-body point cloud model of the digital human, the number of noise points is reduced from 10,000 to 200, and the point cloud deviation is controlled within 0.02mm, which can clearly present the three-dimensional structure of the skeletal joints.

[0098] As an embodiment of the present invention, the step of performing noise reduction and smoothing operation on the filtering parameters to construct an optimized point cloud model corresponding to the image point cloud data includes: identifying the noise feature level corresponding to the filtering parameters; filtering high-density noise blocks in the noise distribution region according to the noise feature level; activating the adaptive filter corresponding to the high-density noise blocks; performing noise reduction and smoothing on the filtering parameters according to the adaptive filter to obtain a residual noise set; and constructing an optimized point cloud model corresponding to the image point cloud data based on the residual noise set.

[0099] The noise feature level refers to an index that quantifies and classifies the characteristics of noise in image point cloud data, such as intensity, distribution density, and range of influence, based on filtering parameters. It divides noise into different levels (such as low, medium, and high) by analyzing the amplitude differences and spatial clustering of noise points, and is used to guide subsequent noise reduction strategies. For example, in the point cloud data of a digital human's back, the noise amplitude is between 0.1-0.3 mm, and the spatial distribution density is 10 noise points per cubic centimeter. The noise characteristic level is determined to be "medium," corresponding to the selection of a medium-intensity noise reduction algorithm. The high-density noise block refers to a block composed of continuous grid cells in the noise density map where the noise density value exceeds the density threshold. This is the part where noise is highly concentrated and has the greatest interference to subsequent processing. It must meet the conditions of continuous cells and overall density exceeding the threshold. For example, in the noise distribution area of ​​the digital human's shoulder, there are 5 continuous grid cells (each 1 cubic centimeter) with noise density values ​​of 90, 85, 95, 88, and 92 noise points / cubic centimeter, all exceeding the threshold of 80 noise points / cubic centimeter. These 5 cells together constitute a high-density noise block. The adaptive filter refers to a filter that can adjust the noise characteristics of the high-density noise block (e.g., The filter automatically adjusts its filtering parameters (such as kernel size and weighting coefficients) to dynamically adapt to the noise reduction needs of different blocks, avoiding over- or under-filtering. For example, for the high-density noise block (80 noise points / millimeter) in the elbow of a digital human, the filter automatically adjusts the kernel size from 3×3 to 5×5, and the weighting coefficients dynamically change according to the noise amplitude to accurately remove noise. The residual noise set refers to the collection of a small number of noise points that remain in the point cloud data after the adaptive filter has denoised and smoothed the data. These noise points are difficult to remove by conventional filtering due to their complex features (such as being highly confused with the features of the real point cloud), and require further fine processing. For example, after denoising the point cloud of the digital human wrist, the residual noise set contains 20 noise points distributed in the joint gap area, with an amplitude of 0.05-0.1mm, which needs to be further optimized in combination with the skeletal topology.

[0100] Furthermore, the identification of the noise feature level corresponding to the filtering parameters can be achieved through frequency domain analysis methods, such as using Fast Fourier Transform to convert point cloud noise into a frequency domain energy spectrum and dividing intensity intervals to obtain noise feature level classification results based on spectral amplitude; the activation of the adaptive filter corresponding to the high-density noise block can be achieved through dynamic convolution kernel methods, such as using a bilateral filtering algorithm to automatically adjust the spatial domain and value domain kernel function parameters according to the local point density to obtain a smart filter instance with regional adaptability; the noise reduction and smoothing of the filtering parameters can be achieved through iterative optimization methods, such as using the moving least squares method to perform multi-round surface fitting and residual analysis on the noise region to obtain a set of coordinates of the unremoved discrete residual noise points; the construction of the optimized point cloud model corresponding to the image point cloud data can be achieved through surface reconstruction technology, such as using the Poisson reconstruction algorithm to convert the filtered point cloud into a triangular mesh model with a continuous surface to obtain an optimized three-dimensional point cloud model that retains geometric features.

[0101] As another embodiment of the present invention, the step of filtering high-density noise blocks in the noise distribution area according to the noise feature level includes: setting a density judgment threshold corresponding to the noise feature level; dividing the noise distribution area into regional grid units based on the density judgment threshold; calculating the noise density values ​​within the regional grid units; generating a noise density map corresponding to the noise density values; and filtering high-density noise blocks in the noise distribution area based on the noise density map.

[0102] The density threshold is a critical value set according to the noise characteristic level to distinguish the degree of noise density. It is the standard for judging whether an area is a high-density noise block. Different noise characteristic levels correspond to different thresholds. The higher the level, the higher the threshold is usually. For example, when the noise characteristic level is "high", the density threshold is set to 80 noise points per cubic centimeter. That is, when the noise density of a certain area exceeds this value, it is judged as a possible high-density noise block, providing a clear boundary for subsequent screening. The area grid unit refers to a three-dimensional grid unit into which the noise distribution area is divided according to a preset size. This facilitates the statistical analysis of noise density in each area. The size of the grid unit needs to be adapted to the noise distribution characteristics. Too small a size will increase the amount of calculation, while too large a size will reduce the accuracy. For example, for the noise distribution area of ​​a digital human chest, the area is divided into area grid units with a volume of 1 cubic centimeter. Each unit has a spatial range of 1 cm in length, 1 cm in width, and 1 cm in height, forming a total of 200 such units, which are used to accurately count the number of noise points in each local area. The noise density value refers to the number of noise points contained within a unit volume of a grid cell. It is an indicator that quantifies the density of noise points. A higher value indicates that the noise is more concentrated in that area. It is calculated by counting the total number of noise points in each grid cell and then dividing by the cell volume. For example, if a grid cell has a volume of 0.5 cubic centimeters and contains 60 noise points, its noise density value is 120 noise points / cubic centimeter, which directly reflects the noise density of that cell. The noise density map refers to a map generated by visualization technology based on the noise density values ​​of each grid cell. It can intuitively show the density changes of noise points throughout the entire distribution area. Different colors or grayscale values ​​are usually used in the map to represent high and low density, making it easy to quickly identify high-density areas. For example, in the generated digital human leg noise density map, the red area represents a noise density of more than 100 noise points / cubic centimeter, and the blue area represents a density of less than 30 noise points / cubic centimeter, clearly showing that there is a significant high-density noise cluster near the knee.

[0103] Furthermore, the setting of the density judgment threshold corresponding to the noise feature level can be achieved through clustering analysis methods, such as using the K-means algorithm to perform unsupervised classification of noise samples and calculate the inter-class distance threshold, thereby obtaining a quantitative judgment threshold for distinguishing high and low density regions; the division of the noise distribution area into regional grid units can be achieved through spatial binning methods, such as using an Octree structure to recursively subdivide the three-dimensional space until a preset accuracy is reached, thereby obtaining a uniformly distributed set of spatial grid units; the statistical analysis of the noise density values ​​within the regional grid units can be achieved through kernel density estimation algorithms, such as using the Parzen window function to calculate the probability density distribution of noise points within each grid unit, thereby obtaining standardized noise density statistics; the generation of the noise density map corresponding to the noise density values ​​can be achieved through interpolation rendering methods, such as using the Marching Cubes algorithm to convert discrete density values ​​into continuous isosurfaces and color rendering, thereby obtaining a three-dimensional noise density visualization map; the filtering of high-density noise blocks in the noise distribution area can be achieved through connected component analysis methods, such as using 3D Flood The Fill algorithm marks adjacent grid cell clusters whose density exceeds a threshold, thereby obtaining a spatially continuous set of high-density noise blocks.

[0104] S3. Collect multi-angle human images corresponding to the optimized point cloud model, identify human joints in the multi-angle human images, and generate two-dimensional skeleton data corresponding to the multi-angle human images based on the human joints.

[0105] This invention acquires multi-angle human images corresponding to the optimized point cloud model and identifies human joints in the multi-angle human images. It can capture information on key parts of digital human limbs from multiple perspectives, providing rich feature anchors for skeletal structure analysis. By accurately locating joints, it can establish a correlation between two-dimensional image information and three-dimensional point cloud data, laying the foundation for subsequent skeleton data generation.

[0106] The multi-angle human body images refer to a series of images captured from different directions and angles, covering the front, side, back, and multiple tilt angles. This comprehensively captures the spatial morphology and contour features of various parts of the human body. By using complementary information from multiple perspective images, blind spots or occlusion problems from single-angle shots can be reduced, providing a more complete reference for subsequent joint point recognition. For example, when photographing a digital human model, images are captured from 12 angles, including 0° (front), 45° (right front), 90° (right side), 135° (right rear), and 180° (back), with each angle spaced 30° apart, forming a multi-angle human body image set covering the entire body, fully presenting the three-dimensional posture of the limbs. The human joint points refer to the points where the human skeleton connects. Feature points are pivotal points in limb movement, possessing clear anatomical locations and motor functions. Examples include the shoulder, elbow, wrist, hip, knee, and ankle joints. These points typically appear as turning points or abrupt changes in limb shape within an image, serving as key markers for analyzing human posture and movement trajectories. For instance, in a digital human side view image, the hip joint (located at the midpoint between the two sides of the pelvis), knee joint (where the thigh connects to the lower leg), and ankle joint (where the lower leg connects to the foot) can be identified. The knee joint, with coordinates (320, 480) in the image coordinate system, is a core reference point for determining the leg flexion angle. Optionally, the acquisition of multi-angle human images corresponding to the optimized point cloud model can be achieved using a multi-view synchronous shooting system, such as Azure. The Kinect DK depth camera array synchronously acquires data around the target object in 360 degrees, thereby obtaining 12 sets of RGB-D human body image sequences from different perspectives. The identification of human body joints in the multi-angle human body images can be achieved through deep learning pose estimation algorithms, such as using a convolutional neural network based on the OpenPose framework to extract key points of the human skeleton in the image, thereby obtaining a three-dimensional pose data set containing 25 joints.

[0107] Furthermore, based on the human joints, the present invention generates two-dimensional skeleton data corresponding to the multi-angle human images, which can integrate scattered joint information into a structured skeleton framework, intuitively presenting the connection relationship and posture features of the digital human limbs; providing a unified reference for feature association between multi-angle images, enhancing the consistency of limb structure under different perspectives, and improving the efficiency and reliability of the overall recognition process.

[0108] The two-dimensional skeleton data refers to the two-dimensional structured data formed by integrating human joints according to the skeleton connection relationship based on connection identifiers. It presents the overall shape of the human skeleton in the form of node coordinates and connection relationships. For example, the generated digital human upper body two-dimensional skeleton data contains 12 joint coordinates. The connection paths such as shoulder joint-elbow joint-wrist joint and neck-shoulder joint are marked by connection identifiers. The data is stored in the form of an array, which can intuitively reflect the bending angle and overall posture of the upper limbs.

[0109] As an embodiment of the present invention, generating two-dimensional skeleton data corresponding to the multi-angle human body image based on the human body joints includes: querying the set of position coordinates corresponding to the human body joints; analyzing the skeleton connection relationships in the multi-angle human body image based on the set of position coordinates; locating key skeleton nodes in the multi-angle human body image based on the skeleton connection relationships; parsing the connection identifiers in the key skeleton nodes; and generating two-dimensional skeleton data corresponding to the multi-angle human body image based on the connection identifiers.

[0110] The location coordinate set refers to the set of coordinate information of all identified human joints in a two-dimensional coordinate system of a multi-angle human image. Each joint corresponds to a unique (x, y) coordinate, used to accurately mark the spatial position of the joint. For example, in a frontal image of a digital human, the identified shoulder joint coordinates are (200, 150), the elbow joint is (220, 250), and the wrist joint is (230, 350). These coordinates together form the location coordinate set, providing a precise positional basis for subsequent analysis of skeletal connections. The skeletal connection relationship refers to the anatomical connection rules between human joints derived from the location coordinate set, i.e., which joints should be connected through skeletal structures, reflecting the natural movement relationships of the limbs. For example, in a multi-angle image, location coordinate calculations show that the shoulder joint (200, 150) and the elbow joint (220, 250) are close together and their connection conforms to the upper limb skeletal orientation. The elbow joint (220, 250) and the wrist joint (230, 350) are similarly connected. Therefore, it is determined that these three are connected by a skeletal structure. The key skeletal nodes refer to joints with high physiological priority and high node connectivity in a multi-branched structure. They are core nodes that support the overall skeletal shape of the human body and determine the trajectory of limb movements. For example, in a digital human skeleton, the shoulder joint node connects the neck, chest, and upper arm, with a node connectivity of 3. Its branch structure involves the main movements of the upper limbs, and its physiological priority is "high". Therefore, the shoulder joint at coordinates (220, 180) is located as a key skeletal node, which is crucial for upper limb posture recognition. The connection identifier refers to the code or symbol set to mark the connection relationship between key skeletal nodes. It is used to clarify the connection type, direction, and priority between nodes, so as to facilitate the computer to quickly identify and process the skeletal structure. For example, in the construction of a two-dimensional skeleton, "J01-J02-1" indicates that the shoulder joint (J01) and the elbow joint (J02) are first-level connections (direct bone connection), and "J02-J03-1" indicates that the elbow joint (J02) and the wrist joint (J03) are first-level connections. The identifiers clearly distinguish the connection attributes of different joints.

[0111] Furthermore, the querying of the coordinate set corresponding to the human joints can be achieved through 3D reconstruction methods, such as using the OpenPose BODY_25 model to output the (x, y, z) coordinate sequence of 25 joints, thereby obtaining a standardized 3D coordinate set; the analysis of the skeleton connection relationship in the multi-angle human images can be achieved through graph structure analysis methods, such as constructing a joint adjacency graph based on the NetworkX library and extracting topological connection edges, thereby obtaining a skeleton relationship table containing 19 sets of skeleton connections; the location of key skeleton nodes in the multi-angle human images can be achieved through a centrality sorting algorithm, such as using the PageRank algorithm to calculate the importance score of each joint in the kinetic chain, thereby obtaining 5 core key skeleton node indices; the parsing of the connection identifiers in the key skeleton nodes can be achieved through hierarchical encoding methods, such as using the limb connection rules defined in the COCO human dataset to generate joint parent-child relationship identifiers, thereby obtaining 17 sets of standardized skeleton connection identifiers; the generation of 2D skeleton data corresponding to the multi-angle human images can be achieved through multi-view fusion methods, such as integrating the detection results of various views through multi-camera triangulation of AlphaPose, thereby obtaining a unified 2D skeleton JSON data file.

[0112] As another embodiment of the present invention, the step of locating key skeleton nodes in the multi-angle human body image based on the skeleton connection relationship includes: analyzing the node connectivity degree corresponding to the skeleton connection relationship; determining highly connected points in the multi-angle human body image based on the node connectivity degree; identifying multi-branch structures in the multi-angle human body image based on the highly connected points; querying the physiological priority corresponding to the multi-branch structure; and locating key skeleton nodes in the multi-angle human body image based on the physiological priority.

[0113] The node connectivity refers to the number of joints directly connected to other joints in the skeleton's connection relationships. It is an indicator of the connection strength of a node in the skeleton structure. The higher the value, the more central the node is in the overall skeleton. For example, the hip joint node is connected to the waist node, the left thigh node, and the right thigh node, so its node connectivity is 3; while the elbow joint is only connected to the shoulder joint and the wrist joint, so its connectivity is 2. Obviously, the hip joint has a higher connectivity. The highly connected points refer to the locations of joints whose node connectivity exceeds a preset threshold. These points occupy a pivotal position in the skeleton structure and play a key role in maintaining the connection of multiple parts. For example, if the node connectivity threshold is set to 2, in the digital human skeleton, the neck node connects to the head, the left shoulder joint, and the right shoulder joint, so its connectivity is 3, exceeding the threshold. Therefore, the coordinate position (250, 100) of the neck is a highly connected point and is the core of the connection between the torso, upper limbs, and head. The multi-branch structure refers to... The skeletal substructures extending from highly connected points and branching in different directions represent the bifurcation of the human limbs. For example, the lumbar node (a highly connected point) connects upwards to the chest, downwards to the pelvis, and to the hip joints to the left and right, forming three branches: "chest-lumbar-pelvis," "lumbar-left hip joint," and "lumbar-right hip joint." These branches together constitute a multi-branched structure that supports the coordinated movement of the trunk and lower limbs. The physiological priority refers to the ranking of the importance of each node in the multi-branched structure based on the physiological structure and motor function of the human body. Nodes with higher priority are usually the core for maintaining basic posture or achieving key movements. For example, in the multi-branched structure of the hip joint, the branch connecting to the trunk (supporting the body's center of gravity) has a higher priority than the branch connecting to the lower limbs. Therefore, the physiological priority of the hip joint node is determined to be "high," while the branch where the toe joints are located has a "low" priority because it only affects subtle movements.

[0114] Furthermore, the analysis of node connectivity corresponding to the skeleton connection relationship can be achieved through a graph theory degree centrality algorithm, such as using the NetworkX library to calculate the number of adjacent edges of each node in the skeleton graph, thereby obtaining a quantified set of node connectivity values; the determination of highly connected points in the multi-angle human body image can be achieved through a threshold filtering method, such as using a percentile ranking method to select skeleton nodes with connectivity in the top 20%, thereby obtaining a list of highly connected point coordinates; the identification of multi-branch structures in the multi-angle human body image can be achieved through topological structure analysis methods, such as using Blender's skeletal system to detect joint clusters with three or more connection directions, thereby obtaining spatial distribution data of multi-branch structures; the query of the physiological priority corresponding to the multi-branch structure can be achieved through an anatomical weight model, such as assigning higher motion transmission weights to core nodes such as the spine and pelvis with reference to medical atlases, thereby obtaining a ranking of the physiological importance of branch structures; the location of key skeleton nodes in the multi-angle human body image can be achieved through kinematic feature extraction methods, such as using the deformable parameters of the SMPL model to identify pivot joints that affect the overall posture, thereby obtaining the indexes of the five key nodes with the greatest motion control.

[0115] Specifically, for a more intuitive understanding of the execution logic and data flow relationships corresponding to the two-dimensional skeleton data generation and application process in this solution, please refer to [reference needed]. Figure 3 .Should Figure 3 The 2D image-based human joint point inference graph, part of the digital human skeleton point recognition system, clearly presents the mapping link from multi-angle human images to 2D skeleton data: the left side focuses on the 2D skeleton data generated based on human joint points, using numbered nodes and colored lines to visualize the connection relationships of the human skeleton (e.g., 0 represents the head apex, 1 represents the neck, etc., with different colored lines corresponding to different limb bone connections); the right side displays multi-angle human images and the application of matched 2D skeleton data, combining the abstract skeleton with real human posture. Together, they present a complete process of "image acquisition → joint point recognition → skeleton construction → posture mapping," providing an intuitive reference for understanding how 2D skeleton data is generated based on human joint points and applied to posture analysis. It should be noted that the diagram is a simplified representation of the human skeleton-posture association logic. In real-world scenarios, the complexity of accurate joint point recognition (such as the impact of joint occlusion and morphological changes on recognition under different actions) and the diversity of skeleton adaptation to real humans (different body types and actions require dynamic adjustment of skeleton parameters) far exceed the diagram's scope. This architecture only provides a concise demonstration of the core logic to help understand the generation and application of 2D skeleton data in digital human skeleton point recognition.

[0116] S4. Determine the position mapping relationship of the two-dimensional skeleton data in three-dimensional space, and transform the three-dimensional joint coordinate set corresponding to the position mapping relationship. Based on the three-dimensional joint coordinate set, locate the preset SMPL model's pose registration point in the optimized point cloud model, and match the skeleton topology structure corresponding to the pose registration point.

[0117] This invention determines the positional mapping relationship of the two-dimensional skeleton data in three-dimensional space and transforms the three-dimensional joint coordinate set corresponding to the positional mapping relationship. This enables the planar skeleton information to be upgraded to three-dimensional space, providing accurate three-dimensional references for skeletal point measurement under different postures and providing complete spatial data support for automatic skeletal point recognition.

[0118] The three-dimensional space refers to a three-dimensional space composed of three coordinate axes: X, Y, and Z. It can accurately describe the position of an object in the three dimensions of length, width, and height, and is the basis for the three-dimensional positioning of digital human skeleton points. In this space, each point has unique three-dimensional coordinates, which can fully present the three-dimensional shape and spatial relationship of the object. For example, in the three-dimensional space of the digital human model, the ground is set as the XY plane, and the vertical direction upward is the Z axis. The coordinates of a certain bone point are (50cm, 30cm, 120cm), which means that the point is 50cm from the origin in the X direction, 30cm in the Y direction, and 120cm in the Z direction (height), clearly showing its spatial position. The position mapping relationship refers to the coordinate transformation rules between the joint points in the two-dimensional skeleton data and the corresponding joint points in the three-dimensional space. It is used to clarify the positional correspondence of the same joint point in the two-dimensional image and the three-dimensional space. It is the core of realizing the two-dimensional to three-dimensional conversion. For example, the coordinates of a certain elbow joint point in the two-dimensional image are (200 pixels, 300 pixels). Through the camera intrinsic parameters, extrinsic parameters and perspective projection principles, its corresponding coordinates in the three-dimensional space are (45cm, 25cm, 90cm). This transformation rule from (200, 300) to (45, 25, 90) is the position mapping relationship. The three-dimensional joint coordinate set refers to the set of all three-dimensional joint coordinates obtained after transforming all joints in the two-dimensional skeleton data through position mapping relationships. It contains the precise position information of each joint in three-dimensional space and is the key data for constructing the three-dimensional skeletal structure of a digital human. For example, the three-dimensional joint coordinate set of the entire digital human contains 24 joint coordinates, including (60cm, 35cm, 110cm) for the shoulder joint, (55cm, 40cm, 80cm) for the hip joint, (55cm, 40cm, 45cm) for the knee joint, etc. These coordinates together constitute the node network of bone connections in three-dimensional space. Optionally, the determination of the position mapping relationship of the two-dimensional skeleton data in three-dimensional space can be achieved by multi-view geometric reconstruction methods, such as: using Epipolar geometric constraints to calculate the basic matrix of two-dimensional joints under different viewpoints, thereby obtaining the precise position mapping relationship across viewpoints; the transformation of the three-dimensional joint coordinate set corresponding to the position mapping relationship can be achieved by triangulation algorithms, such as: using the Direct Linear Transform method to back-project the matched two-dimensional joints to three-dimensional space, thereby obtaining complete three-dimensional joint coordinate point cloud data.

[0119] Furthermore, based on the three-dimensional joint coordinate set, the present invention locates the pose registration points of the preset SMPL model in the optimized point cloud model. With the help of the predefined skeletal structure of the SMPL model, it can provide a standardized pose reference for the optimized point cloud, and can efficiently integrate the three-dimensional joint coordinates and point cloud data, improve the accuracy and consistency of digital human skeleton pose restoration, and help to efficiently generate a structured skeletal framework.

[0120] The preset SMPL model refers to a parametric 3D human template model. Using preset skeletal topology and skin vertex information, it can generate 3D human mesh models with different postures and body shapes based on a few key parameters (such as joint angles and body shape parameters). This model contains 23 major joints and approximately 6890 vertices, accurately simulating the natural movement patterns of the human body. For example, when parameters such as a shoulder joint rotation angle of 30° and an elbow joint bending angle of 90° are input, the SMPL model can generate an upper limb mesh with the corresponding posture. Its skeletal connections highly match the real human anatomy. Digital human skeleton point recognition provides a standardized pose reference; the pose registration point refers to the point selected from the candidate point set that has the smallest pose deviation index with the standard coordinate data of the SMPL model. It is the best matching point between the SMPL model and the optimized point cloud model at the joint position, used to achieve pose alignment between the two. For example, in the candidate point set of the digital human ankle joint, the pose deviation index of a certain point is 0.05 (the minimum value in the set), and its three-dimensional coordinates are (52cm, 45cm, 15cm). This point is determined as the pose registration point, so that the ankle joint of the SMPL model corresponds accurately with the ankle joint of the point cloud model.

[0121] As an embodiment of the present invention, the step of locating the attitude registration point of the preset SMPL model in the optimized point cloud model according to the three-dimensional joint coordinate set includes: calculating the joint position value corresponding to the three-dimensional joint coordinate set; querying the standard coordinate data corresponding to the preset SMPL model; identifying the candidate point set near each joint position in the optimized point cloud model based on the joint position value and the standard coordinate data; analyzing the attitude deviation index corresponding to the candidate point set; and locating the attitude registration point of the preset SMPL model in the optimized point cloud model based on the attitude deviation index.

[0122] The joint position value refers to an index calculated using a formula to comprehensively measure the spatial dispersion of the overall three-dimensional joint coordinate set. It reflects the deviation of joint points from the average position. The value is calculated based on the deviation of the three-dimensional coordinates of all joints from the mean coordinates. A larger value indicates a more dispersed spatial distribution of the joints. For example, if the coordinates of the upper limb joints (shoulder, elbow, and wrist) of a digital human are (0.5, 0.3, 1.2) m, (0.6, 0.4, 1.1) m, and (0.7, 0.5, 1.0) m respectively, the calculated value... The joint position value Gp = 0.12m reflects the dispersion of these joints relative to the average position, helping to determine whether the posture is extended or flexed. The standard coordinate data refers to the three-dimensional coordinate data of each joint point in the preset SMPL model under standard postures (such as standing, with arms hanging naturally). These are standardized reference values ​​obtained through statistical analysis of a large amount of human data and are used as a benchmark for joint position comparison. For example, the standard coordinate data of the shoulder joint of a standard adult male in the SMPL model is (55cm, 30cm, 115cm). This data is set based on human anatomical characteristics. The candidate point set can serve as a reference for the actual joint position values ​​to determine the difference between the actual posture and the standard posture. The candidate point set refers to the set of point cloud data located within a certain range around each joint position value in the optimized point cloud model. These points are potential points that can be matched with the joint points of the SMPL model and need to be further screened to determine the final registration points. For example, for the knee joint position values ​​(50cm, 40cm, 60cm), the search range is set to a spherical region with a radius of 3cm. This region contains 80 point cloud points, which together constitute the candidate point set. Each point can be a registration point corresponding to the knee joint of the SMPL model. The posture deviation index is a quantitative indicator used to measure the posture difference between each point in the candidate point set and the standard coordinate data of the SMPL model. It is obtained by calculating parameters such as spatial distance and angle deviation. The smaller the value, the closer the posture is to the standard and the higher the matching degree. For example, the spatial distance between a candidate point and the standard coordinate of the hip joint of the SMPL model is 1.2cm and the angle deviation is 5°. The posture deviation index calculated by the algorithm is 0.08, while the deviation index of another candidate point is 0.3. Obviously, the former is more suitable as a registration point.

[0123] Furthermore, the query for the standard coordinate data corresponding to the preset SMPL model can be achieved through the parameterized human model interface, such as: calling the get_J_regressor() method of the SMPL model to extract the standard joint regression matrix of 6890 vertices, thereby obtaining the standard joint coordinate data under the T-pose; the identification of the candidate point set near each joint position in the optimized point cloud model can be achieved through a spherical neighborhood search algorithm, such as: using the RadiusSearch function of the PCL library to establish a spherical search area with a radius of 10cm centered on the joint, thereby obtaining a subset of candidate point clouds around each joint; the analysis of the pose deviation index corresponding to the candidate point set can be achieved through the nearest neighbor distance statistical method, such as: calculating the mean Hausdorff distance from the candidate point cloud to the corresponding joint surface of the SMPL model, thereby obtaining the quantified local pose deviation index; the positioning of the preset SMPL model pose registration point in the optimized point cloud model can be achieved through a non-rigid ICP algorithm, such as: using the Coherent Point Drift method to iteratively optimize the correspondence between model vertices and point clouds, thereby obtaining the optimal set of model pose registration point coordinates.

[0124] In another embodiment of the present invention, calculating the joint position values ​​corresponding to the three-dimensional joint coordinate set includes:

[0125]

[0126] Where Gp represents the joint position value (unit: m) corresponding to the three-dimensional joint coordinate set, M represents the total number of joints, j represents the joint index, and x represents the joint position value (unit: m). j The x-coordinate (in meters) and y-coordinate of the j-th joint in three-dimensional space are represented by the following values. j The z-coordinate represents the y-coordinate (in meters) of the j-th joint in three-dimensional space. j μ represents the z-coordinate (in meters) of the j-th joint in three-dimensional space. x μ represents the average x-coordinate of all joints (unit: m). y μ represents the average y-coordinate of all joints (unit: m). z This represents the average z-coordinate of all joints (unit: m).

[0127] Specifically, the x, y, and z coordinates refer to the specific coordinate values ​​of the j-th joint on the x-axis (e.g., left-right direction), y-axis (e.g., front-back direction), and z-axis (e.g., vertical direction) in three-dimensional space, respectively, used to accurately locate the spatial position of the joint. For example, the x-coordinate of the j-th joint (elbow joint) is... j = 0.4m (meaning 0.4 meters from the origin in the left and right directions), y j =0.2m (0.2 meters from the origin in both forward and backward directions), zj =0.1m (1.0 meter vertically from the origin), the three coordinates together determine the unique position of the joint in space; the average x-coordinate, average y-coordinate, and average z-coordinate refer to the arithmetic mean of the coordinates of all joints on the x, y, and z axes, reflecting the center position of the joint group in each axis. For example, if the x-coordinates of the 5 joints are 0.3m, 0.4m, 0.5m, 0.6m, and 0.7m respectively, then μ x = (0.3 + 0.4 + 0.5 + 0.6 + 0.7) ÷ 5 = 0.5 m; similarly, μ can be calculated. y μ z It can be used as a reference benchmark to calculate joint position values ​​and measure the deviation of a single joint from the center of the overall joint group.

[0128] This invention, by matching the skeletal topology corresponding to the posture registration points, can leverage the standardized skeletal associations of the SMPL model to provide a clear skeletal connection logic for optimizing point clouds. By precisely aligning the topological relationships, it strengthens the structural consistency between 3D joint coordinates and point cloud data, thereby improving the accuracy of skeletal posture reconstruction and promoting the efficient conversion from point cloud data to structured skeletal models.

[0129] The skeletal topology refers to a structured connection framework built based on stable reference positions, where each posture registration point is constructed according to the connection rules of the human skeleton (such as joint subordination and bone orientation). It reflects the overall layout and association logic of the skeleton. For example, in the generated digital human lower limb skeletal topology, the hip joint is connected to the knee joint through a stable reference position, and the knee joint is then connected to the ankle joint. At the same time, the length ratio of each bone (thigh:lower leg = 1:0.9) and the connection angle range (the knee joint can bend from 0° to 150°) are marked, fully presenting the association relationship of the lower limb skeleton.

[0130] As an embodiment of the present invention, matching the skeletal topology corresponding to the attitude registration point includes: parsing the spatial position information corresponding to the attitude registration point; identifying the geometric intervals corresponding to key positioning intervals in the position information; filtering the effective connection portions in the geometric intervals; marking the stable reference positions corresponding to the effective connection portions; and matching the skeletal topology corresponding to the attitude registration point based on the stable reference positions.

[0131] The spatial location information refers to the precise coordinates and distribution characteristics of the posture registration points in three-dimensional space, including the X, Y, and Z axis coordinates of each registration point and the relative positional relationships between points. This is the fundamental data for analyzing the spatial distribution of the skeleton. For example, the spatial location information of the waist posture registration point in a digital human is coordinates (55cm, 40cm, 85cm), which differs from the hip joint registration point (55cm, 40cm, 70cm) by 15cm in the Z-axis direction, and forms a specific spatial angle with the shoulder registration point (60cm, 35cm, 110cm). This information collectively describes the position of the waist within the skeletal structure. The key positioning points refer to the poses that play a core positioning role within the skeletal structure. The region between attitude registration points, typically the range between registration points of adjacent joints or important bone endpoints, is a key interval for analyzing skeletal connectivity. For example, the regions between shoulder joint registration points (60cm, 35cm, 110cm) and elbow joint registration points (45cm, 30cm, 90cm), and between elbow and wrist joint registration points (35cm, 25cm, 80cm), are both key positioning intervals, corresponding to the skeletal connectivity intervals of the upper arm and forearm, respectively. The geometric interval refers to the geometric distance and angular difference between attitude registration points in three-dimensional space. It is an indicator for quantifying bone length, inter-joint distance, and spatial angles, reflecting the physical morphological characteristics of the bones. The geometric interval between the hip joint registration points (55cm, 40cm, 70cm) and the knee joint registration points (55cm, 40cm, 40cm) is a straight-line distance of 30cm (corresponding to the length of the thigh bone). The geometric interval between the knee and ankle joint registration points (55cm, 40cm, 10cm) is 30cm, and the two form a 120° bending angle at the knee joint. The effective connection portion refers to the part within the geometric interval that conforms to the laws of human anatomy and can truly reflect the skeletal connection relationship. Invalid intervals caused by noise or errors are eliminated, which is an effective basis for constructing the skeletal topology. For example, in the geometric interval between the shoulder and elbow joints, there is a section that deviates from the correct direction due to point cloud noise. After screening, the intervals (5cm in length) of the normal skeletal orientation are retained as effective connection parts that conform to the natural orientation of the upper arm bones, and are used for skeletal connection construction. The stable reference position refers to the feature point position in the effective connection part that is relatively fixed and not easily affected by posture changes. It is usually the midpoint of the bone or the reference point of the joint connection, which serves as the anchor point for skeletal topology matching. For example, in the effective connection part of the upper arm (25cm in length), the point (52.5cm, 32.5cm, 100cm) 12.5cm away from the shoulder joint is the midpoint of the upper arm. This position is relatively stable when the arm is bent or extended and is marked as a stable reference position for aligning the skeletal topology.

[0132] Furthermore, the parsing of the spatial position information corresponding to the attitude registration points can be achieved through three-dimensional coordinate transformation methods, such as using the Eigen library to calculate the rotation and translation matrix of the registration point set relative to the world coordinate system, thereby obtaining spatial pose information containing position and orientation; the identification of the geometric intervals corresponding to key locations in the position information can be achieved through geodesic distance algorithms, such as using the Dijkstra shortest path method of the VTK library to calculate the shortest surface distance between skeleton nodes, thereby obtaining accurate anatomical geometric interval data; the filtering of effective connection parts in the geometric intervals can be achieved through kinematic constraint detection methods, such as based on Biomechanics. The toolkit verifies whether the joint spacing is within the range allowed by human physiology, thereby obtaining effective connection segments that conform to anatomical norms. The stable reference positions corresponding to the effective connection parts can be marked by rigid body stability analysis methods, such as detecting the static characteristics of inertial reference frame nodes such as the pelvis and spine using ROS TF trees, thereby obtaining a set of stable reference point markers for global positioning. The matching of the skeletal topology corresponding to the attitude registration points can be achieved by graph neural network methods, such as using GraphCNN to learn the topological features of the SMPL skeleton graph and aligning it with the point cloud registration results, thereby obtaining a skeletal connection relationship map that conforms to anatomical rules.

[0133] Specifically, for a more intuitive understanding of the execution logic and data flow relationships corresponding to the bone size measurement and calibration process in this solution, please refer to [link / reference]. Figure 4 .Should Figure 4 As a digital human skeletal point recognition system, the anthropometric diagram clearly presents the complete link from skeletal topology construction to size measurement output: the input layer relies on the constructed skeletal topology (the joints and connections marked on the model) to provide a basic reference for size measurement; the processing layer transforms the spatial relationships of the skeleton into quantified size data through a step-by-step logic of "locating posture registration points → matching skeletal topology → calculating size measurement values ​​→ data calibration"; the output layer uses "size parameters of various parts of the human body (such as height 178.14cm, head circumference 56.59cm, etc.)" as the final result. It should be noted that the diagram is a simplified representation of the skeletal size association logic. In real-world scenarios, the complexity of accurate size calculation (such as dynamic adjustments to circumference measurements under different body postures) and the diversity of calibration adaptation (different application scenarios have different requirements for size accuracy) far exceed the diagram's representation. This architecture only provides a concise display of the core logic, offering an intuitive reference for understanding the systematic approach to skeletal size measurement and calibration.

[0134] S5. Based on the skeletal topology, determine the size measurement values ​​corresponding to the skeletal points of the digital human in the target scene, and perform data calibration on the size measurement values ​​to generate automatic recognition results corresponding to the skeletal points of the digital human in the target scene.

[0135] Based on the aforementioned skeletal topology, this invention determines the dimensional measurements of the skeletal points of the digital human in the target scene. By leveraging standardized skeletal connections, it provides a clear structural reference for dimensional measurements, ensuring that the measured values ​​are consistent with the anatomical patterns of the real human body. Simultaneously, it provides a quantitative data foundation for applications such as digital human posture analysis and motion simulation, improving the accuracy and practicality of digital human technology in scenario applications.

[0136] The size measurement value refers to the actual size of the bone (such as length and diameter) determined based on the three-dimensional distance value, combined with the morphology of the bone segment (such as thickness and curvature) and anatomical standards (such as the reference range of bone diameter). For example, the three-dimensional distance value of the thigh bone segment is calculated to be 210mm. Then, the diameter at the midpoint of the bone segment is statistically analyzed through point cloud data. Combined with the human bone size standard, the length measurement value of the thigh bone of the digital human is determined to be 210mm and the diameter measurement value is 30mm, which is used as the quantitative result of the bone size of the digital human.

[0137] As an embodiment of the present invention, determining the size measurement value corresponding to the digital human skeleton point in the target scene based on the skeleton topology includes: analyzing the joint connection relationship corresponding to the skeleton topology; determining the bone segment region of the digital human skeleton point in three-dimensional space based on the joint connection relationship; locating the joint coordinate point in the bone segment region; querying the three-dimensional distance value corresponding to the joint coordinate point; and determining the size measurement value corresponding to the digital human skeleton point in the target scene based on the three-dimensional distance value.

[0138] The joint connection relationships refer to the connection logic between different joints in the skeletal topology, formed according to human anatomy. This clarifies how joints are interconnected through bones, including connection order, angles, and support relationships. For example, the joint connection relationships of the digital human's upper limb are: shoulder joint → elbow joint → wrist joint. The shoulder joint can rotate in multiple directions and is connected to the elbow joint via the humerus. The elbow joint's bending angle ranges from approximately 0° to 150°. These rules constitute the upper limb joint connection relationships and are the basis for measuring bone dimensions. The bone segment region refers to the spatial region of the bone defined by adjacent joints in three-dimensional space, based on the joint connection relationships. This corresponds to a single bone segment in human anatomy. For example, based on the connection relationship between the shoulder joint (coordinates: X1, Y1, Z1) and the elbow joint (coordinates: X2, Y2, Z2), a spatial region is defined between them. This region corresponds to the upper arm bone segment, and its shape is affected by joint position and human posture. A near-cylindrical bone segment region can be delineated using point cloud data. The joint coordinate points are used to locate the measurement points for bone dimensions. The joint coordinate points refer to the precise coordinate values ​​of the joints at both ends of the bone segment in three-dimensional space. They are key reference points for determining the spatial position and length of the bone segment. For example, the joint coordinate points of the knee joint in a digital human are (300, 400, 500) mm, and those of the hip joint are (300, 400, 700) mm. These coordinates accurately mark the joint positions. By calculating the relationship between the two points, the length, direction, and other information of the bone segment can be derived, providing a coordinate reference for dimension measurement. The three-dimensional distance value refers to the straight-line distance between the joints at both ends of the bone segment or the curved distance value that conforms to the curvature of the bone, calculated using the joint coordinate points and the spatial distance formula. It reflects the spatial length of the bone segment. Taking the knee and hip joints as examples, if the straight-line distance calculated by the formula is 200 mm, but because the thigh bone has a natural curvature, the actual three-dimensional distance value along the bone shape is calculated as 210 mm after curve fitting. This value is closer to the actual bone length and is used to accurately measure bone dimensions.

[0139] Furthermore, the analysis of the joint connection relationships corresponding to the skeletal topology can be achieved using graph theory adjacency matrix methods, such as: using the NetworkX library to construct the adjacency matrix of the skeleton graph and extracting the connection edge attributes, thereby obtaining a topology data table containing 24 sets of joint connection relationships; the determination of the bone segment regions of the digital human skeleton points in three-dimensional space can be achieved using the axial bounding box method, such as: using the AABB algorithm of the PCL library to generate a minimum bounding cube for each bone segment, thereby obtaining a spatial region division of 15 bone segments; the location of the joint coordinate points in the bone segment regions can be achieved using centroid calculation methods, such as: using... Open3D's KDTree query for the density center of the bone segment point cloud yields the precise 3D coordinates of 28 joints. The 3D distance values ​​corresponding to these joint coordinates can be obtained using a Euclidean distance algorithm, such as calling NumPy's `linalg.norm` function to calculate the linear distance between adjacent joints, resulting in 42 sets of 3D measurements of joint spacing. The determination of the dimensions corresponding to the skeletal points in the target scene can be achieved through a scaling method, such as proportional scaling based on the standard height parameters of the SMPL model, to obtain millimeter-level precision in bone length measurements.

[0140] This invention generates automatic recognition results corresponding to the skeletal points of the digital human in the target scene by calibrating the size measurement values. This eliminates systematic errors and environmental interference in the measurement process, ensuring that the recognition results highly match the real skeletal features. It can unify the measurement standards in different scenarios, enhance the consistency and reliability of the recognition results, and lay a solid foundation for the large-scale application of digital human-related technologies.

[0141] The automatic recognition result refers to a structured data set automatically output by the system after data calibration of the dimensional measurements. This set includes the location, size, and topological relationships of the skeletal points of the digital human. It accurately reflects the spatial distribution and morphological characteristics of the skeleton. The result covers key information such as the three-dimensional coordinates of each skeletal point, the length of the bone segment, and the joint angle. It can be directly used for digital human posture analysis or model construction. For example, the automatic recognition result for the lower limbs of the digital human includes the coordinates of the hip joint (55cm, 40cm, 70cm), the coordinates of the knee joint (55cm, 40cm, 40cm), the femur length measurement of 45cm, the knee flexion angle of 110°, and the lower... The connection topology of the limb skeleton provides complete data support for subsequent motion simulation. Optionally, the data calibration of the size measurement values ​​can be achieved through a least squares optimization method, such as using the Levenberg-Marquardt algorithm to fit the residual curve between the measurement values ​​and the standard bone length, thereby obtaining a calibrated set of accurate size parameters. The automatic recognition results corresponding to the digital human skeleton points in the target scene can be achieved through a deep learning detection network, such as using MediaPipe's BlazePose model to predict 33 skeletal key points in the image in real time, thereby obtaining an automatic recognition result JSON file with confidence.

[0142] Compared to the problems described in the background technology, this invention reduces reliance on dedicated point cloud equipment and lowers hardware costs and usage barriers by identifying the RGB lens array corresponding to the digital human in the target scene and acquiring omnidirectional images corresponding to the RGB lens array. Simultaneously, the omnidirectional images provide rich multi-angle visual information for subsequent 3D reconstruction, improving the integrity and accuracy of point cloud data and laying an efficient foundation for the entire skeletal point recognition process. This invention also accurately locates interfering data by detecting noise distribution areas in the image point cloud data and analyzing the corresponding filtering parameters, providing a targeted basis for noise reduction and avoiding indiscriminate processing that damages effective information. By performing operations with adapted filtering parameters, the quality of the point cloud model can be effectively optimized, facilitating the efficient construction of accurate skeletal topology. Furthermore, this invention, by acquiring multi-angle human images corresponding to the optimized point cloud model and identifying human joints in the multi-angle human images, can capture data from multiple perspectives. This invention captures key limb information of a digital human, providing rich feature anchors for skeletal structure analysis. By accurately locating joints, it establishes a correlation between two-dimensional image information and three-dimensional point cloud data, laying the foundation for subsequent skeleton data generation. Furthermore, by determining the positional mapping relationship of the two-dimensional skeleton data in three-dimensional space and transforming the corresponding three-dimensional joint coordinate set, the planar skeleton information can be upgraded to three-dimensional space. This provides accurate three-dimensional references for skeletal point measurement under different postures and complete spatial data support for automatic skeletal point recognition. Finally, based on the skeletal topology, this invention determines the dimensional measurement values ​​corresponding to the skeletal points of the digital human in the target scene. By leveraging standardized skeletal connection relationships, it provides clear structural references for dimensional measurement, ensuring that the measured values ​​are consistent with the anatomical laws of the real human body. Simultaneously, it provides a quantitative data foundation for applications such as digital human posture analysis and motion simulation, improving the accuracy and practicality of digital human technology in scenario applications. Therefore, the automatic skeletal point recognition method and system provided by this invention can improve the accuracy, efficiency, and automation of digital human skeletal point recognition.

[0143] like Figure 5 The diagram shown is a functional block diagram of an automatic skeletal point recognition system for digital humans according to the present invention.

[0144] The skeletal point automatic recognition system 200 for digital humans described in this invention can be installed in an electronic device. Depending on the functions implemented, the skeletal point automatic recognition system for digital humans may include a data generation module 201, a model construction module 202, a skeleton data module 203, a structure matching module 204, and a result generation module 205. The modules described in this invention can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0145] In this embodiment of the invention, the functions of each module / unit are as follows:

[0146] The data generation module 201 is used to identify the RGB lens array corresponding to the digital human in the target scene, and to collect the omnidirectional shooting image corresponding to the RGB lens array, and to perform three-dimensional reconstruction on the omnidirectional shooting image to generate image point cloud data corresponding to the omnidirectional shooting image.

[0147] The model building module 202 is used to detect the noise distribution area in the image point cloud data, analyze the filtering parameters corresponding to the noise distribution area, and perform noise reduction and smoothing operation on the filtering parameters to build an optimized point cloud model corresponding to the image point cloud data.

[0148] The skeleton data module 203 is used to collect multi-angle human images corresponding to the optimized point cloud model, identify human joints in the multi-angle human images, and generate two-dimensional skeleton data corresponding to the multi-angle human images based on the human joints.

[0149] The structure matching module 204 is used to determine the position mapping relationship of the two-dimensional skeleton data in three-dimensional space, and convert the three-dimensional joint coordinate set corresponding to the position mapping relationship. Based on the three-dimensional joint coordinate set, it locates the preset SMPL model's pose registration point in the optimized point cloud model and matches the skeleton topology structure corresponding to the pose registration point.

[0150] The result generation module 205 is used to determine the size measurement values ​​corresponding to the skeletal points of the digital human in the target scene based on the skeletal topology, and to perform data calibration on the size measurement values ​​to generate automatic recognition results corresponding to the skeletal points of the digital human in the target scene.

[0151] In detail, the modules in the automatic skeletal point recognition system 200 for realizing digital humans described in this embodiment of the invention employ the same methods as described above. Figure 1 This method uses the same technique as the one described above for automatically recognizing skeletal points in a digital human, and can produce the same technical effect, so it will not be elaborated here.

[0152] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. In the above multiple embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for automatically recognizing skeletal points in a digital human, characterized in that, The method includes: Identify the RGB lens array corresponding to the digital human in the target scene, and acquire the omnidirectional images corresponding to the RGB lens array. Perform three-dimensional reconstruction on the omnidirectional images to generate image point cloud data corresponding to the omnidirectional images. The noise distribution area in the image point cloud data is detected, and the filtering parameters corresponding to the noise distribution area are analyzed. A noise reduction and smoothing operation is performed on the filtering parameters to construct an optimized point cloud model corresponding to the image point cloud data. Collect multi-angle human images corresponding to the optimized point cloud model, identify human joints in the multi-angle human images, and generate two-dimensional skeleton data corresponding to the multi-angle human images based on the human joints. Determine the position mapping relationship of the two-dimensional skeleton data in three-dimensional space, and transform the three-dimensional joint coordinate set corresponding to the position mapping relationship. Based on the three-dimensional joint coordinate set, locate the preset SMPL model's pose registration point in the optimized point cloud model, and match the skeleton topology structure corresponding to the pose registration point. Based on the skeletal topology, the size measurement values ​​corresponding to the skeletal points of the digital human in the target scene are determined, and the size measurement values ​​are calibrated to generate automatic recognition results corresponding to the skeletal points of the digital human in the target scene.

2. The method for automatic skeletal point recognition of a digital human as described in claim 1, characterized in that, The step of performing three-dimensional reconstruction on the omnidirectional captured image to generate image point cloud data corresponding to the omnidirectional captured image includes: Query the feature descriptors in the omnidirectional captured image; Based on the feature descriptor, the inter-image matching degree corresponding to the omnidirectional captured image is calculated using the following formula. Among them, S I,J N represents the image matching degree between the omnidirectional images I and J. I N represents the total number of feature points in image I. J Let J represent the total number of feature points in image J, L represent the number of valid matching point pairs verified by geometry, and k represent the index of the number of feature point pairs. Let represent the descriptor vector of the k-th matching point pair in image I. Let σ represent the descriptor vector of the k-th matching point pair in image J, and σ represent the similarity bandwidth parameter. Analyze the positional mapping relationship corresponding to the inter-graph matching degree; Based on the position mapping relationship, the three-dimensional image point set corresponding to the all-around captured image is divided; Mark the key feature points in the three-dimensional image point set; Based on the key feature points, image point cloud data corresponding to the omnidirectional captured image is generated.

3. The method for automatic skeletal point recognition of a digital human as described in claim 1, characterized in that, The step of performing noise reduction and smoothing operations on the filtering parameters to construct an optimized point cloud model corresponding to the image point cloud data includes: Identify the noise characteristic level corresponding to the filtering parameters; Based on the noise characteristic level, high-density noise blocks in the noise distribution area are selected; Activate the adaptive filter corresponding to the high-density noise block; Based on the adaptive filter, noise reduction and smoothing are performed on the filtering parameters to obtain a set of residual noise points; Based on the residual noise set, an optimized point cloud model corresponding to the image point cloud data is constructed.

4. The method for automatic skeletal point recognition of a digital human as described in claim 3, characterized in that, The step of filtering high-density noise blocks in the noise distribution area based on the noise characteristic level includes: Set the density determination threshold corresponding to the noise feature level; Based on the density determination threshold, the noise distribution area is divided into regional grid units; Statistically calculate the noise density value within the grid cells of the region; Generate a noise density map corresponding to the noise density value; Based on the noise density map, high-density noise blocks are selected in the noise distribution area.

5. The method for automatic skeletal point recognition of a digital human as described in claim 1, characterized in that, The step of generating two-dimensional skeleton data corresponding to the multi-angle human body image based on the human body joints includes: Query the set of location coordinates corresponding to the human body joint points; Based on the set of position coordinates, analyze the skeleton connection relationships in the multi-angle human body images; Based on the skeleton connection relationship, key skeleton nodes in the multi-angle human body image are located; Parse the connection identifiers in the key skeleton nodes; Based on the connection identifier, two-dimensional skeleton data corresponding to the multi-angle human body image is generated.

6. The method for automatic skeletal point recognition of a digital human as described in claim 5, characterized in that, The step of locating key skeleton nodes in the multi-angle human body image based on the skeleton connection relationship includes: Analyze the node connectivity corresponding to the skeleton connection relationship; Based on the node connectivity, highly connected points in the multi-angle human body image are determined; Based on the highly connected points, identify the multi-branch structure in the multi-angle human body image; Query the physiological priority corresponding to the multi-branch structure; Based on the physiological priority, key skeletal nodes in the multi-angle human body image are located.

7. The method for automatic skeletal point recognition of a digital human as described in claim 1, characterized in that, The step of locating the preset pose registration points of the SMPL model in the optimized point cloud model based on the three-dimensional joint coordinate set includes: The joint position values ​​corresponding to the three-dimensional joint coordinate set are calculated using the following formula: Where Gp represents the joint position value corresponding to the three-dimensional joint coordinate set, M represents the total number of joints, j represents the joint index, and x represents the joint position value. j Let x and y represent the x-coordinate of the j-th joint in three-dimensional space. j Let y be the y-coordinate of the j-th joint in three-dimensional space, and z be the z-coordinate of the j-th joint. j Let μ represent the z-coordinate of the j-th joint in three-dimensional space. x μ represents the average x-coordinate of all joints. y μ represents the average y-coordinate of all joints. z This represents the average z-coordinate of all joints; Query the standard coordinate data corresponding to the preset SMPL model; Based on the joint position values ​​and standard coordinate data, identify the candidate point set near each joint position in the optimized point cloud model; Analyze the attitude deviation index corresponding to the candidate point set; Based on the attitude deviation index, the preset attitude registration point of the SMPL model in the optimized point cloud model is located.

8. The method for automatic skeletal point recognition of a digital human as described in claim 1, characterized in that, The matching of the skeleton topology corresponding to the pose registration point includes: Analyze the spatial position information corresponding to the attitude registration point; Identify the geometric intervals corresponding to key locations in the location information; Filter the valid connection portions in the geometric interval; Mark the stable reference position corresponding to the effective connection portion; Based on the stable reference position, the skeletal topology corresponding to the attitude registration point is matched.

9. The method for automatic skeletal point recognition of a digital human as described in claim 1, characterized in that, The step of determining the size measurement values ​​corresponding to the skeletal points of the digital human in the target scene based on the skeletal topology includes: Analyze the joint connection relationships corresponding to the skeletal topology; Based on the joint connection relationship, the bone segment region of the digital human skeleton point in three-dimensional space is determined; Locate the joint coordinate points in the bone segment region; Query the three-dimensional distance value corresponding to the joint coordinate point; Based on the three-dimensional distance value, the size measurement value corresponding to the digital human skeleton point in the target scene is determined.

10. A system for automatically recognizing skeletal points in a digital human, characterized in that, The system includes: The data generation module is used to identify the RGB lens array corresponding to the digital human in the target scene, and to collect the omnidirectional shooting images corresponding to the RGB lens array, and to perform three-dimensional reconstruction on the omnidirectional shooting images to generate image point cloud data corresponding to the omnidirectional shooting images. The model building module is used to detect the noise distribution area in the image point cloud data, analyze the filtering parameters corresponding to the noise distribution area, and perform noise reduction and smoothing operations on the filtering parameters to build an optimized point cloud model corresponding to the image point cloud data. The skeleton data module is used to collect multi-angle human images corresponding to the optimized point cloud model, identify human joints in the multi-angle human images, and generate two-dimensional skeleton data corresponding to the multi-angle human images based on the human joints. The structure matching module is used to determine the position mapping relationship of the two-dimensional skeleton data in three-dimensional space, and convert the three-dimensional joint coordinate set corresponding to the position mapping relationship. Based on the three-dimensional joint coordinate set, the module locates the preset SMPL model's pose registration point in the optimized point cloud model and matches the skeleton topology structure corresponding to the pose registration point. The result generation module is used to determine the size measurement values ​​corresponding to the skeletal points of the digital human in the target scene based on the skeletal topology, and to perform data calibration on the size measurement values ​​in order to generate automatic recognition results corresponding to the skeletal points of the digital human in the target scene.

Images