Tree model-based graph convolution regression grouping method and system

Abstract

The application relates to the technical field of human posture estimation, and particularly provides a graph convolution regression grouping method and system based on a tree model, which comprises the following steps: a bidirectional mapping relationship between nodes of each convolution layer in a graph convolution module and key point nodes of a corresponding level in an H-Keypoint tree diagram model is established, so that a structured key point tree topology feature map is obtained; a class attribution confidence and a spatial position offset of each key point are output; an abnormal key point is re-distributed to a correct group, so that a corrected key point grouping result is obtained; a non-maximum suppression treatment is performed on the corrected key point grouping result, so that overlapping posture candidate results are suppressed; the grouping result subjected to the non-maximum suppression treatment is taken as a grouping connection output, so that a complete human posture skeleton is formed; and joint training is performed, so that a trained graph convolution regression grouping network based on the tree model is obtained. The application significantly improves the robustness and accuracy of human posture estimation in a complex scene.

Description

Technical Field

[0001] This invention relates to the field of human pose estimation technology, and in particular to a graph convolutional regression grouping method and system based on a tree model. Background Technology

[0002] Human pose estimation aims to locate key points of the human body and recover the complete skeletal structure from images, and has important applications in fields such as behavior recognition and human-computer interaction. Bottom-up methods have attracted widespread attention because their computational complexity does not increase linearly with the number of people. The core process is as follows: first, detect all candidate key points of people in the image, and then assign each key point to the corresponding person instance through a grouping strategy.

[0003] Currently, mainstream grouping strategies mainly include methods based on Part Affinity Fields (PAFs) and methods based on embedded vector clustering. However, existing methods still have the following technical shortcomings when facing complex real-world scenarios: First, there is an error in the association of key points across characters. In scenarios with dense interaction among multiple people, body parts of different characters often overlap. Existing grouping methods mostly rely on local or paired features, lacking effective modeling of global context information and coupling relationships between different characters. This makes it difficult for the network to distinguish the attribution of key points of adjacent characters, and it is very easy to incorrectly connect the key points of different characters, resulting in hybrid postures.

[0004] Second, the geometric constraints within the same person are not fully utilized. When a person is in a complex pose, such as self-occlusion or unconventional movements, the image often lacks clear limb detail cues. Existing methods fail to explicitly and robustly extract and utilize the inherent geometric prior information of the human body, such as skeletal length proportions and joint angle constraints, which leads to connection errors of key points within the same person when visual evidence is lacking.

[0005] Third, existing graph model methods lack topological adaptability. Some studies have attempted to introduce graph models to model the relationships between key points, but the graph structures they construct are mostly fully connected graphs or fixed neighborhood graphs, which are difficult to flexibly adapt to the dynamic changes in the human body's topology under different postures, and the computational cost is high on dense graphs.

[0006] To address the aforementioned issues, this invention provides a tree-based graph convolutional regression grouping method. By constructing a tree topology model that conforms to human kinematics and designing a dedicated graph convolutional regression network, it performs joint regression on the category classification and spatial location relationship of key points from a global perspective, thereby improving the grouping accuracy in dense and complex scenes. Summary of the Invention

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In one aspect, the present invention provides a graph convolutional regression grouping method based on a tree model, comprising the following steps: All candidate keypoints and their corresponding feature vectors output by the keypoint detection network in the target image are obtained. The candidate keypoints are then input into a pre-constructed H-Keypoint tree graph model. The H-Keypoint tree graph model divides the candidate keypoints into a multi-level parent-child node structure according to human kinematic constraints. The parent node represents a limb unit and is used to characterize the spatial attachment relationship between adjacent joints. According to the parent-child node structure, each candidate keypoint is mapped to the corresponding node position in the H-Keypoint tree graph model. The inner, outer, and connecting layer nodes of each convolutional layer in the graph convolution module are then linked to the keypoint nodes at the corresponding levels in the H-Keypoint tree graph model to obtain a structured keypoint tree topology feature map. The keypoint tree topology feature map is input into the grouping graph convolution module. The grouping graph convolution module performs feature regression on each node in the keypoint tree topology feature map, and outputs the category confidence and spatial position offset of each keypoint. Group label detection is performed on the regression results. When an abnormal grouping of keypoints is detected, the out-of-group detection mechanism is invoked. Based on the parent-child dependency relationship of each node in the keypoint tree topology feature map and the feature similarity of adjacent nodes, the abnormal keypoints are reassigned to the correct group, and the corrected keypoint grouping results are obtained. Through the inter-layer information transfer of the graph convolution module, the feature dependency correlation between adjacent level nodes is strengthened to ensure the spatial consistency of the connection relationship of each keypoint within the same group. Non-maximum suppression is applied to the corrected keypoint grouping results to suppress overlapping pose candidate results, retaining the candidate pose with the highest detection score as the final predicted pose. The grouping results after non-maximum suppression are used as the group connection output to form a complete human pose skeleton. The grouped graph convolutional module and the front-end keypoint detection network are jointly trained through end-to-end backpropagation to update the node mapping parameters of the H-Keypoint tree graph model and the network weights of the grouped graph convolutional module until the model converges, resulting in a trained tree-based graph convolutional regression grouping network used for end-to-end human pose estimation of the input image.

[0008] Another aspect of the present invention provides a graph convolutional regression grouping system based on a tree model, comprising: The feature map acquisition module is used to acquire all candidate keypoints and their corresponding feature vectors output by the keypoint detection network in the target image, and input the candidate keypoints into the pre-constructed H-Keypoint tree graph model. The H-Keypoint tree graph model divides the candidate keypoints into a multi-level parent-child node structure according to human kinematic constraints. Among them, the parent node represents a limb unit and is used to represent the spatial attachment relationship between adjacent joints. According to the parent-child node structure, each candidate keypoint is mapped to the corresponding node position of the H-Keypoint tree graph model, and the inner layer nodes, outer layer nodes and connecting layer nodes of each convolutional layer in the graph convolution module are respectively established with the keypoint nodes of the corresponding level in the H-Keypoint tree graph model to obtain a structured keypoint tree topology feature map. The result correction module is used to input the keypoint tree topology feature map into the grouping graph convolution module. The grouping graph convolution module performs feature regression on each node in the keypoint tree topology feature map, and outputs the category confidence and spatial position offset of each keypoint. The regression results are then subjected to group label detection. When an abnormal grouping of keypoints is detected, the out-of-group detection mechanism is invoked. Based on the parent-child dependency relationship of each node in the keypoint tree topology feature map and the feature similarity of adjacent nodes, the abnormal keypoints are reassigned to the correct group, resulting in the corrected keypoint grouping results. Through the inter-layer information transfer of the graph convolution module, the feature dependency correlation between adjacent level nodes is strengthened to ensure the spatial consistency of the connection relationship of each keypoint within the same group. The non-maximum suppression processing module is used to perform non-maximum suppression processing on the corrected keypoint grouping results, suppressing overlapping pose candidate results and retaining the candidate pose with the highest detection score as the final predicted pose. The grouping results after non-maximum suppression processing are used as group connection outputs to form a complete human pose skeleton. The grouped graph convolution module and the front-end keypoint detection network are jointly trained through end-to-end backpropagation to update the node mapping parameters of the H-Keypoint tree graph model and the network weights of the grouped graph convolution module until the model converges, resulting in a trained tree-based graph convolutional regression grouping network used for end-to-end human pose estimation of the input image.

[0009] This invention uses a tree model structure to reorganize the key point feature information of the human body and defines group labels to standardize the grouping rules of key points of different people, thus solving the problem of incorrect grouping of key points of different people. A grouped graph convolution module is proposed, which internally formulates a three-layer human body key point structure based on the dependency and correlation of adjacent key points, solving the problem of incorrect connection of key points within the same group. Attached Figure Description

[0010] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the graph convolutional regression grouping method based on a tree model provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the graph convolutional regression grouping method based on a tree model provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the key point tree diagram analysis of the COCO dataset provided in Embodiment 2 of the present invention; Figure 4 This is the human body grouping label association diagram provided in Embodiment 2 of the present invention; Figure 5 This is the information propagation distribution diagram of the three layers of the grouped graph convolutional module provided in Embodiment 4 of the present invention; Figure 6 The diagram illustrates different data structures provided in Embodiment 4 of the present invention; (a) in the diagram represents a Euclidean data structure; and (b) represents a non-Euclidean data structure. Figure 7 This is a block diagram of the tree-based graph convolutional regression grouping system provided in Embodiment 6 of the present invention; Figure 8 The FF-DEKR network structure diagram with improved key point grouping and loss function provided by this invention; Figure 9 A block diagram of the electronic device provided by the present invention; Figure 10 A block diagram of a computer-readable storage medium provided for this invention.

[0011] Reference numerals in the attached figures: 1. Feature map acquisition module; 2. Result correction module; 3. Nonmaximum suppression processing module; 4. Central processing unit / microprocessor / main control chip; 5. Storage medium; 6. Data bus; 7. Input / output bus / external bus / device bus; 8. Display; 9. Input / output device; 10. Computer-readable instructions; 11. Non-transitory computer-readable storage medium. Detailed Implementation

[0012] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0013] Hereinafter, the terms "first," "second," etc., are used for descriptive convenience only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0014] In this invention, unless otherwise explicitly specified and limited, the term "connection" should be interpreted broadly. For example, "connection" can be a fixed mechanical connection, a detachable mechanical connection, or an integral part; or, "connection" can be a direct connection or an indirect connection through an intermediate medium. Furthermore, unless otherwise explicitly specified and limited, the term "coupling" should be interpreted broadly. For example, "coupling" can be a direct electrical connection, such as physical contact and electrical conduction between two components; it can also be understood as an electrical connection between different components in a circuit structure through physical lines capable of transmitting electrical signals, such as copper foil or wires on a printed circuit board (PCB), to transmit electrical signals; or, "coupling" can be an indirect electrical connection between two components through an intermediate medium; or, "coupling" can be an electrical connection between two components in a non-contact manner, such as an electrical connection between two components using capacitive coupling to transmit electrical signals.

[0015] In this embodiment of the invention, directional terms such as "up," "down," "left," and "right" may be defined relative to the orientation of the components shown in the accompanying drawings. It should be understood that these directional terms can be relative concepts, used for relative description and clarification, and can change accordingly depending on the orientation of the components in the accompanying drawings.

[0016] Example 1: As Figure 1 As shown, this embodiment of the invention provides a graph convolutional regression grouping method based on a tree model, comprising the following steps: Step S100: Obtain all candidate keypoints and their corresponding feature vectors output by the keypoint detection network in the target image, and input the candidate keypoints into the pre-constructed H-Keypoint tree graph model; the H-Keypoint tree graph model divides the candidate keypoints into a multi-level parent-child node structure according to human kinematic constraints; where the parent node represents a limb unit and is used to represent the spatial attachment relationship between adjacent joints; according to the parent-child node structure, map each candidate keypoint to the corresponding node position of the H-Keypoint tree graph model, and establish a bidirectional mapping relationship between the inner layer nodes, outer layer nodes and connecting layer nodes of each convolutional layer in the graph convolution module and the keypoint nodes of the corresponding level in the H-Keypoint tree graph model, respectively, to obtain a structured keypoint tree topology feature map; Step S200: Input the keypoint tree topology feature map into the grouping graph convolution module. The grouping graph convolution module performs feature regression on each node in the keypoint tree topology feature map, outputting the category confidence and spatial position offset of each keypoint. Perform group label detection on the regression results. When an abnormal grouping of keypoints is detected, call the out-of-group detection mechanism. Based on the parent-child dependency relationship of each node in the keypoint tree topology feature map and the feature similarity of adjacent nodes, the abnormal keypoints are reassigned to the correct group, resulting in the corrected keypoint grouping results. Through the inter-layer information transfer of the graph convolution module, the feature dependency correlation between adjacent level nodes is strengthened to ensure the spatial consistency of the connection relationship of each keypoint within the same group. Step S300: Perform non-maximum suppression processing on the corrected keypoint grouping results to suppress overlapping pose candidate results and retain the candidate pose with the highest detection score as the final predicted pose; use the grouping results after non-maximum suppression processing as the group connection output to form a complete human pose skeleton; jointly train the grouped graph convolutional module and the front-end keypoint detection network through end-to-end backpropagation, update the node mapping parameters of the H-Keypoint tree graph model and the network weights of the grouped graph convolutional module until the model converges, and obtain the trained tree-based graph convolutional regression grouping network, which is used for end-to-end human pose estimation of the input image.

[0017] The target image is the original input image for human pose estimation, in RGB or grayscale format, containing at least one instance of a person to be detected, serving as the input data source for the entire pose estimation network. The keypoint detection network is a deep convolutional neural network used to detect the coordinates of human joints in the target image, outputting all possible candidate locations of human keypoints in the image, along with their corresponding confidence scores and feature descriptor vectors. Candidate keypoints are the set of all pixel locations in the target image that may belong to human joints, detected by the keypoint detection network, including keypoints belonging to different individuals and potential false positives. Each candidate keypoint contains its coordinates and a confidence score. The feature vector is a deep feature descriptor associated with each candidate keypoint, extracted from the image by the keypoint detection network, used to characterize the appearance semantic information of the keypoint and its contextual relationship with surrounding pixels. Human kinematic constraints are the inherent biomechanical limitations of the human skeletal structure, including but not limited to the connection relationships between adjacent joints, the proportional range of limb lengths, and the limits of joint range of motion, used to construct a tree-like topology model that conforms to the natural structure of the human body. The multi-level parent-child node structure organizes the human skeleton into a tree-like topology, following a hierarchical relationship from the center of the torso to the extremities. Parent nodes represent joints close to the torso, and child nodes represent joints farther from the torso, forming a hierarchical node organization. A limb unit is a bone segment composed of a pair of adjacent joints, corresponding to a parent-child node edge in the tree diagram model, representing the rigid connection between adjacent joints. Spatial dependency relationships are the relative spatial constraints between keypoints of parent and child nodes, including relative distance and orientation information. The graph convolution module is a feature extraction unit constructed using a graph convolutional neural network, used to aggregate neighborhood information and update features in a non-Euclidean space on the keypoint tree topology feature map. The inner, outer, and connecting layer nodes of the convolutional layers are as follows: inner nodes refer to deep semantic keypoints near the root node or the center of the torso in the tree topology; outer nodes refer to detailed keypoints near the extremities or limb endpoints in the tree topology; connecting layer nodes refer to intermediate keypoints located between the inner and outer layers, used to transfer and fuse information between different levels. The bidirectional mapping relationship is a two-way correspondence index relationship established between the convolutional nodes at each level in the graph convolutional module and the corresponding keypoint nodes at the corresponding level in the H-Keypoint tree graph model. The structured keypoint tree topology feature map is graph-structured data obtained by organizing the feature vectors of candidate keypoints according to the parent-child structure of the tree graph model, including node feature matrices and adjacency matrices. The grouped graph convolutional module is a graph convolutional network component used to perform keypoint feature regression tasks. It takes the keypoint tree topology feature map as input and outputs the grouped prediction results of each node through multi-layer graph convolution operations. The feature regression task is the prediction task performed by the grouped graph convolutional module for each input node, calculating the probability distribution of the node belonging to various person instances and its spatial coordinate offset through regression.The category assignment confidence score is the probability value predicted by the grouped graph convolutional module for each candidate keypoint to belong to each person instance, used to determine which person group the keypoint should be assigned to. The spatial position offset is the coordinate difference between the detected position of the candidate keypoint and the actual joint position, obtained by regression from the grouped graph convolutional module, used to correct the localization accuracy of the keypoint. Group label detection is the process of verifying the consistency of the keypoint assignment labels output by the grouped graph convolutional module, used to identify keypoints with anomalous assignments. Anomalous keypoint assignment occurs when a candidate keypoint is assigned to a person group it should not belong to, including cross-person connection errors and isolated keypoints. The out-of-group detection mechanism is a correction strategy triggered when an anomalous keypoint assignment is detected. It iterates through the set of candidate keypoints outside the current group and calculates the correct group for the anomalous keypoint based on feature similarity. Feature similarity is a measure of the similarity between the feature vectors of two keypoints, calculated using cosine similarity or Euclidean distance, used to determine the degree of matching between a keypoint and different groups. A group is the set of all keypoints contained in the same person instance, with each group corresponding to an independent person in the image. Inter-layer information transfer refers to the feature propagation process between adjacent convolutional layers in the graph convolution module. Lower-layer nodes pass aggregated neighborhood information to upper-layer nodes, achieving multi-scale context fusion. Feature dependency correlation refers to the mutual influence relationship between features of adjacent nodes in the tree topology. Information transfer through graph convolution enhances the feature coupling between adjacent keypoints. Spatial consistency ensures that the pose skeleton formed by connecting keypoints within the same group satisfies human kinematic constraints in terms of geometric structure, without any abnormal connections that violate human structure. Non-maximum suppression is a post-processing operation that filters multiple overlapping pose candidate results within the same group, suppressing redundant candidates with low confidence. Overlapping pose candidate results are multiple pose assumptions that may arise for the same person instance in an image, with highly overlapping keypoint positions and connections. The highest detection score is the pose result with the highest confidence retained among multiple overlapping pose candidates after non-maximum suppression. The human pose skeleton is a structured output composed of detected keypoints and their connections, presented in the form of a skeleton graph. Each node represents a keypoint, and each edge represents a limb connection between adjacent keypoints. The front-end keypoint detection network is the keypoint detection network component located before the grouped graph convolutional module. End-to-end backpropagation involves propagating the output loss of the grouped graph convolutional module layer by layer forward to the front-end keypoint detection network. Joint training involves synchronously updating the parameters of the front-end keypoint detection network and the grouped graph convolutional module as a single network. Node mapping parameters are parameterized representations of the bidirectional mapping relationship between keypoint nodes in the H-Keypoint tree model and convolutional layer nodes in the graph convolutional module. Network weights are learnable parameter matrices for each convolutional and fully connected layer in the graph convolutional module. Model convergence is defined as the loss function value stabilizing during network training and no longer significantly decreasing over multiple iterations.

[0018] The principles described in the above embodiments are referenced in the appendix. Figure 2 The H-Keypoint tree graph model in this embodiment utilizes human kinematic constraints to divide keypoints into multi-level parent-child node structures, forming limb unit representations and clarifying joint spatial dependencies. By mapping candidate keypoints to corresponding node positions and establishing bidirectional mapping relationships with the inner, outer, and connecting layer nodes of the graph convolution module, a structured keypoint tree topological feature map is generated, effectively modeling the hierarchical spatial dependency characteristics of human posture. The grouped graph convolution module performs feature regression on the tree topological feature map, outputting keypoint category confidence and spatial offset. Group label detection, combined with an out-of-group detection mechanism, dynamically corrects abnormal keypoint groupings based on parent-child dependencies and the feature similarity of adjacent nodes, ensuring the topological rationality of the grouping results. Inter-layer information transmission strengthens the feature dependency correlation between adjacent level nodes, ensuring the spatial consistency of keypoint connection relationships within the same group. Non-maximum suppression is used to filter overlapping pose candidates, and the highest detection score is retained as the final predicted pose to output a complete human skeleton. An end-to-end joint training strategy optimizes the H-Keypoint tree node mapping parameters and the weights of the grouped graph convolution module to improve the end-to-end estimation accuracy of the human pose.

[0019] In summary, this embodiment significantly improves the robustness and accuracy of human pose estimation in complex scenarios through tree-structured representation, dynamic grouping correction, and spatial consistency constraints.

[0020] The DEKR network utilizes adaptive convolution to learn keypoint features, concentrating the activation range of each pixel around the keypoint rather than limiting it to its neighborhood; this makes the keypoint regression location more accurate. Simultaneously, the network employs a multi-branch structure, establishing an independent branch for each keypoint, allowing detected keypoints to regress to their respective locations based on their feature information. Finally, the network designs virtual center points, the number of which matches the number of people in the image. The coordinates of the virtual center points are the average coordinates of all keypoints for each person. The dendrogram model is a machine learning model based on data structures. Its principle is based on progressively segmenting the input image into different subsets, ultimately constructing a decision tree or regression tree for prediction or classification. Tree models have intuitive interpretability and good generalization ability.

[0021] Example 2: Based on Example 1, the H-Keypoint tree diagram model constructed in step S100 of this embodiment of the invention specifically includes a three-layer structure, with each layer representing data of different properties. Taking the keypoints of the COCO2017 dataset as an example for analysis and explanation, the H-Keypoint tree diagram model establishes an independent tree structure diagram for each person in the image; the entire tree diagram is divided into three layers, named as: ancestor node, parent node, and child node; Ancestor nodes are used to distinguish different individual instances. Each ancestor node embeds feature information from an individual reference label; details will be provided in the next section. The second layer, parent nodes, divides the 17 keypoints into five groups based on human geometry: head unit (label 1), left upper limb unit (label 2), right upper limb unit (label 3), left lower limb unit (label 4), and right lower limb unit (label 5). Each parent node constrains the connections between limbs within a unit, thus uniting keypoints within the same range and preventing omissions. The third layer consists of child nodes, which expand upon the parent nodes, extending the seventeen keypoints to each unit. Child nodes contain keypoint label information and keypoint regression information for each keypoint, encoded separately in the tree structure according to the keypoint's characteristics. For example, the head unit includes left eye-right eye-nose-left ear-right ear, and the right lower limb unit includes right hip-right knee-right ankle. The analysis of the human body grouping tree diagram is as follows... Figure 3 As shown.

[0022] The final human body grouping label association diagram is as follows: Figure 4 As shown, analysis of the tree diagram reveals that: ancestor nodes are responsible for distinguishing the identities of different people in the image, preventing errors in keypoint detection between individuals; parent nodes are responsible for dividing the 17 types of keypoints into five groups based on prior skeletal features, strengthening the collaborative information between keypoints after grouping, and checking whether any keypoints in the group are missing, such as left shoulder-left elbow-left wrist. This helps the model better understand the collaborative movements between keypoints, improves its ability to understand complex movements, simplifies the learning model, and enhances the consistency of limb movements; child nodes are branches of each parent node, strengthening the pose coherence between adjacent child nodes. Parent nodes and child nodes at the same level are considered sibling nodes, which strengthen the internal correlation features of keypoints.

[0023] The H-Keypoint tree diagram model is an independent tree structure built for each person instance, used to organize and manage the hierarchical relationships of key points in the human body. It clearly defines the model's topological structure, node hierarchy rules, and their mapping relationship with the graph convolution module. Ancestor nodes are the first-level nodes of the H-Keypoint tree diagram model, used to distinguish different person instances. Each ancestor node embeds individual reference label feature information to identify the person corresponding to the tree structure. Parent nodes are the second-level nodes of the H-Keypoint tree diagram model, used to divide multiple key points into limb unit groups based on human geometry. Each parent node corresponds to a limb unit group, used to constrain the connection relationships of key points within that group and check for missing key points. Child nodes are the third-level nodes of the H-Keypoint tree diagram model, extending from the parent nodes, containing key point label information and key point regression information for each key point, encoded on the tree structure according to the different categories of key points. The limb units are functional groups formed by dividing seventeen key points based on prior skeletal features of the human body. Specifically, they include five limb units: head, left upper limb, right upper limb, left lower limb, and right lower limb. Each limb unit is constrained and managed by its corresponding parent node. Sibling nodes refer to the relationships between parent nodes at the same level and among child nodes under the same parent node. Information transmission between sibling nodes strengthens the correlation features within key points. Individual reference labels are identity information embedded in each ancestor node, used to distinguish different person instances and prevent confusion of key points between different persons during grouping. Key point label information identifies the category attribute of the key point, such as belonging to a specific body part like the nose, left shoulder, or right knee. Key point regression information represents the spatial coordinates of the key point in the image and related regression parameters.

[0024] In the above embodiments, the graph convolutional regression grouping method based on the tree model achieves technical optimization of multi-dimensional human keypoint detection through the three-layer structure of the H-Keypoint tree graph model. The embedding of individual reference label features of ancestor nodes solves the identity confusion problem in multi-person scenarios. Instance-level feature differentiation ensures that the keypoint regression results of different individuals do not interfere with each other, improving the robustness of the model in dense crowds. The parent node adopts a five-unit grouping mechanism based on human geometric prior partitioning, clustering the seventeen keypoints according to anatomical structure to form five structured representations: head unit, left and right upper and lower limb units. Grouping constraints through graph convolutional layers significantly enhance the spatial correlation learning of keypoints within the same limb, while unit-level integrity checks prevent the omission of local keypoints. For example, the shoulder-elbow-wrist node group of the left upper limb unit strengthens biomechanical constraints through edge connections of graph convolution. The hierarchical expansion structure of child nodes retains the grouping advantages of parent nodes and achieves fine-grained pose coherence modeling through graph convolutional message passing between adjacent child nodes. Feature interactions between sibling nodes at the same level further capture the collaborative motion patterns of cross-regional keypoints, such as the linkage features between head units and upper limb units in waving gestures. The hierarchical graph convolutional architecture of the tree diagram transforms the original flat keypoint detection problem into structured feature learning. Through layer-by-layer feature refinement of ancestor-parent-child nodes, it achieves multi-scale feature fusion from instance recognition to local limbs and then to single-point regression. Compared with traditional unstructured keypoint detection networks, it improves computational efficiency while reducing the false detection rate under complex poses.

[0025] Example 3: Based on Example 1, the process of grouping and labeling key points in the H-Keypoint tree diagram in step S100 of this embodiment of the invention specifically includes the following steps: Step S101: Obtain the original grouping results of each keypoint output by the grouped graph convolution module and the keypoint heatmap output by the keypoint detection network. For each keypoint heatmap, define and output the keypoint heatmap. Figure 1 A corresponding group reference label; Step S102: Assign values ​​to the group reference labels according to the current grouping results, assigning similar label values ​​to each key point belonging to the same person; obtain the individual reference label for each person in the image, and take the average of the group reference labels of all key points of the same person to obtain the individual reference label for the person; the number of key points participating in the mean calculation is based on the actual number of key points detected for that person and participates in the normalization calculation to obtain the individual reference label set for each person. Step S103: Perform dual detection on the original grouping results using group reference labels and individual reference labels; for each grouped person, calculate the mean square error between the group reference label and the individual reference label of each key point within the person, and determine whether there is a grouping error in the key points within the group based on the comparison result of the mean square error and the preset threshold, and obtain the detection result within the group. Step S104: When there are key points with grouping errors, calculate the normal function value between the person with the key point with grouping errors and the individual reference labels of other people in the image. Based on the normal function value, determine whether there is a key point misassignment relationship between other people and the person with the key point with the error. Then, reassign the key point with the error to the group of people with the smallest difference from its individual reference label to obtain the corrected key point grouping label result.

[0026] Among them, the grouping reference label refers to the label defined and output for each keypoint heatmap during the keypoint grouping process, and its relation to the keypoint heatmap. Figure 1A keypoint is assigned a corresponding label value. Group reference labels are assigned values ​​based on the current grouping results, representing the person affiliation information of each keypoint. Their numerical distribution reflects the degree of association between the keypoint and each person's group. Individual reference labels are identity-defining labels defined for each person in the image. Individual reference labels are obtained by averaging the group reference labels of all keypoints for the same person, representing the overall label feature center of the person and serving as a reference benchmark for subsequent intra-group and out-of-group detection. Keypoint heatmaps are heatmaps output by the keypoint detection network, used to display the spatial distribution of all keypoints of the same category in the image. Each heatmap corresponds to a keypoint category, and all keypoint heatmaps output by the network together constitute the complete location information of all candidate keypoints in the image. The original grouping results are the keypoint person affiliation assignment results directly output after feature regression of each keypoint by the grouping map convolution module, without label detection correction. Normalized calculation refers to the process of averaging the grouped reference labels of all keypoints for the same person to obtain individual reference labels. The denominator in the mean calculation is based on the actual number of keypoints detected for that person, rather than a standardized set of seventeen keypoints. This accommodates situations where the number of detected keypoints varies among different people due to occlusion or cropping. Mean squared error (MSE) refers to the mean squared error calculated between the grouped reference labels of each keypoint within a grouped person and the individual reference label of that person. The magnitude of this MSE value is used to quantify whether each keypoint belongs to the current person's group: if the MSE value is less than a preset threshold, the keypoint is considered to belong to that person; if the MSE value is greater than or equal to the preset threshold, the keypoint is considered to have a grouping error. The preset threshold is the benchmark value used to determine the correctness of keypoint grouping; it refers to a pre-set MSE comparison threshold used to compare with the calculated MSE value to determine whether a keypoint belongs to the current person's group. The normal function value refers to the value calculated using a normal function on the individual reference labels of two different people during out-of-group detection. The normal function value is used to quantitatively measure the degree of difference in individual reference labels between two groups of people to determine the possibility of misassignment of keypoints between the two groups. The criterion for determining the final assignment of erroneous keypoints by minimizing the difference in individual reference labels means that when a keypoint with a grouping error exists, it is compared with the individual reference labels of each person in the image, and the group of people with the smallest absolute difference in their individual reference labels is selected as the correct assignment of the erroneous keypoint. The intra-group detection result refers to the set of information output regarding whether there are grouping errors among the keypoints within a group, after comparing the mean square error between the grouping reference labels of each keypoint within a group and the individual reference label of that person with a preset threshold.The keypoint misassignment relationship is used to describe the concept of grouping error state, which refers to the mapping state where at least one keypoint is assigned to the wrong character group between two different character groups. The state is identified by the normal function value between the reference labels of the two characters being less than a predetermined judgment criterion.

[0027] In the above embodiments, this invention addresses the grouping errors of key points for different individuals. Error detection is performed on the grouping results. Baseline grouping methods are prone to widespread key point grouping errors. By introducing label information based on the original method, key points belonging to the same person are embedded with the same label. This label is then embedded into the grandparent node of the tree model structure to perform error correction checks on the grouping results, ensuring that key points for different individuals are correctly grouped. The purpose of key point grouping labels is to add a new method to check for grouping errors after the regression grouping results are output, and to provide a solution—that is, to revert the key point to the correct individual, thus resolving the incorrect grouping in image grouping visualization. When performing key point detection, the DEKR network outputs 17 key point heatmaps and one virtual center point heatmap. The key point heatmap displays all key points in the image sequentially. For example, the first heatmap outputs the nose key points of all individuals in the image, the second heatmap outputs the left eye key points of all individuals in the image, and so on. Simultaneously with outputting the heatmap key points, grouping reference labels for the key points are defined and output. , Labels for key points; group label heatmap and key point heatmap Figure 1 The labels here are defined according to the grouping results; simply assign similar labels to key points belonging to the same person.

[0028] Under the existing grouping, define the following for each individual: Individual Reference Tag , This refers to the number of people in the image; individual reference labels. It is achieved by grouping all key points of the same person using reference tags. The average value is calculated using the following formula: in This refers to the number of times key points of a person appear in the image. Since it's impossible for everyone to have 17 key points—for example, only the upper body might appear—this embodiment defines... Make individual reference labels The computational normalization, It refers to the first Personal number is The true coordinates of the key points; Grouping key points using reference labels and individual reference labels Once the results are prepared, two types of tests are performed on the existing grouping results: within-group test and out-of-group test. 1. In-group testing refers to checking the key points within each person in the already grouped results to verify whether there are any grouping errors in the key points within the group; the specific operation method is to first calculate the grouping reference labels for all key points of this person. and individual reference labels mean square error The specific formula is as follows: Mean square error The smaller the value, the closer the key point label is to the individual reference label within the group, which proves that the key point belongs to this person. If the mean square error is large, it proves that the key point does not belong to this person. 2. Out-of-group detection is used to detect individual reference labels among different people in images of multiple people's pose estimation. The relationship between these two values ​​is calculated using a normal function. Based on this relationship, it is determined whether there is a possibility of misgrouping of key points between the two individuals. The correct group is then identified for erroneous key points in the in-group detection results, and regression is performed. The specific formula is as follows: in and These are the individual reference labels for two characters. The greater the difference between the average values ​​of the individual reference labels of any two groups, the more it proves that there are no key points of grouping errors between the two characters. The smaller the difference between the average values ​​of the individual reference labels of any two groups, the more it proves that there are key points of grouping errors between the two characters. Analysis shows that this embodiment specifies that a point is defined as an erroneous grouping point only when the mean square error of the in-group detection has a large deviation. Then, the network finds the original group of the erroneous grouping key point based on the out-of-group detection. This setting method can greatly avoid the waste of computing resources. First, the existing grouping results are screened for errors. After the screening results show key points with large deviations, the correct position of the erroneous grouping key point is found through out-of-group detection.

[0029] In summary, the H-Keypoint tree diagram contributes to keypoint grouping in two ways: firstly, it proposes using a tree diagram to group and classify human keypoints, emphasizing the consistency of limb units; secondly, it proposes new grouping rules, screening for incorrect groupings based on existing methods, and providing effective solutions after identifying incorrectly grouped keypoints. The H-Keypoint tree diagram helps strengthen synergy, consistency, and coherence, improves the model's understanding and modeling ability of human poses, and solves the problem of incorrect keypoint grouping.

[0030] Example 4: Based on Example 1, the grouped graph convolution module in step S200 of this embodiment specifically includes modeling the relationships between key points to enhance the correlation and dependency between adjacent key points; defining key points as nodes of the grouped graph convolution module, including second-order information of the skeletal data: the relative position and angle of the bones. The connections between key points serve as edges of the grouped graph convolution module; the grouped graph convolution module can automatically match the number of modules according to the number of characters; a three-layer structure is designed for the grouped graph convolution module based on the connection method of the human skeleton: The outer key points are the terminal key points of the head and limb units: left wrist, right wrist, left ankle, right ankle, left eye, right eye, left ear, and right ear; The inner key points are the key points connecting the inner edges of the four limb units of the human body: nose, left shoulder, right shoulder, left hip, and right hip; there are many key points on the edge of the head, so the nose is chosen as the central inner key point here. The key points in the connecting layer are those between the outer key points and the inner key points: left elbow, right elbow, left knee, right knee; the specific module layer distribution is as follows: Figure 5 As shown.

[0031] The grouped graph convolution module updates the representation of each node by aggregating the neighbor information of each keypoint. Nodes can obtain connection information through their connected upstream and downstream nodes, thereby strengthening the correlation and dependency between adjacent keypoints. This module chooses to use weight sharing to ensure that the influence of adjacent keypoints on the target node is consistent. The output of the graph convolution module is combined with the original keypoint features to obtain a richer representation. Using the graph convolution module to strengthen the correlation between keypoints can model nonlinear keypoints on a global structure, which is very important for tasks involving the overall body structure or global actions, because it enables the model to better understand and predict the overall keypoint layout, thereby improving the model's expressive power. At the same time, the grouped graph convolution module has a certain flexibility and can adapt to different keypoint correlation patterns. By adjusting the graph construction method, it can adapt to the differences in the correlation between keypoints in different scenarios. Information transfer and aggregation between keypoints helps to eliminate some local errors and improve the accuracy of keypoint grouping. The introduction of the grouped graph convolution module makes keypoint correlation learning an end-to-end process, rather than simply treating correlation as a post-processing step. The model gradually aggregates and propagates node information through multi-layered grouped graph convolutional modules, thereby achieving feature learning for the entire graph. In each layer, node features consider both their own characteristics and those of their neighbors. By learning layer-by-layer representations, the network can progressively extract higher-level graph features. The grouped graph convolutional module consists of three parts: Semantic Graph Convolution (SemGConv) + BN + ReLU. The key idea of ​​Semantic Graph Convolution is to learn the channel weights of the implicit prior edges in the graph and combine them with kernel matrices, significantly improving the capabilities of graph convolution.

[0032] To address the issue of incorrect keypoint connections within the same character, a grouped graph convolutional module based on the DEKR network is proposed. The module has a three-layer structure and embeds second-order skeletal information to strengthen the correlation between adjacent keypoints. By using the edge information of the detected keypoints to find pixels with strong correlation, the missed or incorrect keypoints can be inferred in reverse, thereby improving the accuracy of keypoint grouping and connection within the same character.

[0033] like Figure 6 The diagram illustrates Euclidean and non-Euclidean data structures. Typical neural networks can only process regular Euclidean data and cannot effectively handle irregular non-Euclidean data. The structure of non-Euclidean data is highly irregular; the surrounding structure of each node may be unique, with different neighboring nodes and boundaries. Keypoints in human pose estimation fall into the category of non-Euclidean data. However, graph convolution can process non-Euclidean data by treating it as a problem of node connections and message propagation in a graph.

[0034] This embodiment introduces second-order information from skeletal data into the grouped graph convolution module. The second-order information of skeletal data is described in detail below. In pose estimation tasks, second-order information from skeletal data refers to the information in addition to the coordinates of keypoints, including relative positions, angles, velocities, and accelerations between keypoints. This information can more richly describe complex human postures and movements. The second-order information used primarily focuses on relative positions and angles.

[0035] Relative positions can be calculated using the Euclidean distance formula between two adjacent keypoints. The label is defined as follows: Key points The coordinates are , No. Key points The coordinates are Then the relative positions of the two key points can be represented as : In practical applications, common relative positional relationships can be made between adjacent key points, such as shoulder and elbow, hip and knee, knee and ankle.

[0036] The angle of second-order skeletal information can be calculated using the dot product formula of vectors, which represents the angle formed by three adjacent keypoints. Human keypoints are divided into limb units, each containing exactly three adjacent keypoints. Taking the left upper limb unit as an example, three points are defined: shoulder, elbow, and wrist. , and Their coordinates are respectively , and The angle formed by these three key points is: in , Representing vectors respectively and Typically, the angles of a set of limb units are analyzed. Normally, the angles of a set of limb units are between 0° and 180°. By analyzing the relative positions and angles, we can gain a more comprehensive understanding of human posture and movement. This information can be used to design and optimize posture estimation models, thereby improving the performance and robustness of the models.

[0037] Example 5: Based on Example 1, the process of performing nonmaximum suppression processing on the corrected keypoint grouping results in step S300 of this embodiment of the invention specifically includes the following steps: Step S301: Obtain the corrected key point grouping results. Based on the spatial coordinates of the key points in each group in the key point grouping results, combine all key points in each group according to the parent-child connection relationship of the tree topology. Generate the initial pose candidate results corresponding to the group, and obtain a pose candidate set composed of multiple initial pose candidate results. Step S302: For each initial pose candidate result in the pose candidate set, calculate the comprehensive detection score of the initial pose candidate result based on the category classification confidence of each key point contained in the initial pose candidate result; and sort all the initial pose candidate results in the pose candidate set in descending order of comprehensive detection score to obtain the sorted pose candidate sequence. Step S303: Select the initial attitude candidate result with the highest comprehensive detection score from the sorted attitude candidate sequence as the benchmark candidate result, calculate the key point spatial position overlap between the remaining initial attitude candidate results and the benchmark candidate result in the attitude candidate sequence; mark the initial attitude candidate results with key point spatial position overlap exceeding the preset overlap threshold as overlapping attitude candidate results and remove them from the attitude candidate sequence. After this round of screening, a preliminary screened attitude candidate sequence is obtained. Step S304: Replace the initial screening candidate sequence with the initial screening candidate sequence, and repeat the screening operation until there are no other initial posture candidate results in the initial screening candidate sequence whose spatial position overlap with the key point of the current benchmark candidate result exceeds the preset overlap threshold; take all the initial posture candidate results that are finally retained after repeated screening as the output result after nonmaximum suppression processing, and obtain the screening posture candidate set used to form the final human posture skeleton.

[0038] The keypoint spatial coordinates refer to the two-dimensional or three-dimensional spatial position information of each keypoint in the target image, output by the grouped graph convolution module in step S200. These coordinates are expressed in pixel coordinates or normalized coordinates and are used to characterize the specific position of each keypoint in the image plane or space, serving as the basis for subsequent calculations of the positional overlap between pose candidate results. The tree topology is a multi-level node organization defined by the H-Keypoint tree diagram model constructed in step S100, including a three-layer structure of ancestor nodes, parent nodes, and child nodes. Each node is connected by parent-child edges according to human kinematic constraints, forming a graph structure with hierarchical progression and acyclic properties. This structure is used to regulate the connection order and dependency relationships between keypoints within the same group. The parent-child connection relationship refers to the rigid connection edge defined by limb units between the parent node keypoint and the child node keypoint in the tree topology. Each parent-child connection edge corresponds to a bone segment in the human skeleton and is used to combine adjacent keypoints with spatial dependencies within the same group into a complete pose skeleton in the order from the center of the torso to the extremities. The pose candidate set refers to the set of initial pose candidate results corresponding to each group in the corrected keypoint grouping results output in step S200. Each initial pose candidate result corresponds to a detected person instance in the image, and the number of candidate results in the set is equal to the number of people detected after grouping correction. The category attribution confidence score refers to the probability value predicted by the grouped map convolution module in step S200 for each candidate keypoint to belong to each person instance. It serves as a quantitative indicator of the reliability of the keypoint grouping results and is used to calculate the comprehensive detection score of each initial pose candidate result in non-maximum suppression processing. The comprehensive detection score refers to a single quantitative score obtained by weighting or summing the category attribution confidence scores of all keypoints within the same initial pose candidate result. It is used to characterize the overall detection reliability of the pose candidate results and serves as the basis for prioritizing each candidate result during sorting and filtering. The keypoint spatial overlap refers to the degree of positional overlap between two initial pose candidate results in image space. It is calculated based on the distance metric between the spatial coordinates of corresponding keypoints of the same category in the two candidate results and is used to quantitatively determine whether two pose candidate results point to the same person instance. The preset overlap threshold refers to a pre-set threshold value for judging the spatial overlap of key points. It is used to compare with the calculated spatial overlap of key points to determine whether two pose candidate results constitute an overlapping pose candidate relationship and to decide whether to remove the candidate results with lower confidence. The filtered pose candidate set refers to the set of all initial pose candidate results that are finally retained after repeated filtering in step S304. There are no cases where the spatial overlap of key points between the pose candidate results in the set exceeds the preset overlap threshold, ensuring that only the pose candidate result with the highest comprehensive detection score is retained for the same person instance, which is used to form the final human pose skeleton.

[0039] In the above embodiments, the non-maximum suppression process of this embodiment optimizes the grouping results of the corrected keypoints through multi-stage screening, ensuring that the final output set of pose candidates achieves optimal spatial arrangement and confidence. Step S301 combines keypoints within a group into initial pose candidates that conform to human kinematic constraints by using the spatial coordinates of keypoints and the parent-child connection relationship of the tree topology; using the hierarchical connection relationship generated in the grouping correction stage, it ensures that each candidate pose has a physically reasonable joint topology, providing a geometric consistency basis for subsequent overlap calculation. Step S302 calculates the comprehensive detection score based on the confidence of keypoint category attribution, and establishes a priority queue of candidate poses by sorting; the point-by-point confidence of the graph convolutional regression output is transformed into an overall pose reliability metric, so that subsequent suppression operations can prioritize the retention of anatomically reasonable poses with high confidence. Steps S303 to S304 adopt an iterative overlap detection mechanism, using the spatial overlap of keypoints as the judgment criterion, and gradually eliminates low-quality candidates that have spatial conflicts with high-scoring benchmark candidates. By dynamically comparing preset overlap thresholds, redundant pose outputs caused by local keypoint detection errors or grouping anomalies are suppressed; the repeated screening process ensures that each pose in the final candidate set has significant distinguishability in the tree topology space and meets the preset confidence and geometric independence requirements.

[0040] In summary, this embodiment is deeply coupled with the tree-based graph convolutional regression grouping method: it relies on the hierarchical key point connection relationship generated by the grouping correction module to realize the physical rational construction of candidate poses; it uses the confidence and spatial offset output by graph convolutional regression to evaluate the quality of candidates; and finally outputs a set of selected pose candidates that conforms to human kinematic constraints and has the optimal spatial distribution, providing a high-precision skeleton connection basis for end-to-end pose estimation.

[0041] Example 6: As Figure 7 As shown, based on Examples 1-5, the tree-model-based graph convolutional regression grouping system provided in this embodiment of the invention includes: Feature map acquisition module 1 is used to acquire all candidate keypoints and their corresponding feature vectors output by the keypoint detection network in the target image, and input the candidate keypoints into the pre-constructed H-Keypoint tree graph model. The H-Keypoint tree graph model divides the candidate keypoints into a multi-level parent-child node structure according to human kinematic constraints. Among them, the parent node represents a limb unit and is used to represent the spatial attachment relationship between adjacent joints. According to the parent-child node structure, each candidate keypoint is mapped to the corresponding node position of the H-Keypoint tree graph model, and the inner layer nodes, outer layer nodes and connecting layer nodes of each convolutional layer in the graph convolution module are respectively established with the keypoint nodes of the corresponding level in the H-Keypoint tree graph model to obtain a structured keypoint tree topology feature map. The result correction module 2 is used to input the keypoint tree topology feature map into the grouping graph convolution module. The grouping graph convolution module performs feature regression on each node in the keypoint tree topology feature map, and outputs the category confidence and spatial position offset of each keypoint. The regression results are used to detect group labels. When an abnormal grouping of keypoints is detected, the out-of-group detection mechanism is invoked. Based on the parent-child dependency relationship of each node in the keypoint tree topology feature map and the feature similarity of adjacent nodes, the abnormal keypoints are reassigned to the correct group, and the corrected keypoint grouping results are obtained. Through the inter-layer information transmission of the graph convolution module, the feature dependency correlation between adjacent level nodes is strengthened to ensure the spatial consistency of the connection relationship of each keypoint within the same group. Non-maximum suppression processing module 3 is used to perform non-maximum suppression processing on the corrected keypoint grouping results, suppress overlapping pose candidate results, and retain the candidate pose with the highest detection score as the final predicted pose; the grouping results after non-maximum suppression processing are used as group connection output to form a complete human pose skeleton; the grouped graph convolution module and the front-end keypoint detection network are jointly trained through end-to-end backpropagation to update the node mapping parameters of the H-Keypoint tree graph model and the network weights of the grouped graph convolution module until the model converges, and the trained tree-based graph convolutional regression grouping network is obtained, which is used for end-to-end human pose estimation of the input image.

[0042] In the above embodiments, the tree-based graph convolutional regression grouping system achieves high-precision structured modeling and spatial consistency optimization of human posture key points through the collaborative action of the feature map acquisition module, result correction module, and non-maximum suppression processing module. The feature map acquisition module uses an H-Keypoint tree graph model to construct a multi-level parent-child node topology from discrete candidate key points according to human kinematic constraints, transforming joint spatial dependencies into learnable graph node connection rules. Through a bidirectional mapping mechanism, nodes in the graph convolutional layer and key points in the tree graph form a hierarchical correspondence, enabling feature vectors to carry hierarchical spatial information of limb units, providing a structured feature representation with prior human anatomy for subsequent regression tasks. The result correction module uses grouped graph convolution to perform a joint regression task on the tree-topology feature map, simultaneously outputting key point category confidence and spatial offset. Through a group label detection mechanism, the group assignment of abnormal keypoints is dynamically corrected based on the dependency relationship between parent and child nodes and the similarity of adjacency features, ensuring that keypoints in the same group meet kinematic constraints. The inter-layer information transfer mechanism of graph convolution enhances the feature dependency between adjacent level nodes, enabling joint connections to maintain spatial consistency during regression and reducing the probability of misgrouping due to occlusion or deformation. The non-maximum suppression processing module filters overlapping pose candidates and retains the highest-scoring result as the final prediction, eliminating the interference of redundant detection on skeleton integrity. The group connection output forms a continuous skeleton structure that conforms to human topology. During end-to-end joint training, the node mapping parameters of the H-Keypoint tree diagram and the group graph convolution weights are optimized synchronously, enabling the network to gradually adapt to joint relationship modeling under complex poses and improving the model's robustness to occlusion, deformation, and multi-scale targets. The overall system achieves unified optimization of keypoint detection and group connections in end-to-end pose estimation through graph convolution feature propagation and hierarchical regression mechanism guided by tree model priors.

[0043] The following are the experimental results and analysis of the graph convolutional regression grouping method based on the tree model of this invention: This invention incorporates an H-Keypoint tree graph model and a grouping graph convolution module into the DEKR grouping algorithm to verify the feasibility of the proposed method. Experiments were conducted on the COCO2017 and CrowdPose datasets, and the experimental results were visualized and analyzed.

[0044] 1. Experimental Environment and Training Details 1.1 Experimental Environment Experimental environment: Ubuntu 20.04.4LTS, Intel Core i7-11800H @ 2.30GHz processor, 16GB RAM, and NVIDIA 3070 GPU. The deep learning framework used was PyTorch, and all programming experiments were conducted in a Python 3.6 environment using the Adam optimizer.

[0045] 1.1.1 Training Details Based on the training details of the baseline network, this experiment trained for a total of 140 epochs on the COCO dataset and 300 epochs on the CrowdPose dataset. The base learning rate for both datasets was set to 1e. -3 The COCO dataset was set to 1e in the 90th and 120th iterations, respectively. -4 and 1e -5 The CrowdPose dataset was set to 1e at the 200th and 260th iterations, respectively. -4 and 1e -5 .

[0046] 1.2 Analysis of Experimental Results The proposed tree-based graph convolutional regression grouping algorithm for keypoint grouping consists of two parts: an H-Keypoint tree graph model and a grouping graph convolution module. Experiments were conducted on the COCO2017 and CrowdPose datasets, respectively. The overall tree-based graph convolutional regression grouping algorithm was also tested, and the specific experimental results are as follows: 1.2.1 COCO2017 dataset This invention compares the experimental results of two improvements—the H-Keypoint tree graph model and the grouped graph convolution module—based on typical bottom-up human pose estimation networks from recent years. Specific experimental results are shown in Table 1. First, the experimental results using the H-Keypoint tree graph model are analyzed. Compared to the baseline network DEKR, the network's average precision (AP) is improved by 0.6, and the average recall rate (AR) is improved by 0.5. Compared to the recent novel Scale-aware network, the average precision (AP) is improved by 1.1. Next, the experimental results using the grouped graph convolution module are analyzed. Compared to the baseline network DEKR, the classic network HigherHRNet, and Scale-aware, the average precision (AP) is improved by 0.5, 1.4, and 1.0, respectively. Finally, the graph convolution regression grouping algorithm based on the tree model, i.e., the overall grouping method, is analyzed. Compared to the baseline network DEKR, the average precision (AP) is improved by 1.0.

[0047] Table 1. Experimental Results of the COCO2017 Dataset

[0048] When the present invention is used alone with the H-Keypoint tree diagram model and the grouped graph convolution module, the average accuracy (AP) of the experiment is improved to varying degrees. Overall, the grouping method proposed in this chapter can be used to estimate human pose more accurately.

[0049] To examine the computational complexity of the proposed grouping method, Table 2 presents the #param.(M) and GFLOPs for different algorithms.

[0050] Table 4.2 Comparison of computational complexity of algorithms for the COCO2017 dataset

[0051] GFLOPs represents the number of floating-point operations performed per second by a computing device, measured in units of 10⁹ operations per second; a higher value indicates a greater demand for computing resources. The table shows that, compared to HigherHRNet and DEKR, the algorithm improves keypoint detection accuracy without significantly altering computational speed. This demonstrates that the improved network model achieves a more effective balance between accuracy and the number of network parameters, reducing both the number of parameters and computational load while maintaining accuracy.

[0052] 2.1 CrowdPose dataset Since there are few experiments on the CrowdPose dataset for bottom-up human pose estimation methods, this invention cites some top-down human pose estimation methods tested on the CrowdPose dataset and compares the two types of methods. The experimental results are shown in Table 3.

[0053] Table 3. Experimental results of CrowdPose

[0054] First, the performance of adding the H-Keypoint tree graph model alone to the network is analyzed. Compared with the baseline network DEKR, it improves the mean accuracy (AP) by 0.8, and compared with the SPPE network, which currently performs well in top-down human pose estimation, it improves the AP by 2.1. Second, the performance of the grouped graph convolution module is analyzed. Compared with the baseline network DEKR and the classic network HigherHRNet, it improves the AP by 0.7 and 1.9, respectively. Finally, the performance of the tree-based graph convolutional regression grouping method is analyzed, improving the AP by 1.5 compared with the baseline network DEKR.

[0055] Analysis of the above data shows that the grouping improvement proposed in this invention performs well on the CrowdPose dataset, and the effect is even better when the H-Keypoint tree diagram model and the grouping graph convolution module are used together.

[0056] 3. Visualization of experimental results The proposed H-Keypoint tree diagram model and grouped graph convolution module were used to visualize and analyze experimental results on the COCO2017 and CrowdPose datasets, respectively. The specific experimental objectives are as follows: Ⅰ. Verify the performance of the H-Keypoint tree diagram model in solving the grouping of key points for different characters.

[0057] II. Verify the performance of the grouped graph convolution module in resolving connection errors of key points for the same person.

[0058] 3.1 COCO2017 dataset The visualization results obtained on the COCO2017 dataset were improved. Specifically, the visualization results from the DEKR network suffered from keypoint grouping errors due to the blurring of figures in the image background. The improved visualization results using the H-Keypoint tree model correctly reverted incorrectly grouped keypoints to their respective groups. The H-Keypoint tree model accurately grouped keypoints.

[0059] The visualization results of the DEKR network show that errors in the connection of adjacent keypoints are caused by object occlusion and errors in the connection of keypoints in the same group due to low image resolution. The visualization results using the group graph convolution module can correct the connection errors.

[0060] After analyzing the visualization results described above, the H-Keypoint tree diagram model can resolve the issue of incorrect grouping of keypoints for different individuals in multi-person pose estimation. The grouped graph convolution module can address the problem of incorrect connection of keypoints for the same individual.

[0061] 3.1.1 CrowdPose dataset Since the COCO2017 dataset has already provided visualization analysis for the two improvements proposed in this chapter, a holistic grouping method (graph convolutional regression grouping method based on a tree model) will be used for visualization analysis on the CrowdPose dataset. The visualization results for the CrowdPose dataset will then be obtained.

[0062] Because the elbows of males and wrists of females in the CrowdPose dataset visualizations are similar in color and location, errors occur in keypoint grouping. The proposed method can correctly distinguish their respective keypoints. Furthermore, the close proximity of the wrist keypoints between males and females leads to keypoint grouping errors; this proposed method can resolve the problem of keypoint grouping errors.

[0063] 3.2 Improved FF-DEKR Network with Enhanced Keypoint Grouping and Loss Function The tree-based graph convolutional regression grouping algorithm and the multi-stage feature fusion network based on keypoint detection error compensation are combined into an improved keypoint grouping and loss function FF-DEKR network, with the network structure as follows: Figure 8 As shown, the image is first input into the FF-DEKR network. After obtaining the keypoint detection structure, a tree-based graph convolutional regression grouping algorithm is used to group and connect the keypoints. Finally, the results are output. The FF-DEKR network with improved keypoint grouping and loss function is subjected to ablation experiments and visualization analysis on the COCO2017 and CrowdPose datasets to verify the effectiveness of the improvements in this invention.

[0064] This invention proposes a novel grouping method for keypoint grouping: a graph convolutional regression grouping method based on a tree model. This method solves two problems in keypoint grouping: when performing multi-person human pose estimation and detection, it standardizes the grouping rules to solve the problem of keypoint grouping errors between different people; and it solves the problem of incorrect connection of keypoints within the same human body.

[0065] The H-Keypoint tree diagram model embeds human keypoints within a decision tree model and arranges them in a specific order. Each child node inputs keypoint label information, regression information, and grouping label information. This invention defines a new grouping specification, introducing grouping reference labels and individual reference labels. Within-group detection is used to check for incorrectly grouped keypoints in the already grouped results. If an incorrectly grouped keypoint is found, out-of-group detection is used to find its original group. This ensures that keypoints belonging to the same person are correctly grouped and strengthens the connectivity of limb units. The grouped graph convolution module designs a three-layer structure based on the human skeletal connection features: outer keypoint layer, inner keypoint layer, and connecting keypoint layer. Each feature information node embeds second-order skeletal information (relative position and angle), modeling the relationships between keypoints using graph convolution, enhancing the dependency between adjacent keypoints, thereby solving the problem of incorrect keypoint connections within the same person. Simultaneously, combined with the H-Keypoint tree diagram model, keypoint grouping is integrated into a complete tree-based graph convolutional regression grouping method. The strategy was tested on the COCO2017 and CrowdPose datasets, and the results were improved compared to the baseline network.

[0066] The keypoint detection and grouping methods proposed in this invention are combined into a new network: the FF-DEKR network with improved keypoint grouping and loss function. Experiments were conducted on the COCO2017 dataset and the CrowdPose dataset, respectively. The results show that the average precision (AP) has been further improved, verifying the effectiveness of the algorithm of this invention.

[0067] Figure 9 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present invention is shown.

[0068] The electronic device may include a central processing unit / microprocessor / main control chip 4; and a storage medium 5 coupled to the central processing unit / microprocessor / main control chip 4 and storing computer-executable instructions therein for performing the steps of various methods of embodiments of the present invention when executed by the processor.

[0069] The central processing unit / microprocessor / main control chip 4 may include, but is not limited to, one or more processors or microprocessors.

[0070] Storage medium 5 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0071] In addition, the electronic device may include (but is not limited to) a data bus 6, an input / output bus / external bus / device bus 7, a display 8, and input / output devices 9 (e.g., keyboard, mouse, speaker, etc.).

[0072] The central processing unit / microprocessor / main control chip 4 can communicate with external devices (8, 9, etc.) via wired or wireless networks (not shown) through the input / output bus / external bus / device bus 7.

[0073] The storage medium 5 may also store at least one computer-executable instruction for performing the steps of various functions and / or methods in the embodiments described herein when run by the central processing unit / microprocessor / main control chip 4.

[0074] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0075] Figure 10 A schematic diagram of a computer-readable storage medium according to an embodiment of the present invention is shown.

[0076] like Figure 10As shown, the non-transitory computer-readable storage medium 11 stores instructions, such as computer-readable instructions 10. When the computer-readable instructions 10 are executed by a processor, the various methods described above can be performed. The non-transitory computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-transitory non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the non-transitory computer-readable storage medium 11 can be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions 10 stored on the non-transitory computer-readable storage medium 11, the various methods described above can be performed.

[0077] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0078] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0079] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods of the various embodiments of the present invention through a computer device (which may be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0081] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A graph convolutional regression grouping method based on a tree model, characterized in that, Includes the following steps: The H-Keypoint tree diagram model divides candidate keypoints into a multi-level parent-child node structure according to human kinematic constraints. Based on the parent-child node structure, each candidate keypoint is mapped to the corresponding node position in the H-Keypoint tree diagram model. The inner, outer, and connecting layer nodes of each convolutional layer in the graph convolution module are then linked to the keypoint nodes at the corresponding levels in the H-Keypoint tree diagram model to obtain a structured keypoint tree topology feature map. The keypoint tree topology feature map is input into the group graph convolution module. The group graph convolution module performs feature regression on each node in the keypoint tree topology feature map and outputs the category confidence and spatial location offset of each keypoint. The regression results are subjected to group label detection. When an out-of-group detection mechanism is called when an abnormal grouping of key points is detected, the corrected key point grouping results are obtained. Non-maximum suppression is applied to the corrected keypoint grouping results, and the grouping results after non-maximum suppression are used as the group connection output to form a complete human posture skeleton. The grouped graph convolutional module and the front-end keypoint detection network are jointly trained through end-to-end backpropagation to obtain a trained tree-based graph convolutional regression grouping network, which is used for end-to-end human pose estimation of the input image.

2. The graph convolutional regression grouping method based on a tree model as described in claim 1, characterized in that, Obtain all candidate keypoints and their corresponding feature vectors from the keypoint detection network in the target image, and input the candidate keypoints into the pre-constructed H-Keypoint tree diagram model.

3. The graph convolutional regression grouping method based on a tree model as described in claim 1, characterized in that, in, The parent node represents a limb unit and is used to characterize the spatial attachment relationship between adjacent joints.

4. The graph convolutional regression grouping method based on a tree model as described in claim 1, characterized in that, The H-Keypoint tree diagram model includes ancestor nodes, parent nodes, and child nodes; Ancestor nodes are feature information of individual reference labels embedded in each ancestor node to distinguish different individual instances; The parent node divides the 17 key points into five groups based on human geometry: head unit, left upper limb unit, right upper limb unit, left lower limb unit, and right lower limb unit; each parent node unites key points within the same range. The child nodes are extensions of the parent nodes, extending the 17 key points to each unit; the child nodes contain key point label information and key point regression information for each key point, which are encoded on the tree structure according to the different key points.

5. The graph convolutional regression grouping method based on a tree model as described in claim 1, characterized in that, The process of grouping and labeling key points in an H-Keypoint tree diagram includes the following steps: Obtain the original grouping results of each key point output by the grouping graph convolution module and the key point heatmap output by the key point detection network. For each key point heatmap, define and output the grouping reference label that corresponds one-to-one with the key point heatmap. The group reference labels are assigned values ​​according to the current grouping results, and the key points belonging to the same person are assigned similar label values; the individual reference label of each person in the image is obtained, and the average value of the group reference labels of all key points of the same person is taken to obtain the individual reference label of the person; The original grouping results are subjected to dual detection using group reference labels and individual reference labels; for each grouped person, the mean square error between the group reference label and the individual reference label of each key point within the person is calculated; the grouping error is compared with a preset threshold to determine whether there is a grouping error in the key points within the group, and the detection results within the group are obtained. When there are misgrouped keypoints, calculate the normal function value between the person with the misgrouped keypoint and the individual reference labels of all other people in the image. Based on the normal function value, determine whether there is a misassignment relationship between the other people and the person with the misgrouped keypoint. Then, reassign the misgrouped keypoint to the group of people with the smallest difference from its individual reference label to obtain the corrected keypoint grouping label result.

6. The graph convolutional regression grouping method based on a tree model as described in claim 1, characterized in that, The grouped graph convolutional module automatically matches the number of modules based on the number of characters, and designs a three-layer structure for the grouped graph convolutional module based on the connection method of the human skeleton: The outer key points are the terminal key points of the head and limb units: left wrist, right wrist, left ankle, right ankle, left eye, right eye, left ear, and right ear; The inner key points are the key points connecting the inner edges of the four limb units of the human body: nose, left shoulder, right shoulder, left hip, and right hip; the nose serves as the central inner key point. The connecting layer keypoints are between the outer keypoints and the inner keypoints: left elbow, right elbow, left knee, right knee.

7. The graph convolutional regression grouping method based on a tree model as described in claim 1, characterized in that, The grouped graph convolution module updates the representation of each node by aggregating the neighbor information of each key point. Nodes can obtain connection information through their connected upstream and downstream nodes. The grouped graph convolution module embeds second-order skeletal information and uses edge information of already detected key points to find pixels with strong correlations. Second-order information in skeleton data refers to the information in pose estimation tasks that includes not only the coordinates of keypoints but also the relative positions, angles, velocities, and accelerations between keypoints.

8. The graph convolutional regression grouping method based on a tree model as described in claim 1, characterized in that, The regression results are subjected to group label detection. When an abnormal grouping of key points is detected, the out-of-group detection mechanism is invoked. Based on the parent-child dependency relationship of each node in the key point tree topology feature map and the feature similarity of adjacent nodes, the abnormal key points are reassigned to the correct group, and the corrected key point grouping results are obtained.

9. The graph convolutional regression grouping method based on a tree model as described in claim 1, characterized in that, After joint training, the node mapping parameters of the H-Keypoint tree graph model and the network weights of the grouped graph convolutional module are updated until the model converges, resulting in the trained tree-based graph convolutional regression grouping network.

10. A tree-model-based graph convolutional regression grouping system, used to implement the tree-model-based graph convolutional regression grouping method as described in any one of claims 1 to 9, characterized in that, include: The feature map acquisition module is used to acquire all candidate key points and their corresponding feature vectors output by the key point detection network in the target image, and input the candidate key points into the pre-constructed H-Keypoint tree diagram model; The H-Keypoint tree graph model divides candidate keypoints into multi-level parent-child node structures according to human kinematic constraints. Each parent node represents a limb unit, representing the spatial dependency relationship between adjacent joints. Based on the parent-child node structure, each candidate keypoint is mapped to its corresponding node position in the H-Keypoint tree graph model. Furthermore, the inner, outer, and connecting layer nodes of each convolutional layer in the graph convolution module are established with the corresponding keypoint nodes in the H-Keypoint tree graph model, resulting in a structured keypoint tree topology feature map. The result correction module is used to input the keypoint tree topology feature map into the grouping graph convolution module. The grouping graph convolution module performs feature regression on each node in the keypoint tree topology feature map, and outputs the category confidence and spatial position offset of each keypoint. The regression results are then subjected to group label detection. When an abnormal grouping of keypoints is detected, the out-of-group detection mechanism is invoked. Based on the parent-child dependency relationship of each node in the keypoint tree topology feature map and the feature similarity of adjacent nodes, the abnormal keypoints are reassigned to the correct group, resulting in the corrected keypoint grouping results. Through the inter-layer information transfer of the graph convolution module, the feature dependency correlation between adjacent level nodes is strengthened to ensure the spatial consistency of the connection relationship of each keypoint within the same group. The non-maximum suppression processing module is used to perform non-maximum suppression processing on the corrected keypoint grouping results, suppressing overlapping pose candidate results and retaining the candidate pose with the highest detection score as the final predicted pose. The grouping results after non-maximum suppression processing are used as group connection outputs to form a complete human pose skeleton. The grouped graph convolution module and the front-end keypoint detection network are jointly trained through end-to-end backpropagation to update the node mapping parameters of the H-Keypoint tree graph model and the network weights of the grouped graph convolution module until the model converges, resulting in a trained tree-based graph convolutional regression grouping network used for end-to-end human pose estimation of the input image.

Images