Pose recognition method and system based on pose recognition neural network

Abstract

This invention provides a posture recognition method and system based on a posture recognition neural network, belonging to the field of posture recognition technology. The invention constructs a standard library containing safe and dangerous posture categories and generates a soft-label sample dataset by annotating several key points. Key points are defined as graph nodes, and feature matrices and adjacency matrices are constructed. A dual-branch fusion network model is designed, extracting topological features of key points through a graph convolution branch and extracting global contextual features of the image through a global vision branch. After fusion, a posture similarity spectrum vector is output. Finally, the posture state is determined by calculating the geometric difference between the image to be analyzed and the most similar posture template in terms of key point distance and joint angles, and comparing its overall difference with the safe / dangerous categories. This invention combines the advantages of data-driven and rule-based validation, significantly improving the robustness of dangerous posture recognition and the interpretability of decisions, effectively reducing safety risks.

Description

Technical Field

[0001] This invention relates to the field of posture recognition technology, specifically to a posture recognition method and system based on a posture recognition neural network. Background Technology

[0002] Pose recognition is an important research area in computer vision and machine learning, aiming to identify and understand human postures and movements in various scenarios by analyzing the positions of key points on the human body in images or videos. With the rapid development of deep learning technology, especially the widespread application of convolutional neural networks and graph neural networks, the accuracy and robustness of pose recognition technology have been significantly improved. Modern pose recognition models can automatically learn high-level features, overcoming the limitations of traditional methods in feature extraction and classification, resulting in more accurate recognition.

[0003] This technology has demonstrated broad application potential across multiple fields. In sports and fitness, posture recognition can analyze athletes' movements to help optimize training results; in security monitoring, by monitoring human posture in public places, abnormal behavior can be identified in a timely manner, improving security. Furthermore, in interactive applications of virtual reality and augmented reality, posture recognition can capture users' movements in real time, enabling a more natural interactive experience. In the field of medical rehabilitation, it has become an important tool for monitoring patients' rehabilitation process and assessing their motor abilities.

[0004] The existing technology has the following shortcomings:

[0005] Many traditional pose recognition methods rely on 2D images for keypoint detection, which is susceptible to occlusion, lighting variations, and background interference, leading to inaccurate keypoint localization. This limitation makes the models perform poorly in complex scenes, especially when multiple people are in the same frame or their poses overlap, where recognition accuracy drops significantly. Secondly, existing technologies often use linear connections to handle spatial relationships between keypoints, failing to fully explore higher-order relationships between nodes and limiting the model's ability to understand complex poses. This makes it difficult for existing methods to perform effective time-series analysis for pose changes in dynamic scenes, resulting in unstable recognition performance. Traditional pose recognition methods often focus on classification tasks and lack a comprehensive assessment of pose hazards, failing to provide effective support for safety monitoring and health management in practical applications, thus limiting their application scope.

[0006] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide a posture recognition method and system based on a posture recognition neural network to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A pose recognition method based on a pose recognition neural network, comprising the following steps:

[0010] Step 1: Obtain an image database containing multiple poses and construct a standard library containing K predefined pose categories; use annotation tools to annotate the key points in each image and assign them to the pose categories in the standard library to generate corresponding pose category label vectors; preprocess the annotated images and integrate them into a sample dataset, where the key points include head, neck, shoulder, elbow, wrist, hip, knee, and ankle.

[0011] Step 2: Define each labeled key point as a node in the graph, obtain the feature vector of each node, including node position coordinates and visibility score, integrate the feature vectors of all nodes on each image into a feature matrix, and then establish key point connection relationships to form edges by connecting key points, and obtain the adjacency matrix.

[0012] Step 3: Construct a dual-branch fusion network model, including a graph convolution branch and a global vision branch. Input the feature matrix and adjacency matrix of the images in the sample dataset into the graph convolution branch, and input the preprocessed image into the global vision branch. Train the model with the pose category label vector corresponding to the image in the sample dataset as the label, so that the model outputs a K-dimensional similarity spectrum vector, where each component represents the percentage of similarity between the image and the corresponding pose category in the standard library.

[0013] Step 4: Define the ideal relative distance and ideal angle for each pose category in the standard library; input the image to be analyzed into the trained dual-branch fusion network model to obtain the similarity spectrum vector of the image to be analyzed; calculate the actual distance and actual angle between each key point in the image to be analyzed, select the N pose categories with the highest similarity in the similarity spectrum, and calculate the difference between the image to be analyzed and these pose categories respectively to determine whether a danger signal is issued.

[0014] Furthermore, the specific logic for obtaining the sample dataset is as follows:

[0015] Image data containing various postures was downloaded from a public dataset. A standard library containing K predefined posture categories was constructed, and all posture categories were divided into two categories: Category I safe postures and Category II dangerous postures. Category I safe postures represent operating postures that comply with safety regulations, including at least standard upright observation, bending over and carrying, and operating at a safe distance. Category II dangerous postures represent typical violations and high-risk postures, including at least waist bending angles exceeding safety and thresholds, arms entering the mechanical danger zone, and loss of balance during high-altitude operations.

[0016] For each image in the image database, at least 14 human keypoints are labeled, including the head, neck, shoulders, elbows, wrists, hips, knees, and ankles. Shoulders include left and right shoulders; elbows include left and right elbows; wrists include left and right wrists; hips include left and right hips; knees include left and right knees; and ankles include left and right ankles. Experts assign human poses in the images to multiple categories in a standard library, and a K-dimensional pose category label vector is generated using soft labeling, represented as follows: ,in, Represents the pose category label vector. This represents the confidence score indicating that the image belongs to the first pose category. This represents the confidence score indicating that the image belongs to the second pose category. This represents the confidence score that the image belongs to the q-th pose category. This represents the confidence score indicating that the image belongs to the Kth pose category; q is the pose category index in the standard library, and K is the number of pose categories in the standard library; the confidence score ranges from [0,1], and The labeled image is then normalized in size, and the image pixel values ​​are scaled to [size value missing]. Within the specified range, the preprocessed images are integrated into a sample dataset.

[0017] Furthermore, the specific criteria used to obtain the feature matrix and adjacency matrix include:

[0018] If a node's keypoint is detectable in the image, its visibility score is defined as 1; if the corresponding keypoint is not detected in the image, its visibility score is defined as 0. The feature vector of each node is obtained as follows:

[0019]

[0020] in, Indicates the first The feature vector of each node Indicates the first The visibility score of each node, These represent the positions of the node on the X-axis and Y-axis, respectively. Indicates the node index;

[0021] By integrating the feature vectors of all nodes in each image and stacking them sequentially, a graph structure feature matrix for the image is constructed, represented as:

[0022]

[0023] in, For the characteristic matrix, This represents the number of nodes, which is also the number of key points. ;

[0024] Based on the normal physiological connection structure, the edges between nodes are defined, and an adjacency matrix is ​​constructed to represent the topological connection relationship of the graph. The node connection relationship is as follows: the head connects to the neck, the neck connects to the left shoulder, right shoulder, left hip and right hip respectively, the left shoulder connects to the left elbow, the left elbow connects to the left wrist, the right shoulder connects to the right elbow, the right elbow connects to the right wrist, the left hip connects to the left knee, the left knee connects to the left ankle, the right hip connects to the right knee, and the right knee connects to the right ankle.

[0025] The adjacency matrix is ​​one The symmetric matrix is ​​represented as:

[0026]

[0027] in, It is an adjacency matrix. Represents a node and nodes The connection relationship between nodes, if nodes and nodes If there is a connection between them, then ,otherwise , It is also the index of the node.

[0028] Furthermore, constructing the dual-branch fusion network model specifically includes:

[0029] The sample dataset is divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5. The dual-branch fusion network consists of a parallel graph convolution branch, a global vision branch, a feature fusion module, and an output layer.

[0030] The graph convolution branch is an L-layer graph convolutional network. The feature matrix and adjacency matrix of the images in the training set are used as the output of this branch. After each graph convolutional layer performs neighborhood information aggregation, aggregation information normalization, feature transformation and nonlinear activation processing, the final feature vector of each node contains the structural information in its multi-hop neighborhood. Global average pooling is applied to the final feature vector of all nodes to obtain the feature vector representing the global structure of the image.

[0031] The global vision branch is used to extract global contextual features from images in the training set. The images in the training set are input into the network, and the feature map before the last fully connected layer is taken. After passing through a global average pooling layer, a global vision feature vector is obtained.

[0032] A feature fusion module is used to input the feature necklaces of the global structure from the two branches and the global visual feature vectors into a fully connected layer for dimensionality reduction and nonlinear transformation; then, a K-dimensional vector is output through the output layer, where each component of the vector represents the percentage of similarity with the corresponding pose category in the standard library; the output layer consists of a K-dimensional fully connected layer and a Softmax activation function;

[0033] The model uses pose category label vectors as supervision signals and multi-class cross-entropy as the loss function to evaluate the difference between the model output and the true value. The Adam optimizer is used to minimize the loss function and iteratively update all parameters in the model. When the change in the loss function on the validation set is less than a preset threshold in multiple consecutive iterations, the model is considered to have completed training.

[0034] Furthermore, the specific logic for determining whether a danger signal should be issued is as follows:

[0035] Define the ideal relative distance between all keypoint pairs in each image in the standard library and the ideal set of angles formed by three keypoints. The set of angles includes: head-neck-left shoulder, head-neck-right shoulder, left shoulder-left elbow-left wrist, right shoulder-right elbow-right wrist, left hip-left knee-left wrist, and right hip-right knee-right wrist.

[0036] The image to be analyzed is input into the trained dual-branch fusion network model to obtain the similarity spectrum vector. Based on the coordinates of key points in the image, the Euclidean distance between all key point pairs is calculated as the actual distance. Then, the cosine theorem is used to calculate the actual angle values ​​of all angles in the angle set. The calculation formula is as follows:

[0037]

[0038] in, This indicates the angle formed by the head, neck, and left shoulder;

[0039] Similarly, the actual angles formed between the head-neck-right shoulder, left shoulder-left elbow-left wrist, right shoulder-right elbow-right wrist, left hip-left knee-left ankle, and right hip-right knee-right ankle are obtained and represented as follows: ;

[0040] The N pose categories with the highest similarity are selected from the similarity spectral vector. The difference between the pose in the image to be analyzed and its corresponding pose category in the standard library is calculated using the following formula:

[0041]

[0042] in, This represents the degree of difference between the image to be analyzed and the q-th pose category in the standard library. This represents the actual relative distance between the i-th node and the j-th node in the image to be analyzed. This represents the set of the m-th actual angles in the image to be analyzed. This represents the ideal relative distance between the i-th node and the j-th node in the q-th pose category of the standard library. This represents the m-th ideal angle set in the standard library, where m is the index of the angle set. These are the weight coefficients for the corresponding terms, and ;

[0043] The sum of the differences between the image to be analyzed and all first-class safe postures is calculated as the safe difference degree, and the sum of the differences between the image to be analyzed and all second-class dangerous postures is calculated as the dangerous difference degree. The magnitudes of the safe difference degree and the dangerous difference degree are determined. If the dangerous difference degree is high, the current posture in the image to be analyzed is determined to be a safe state, and the posture category corresponding to the first safe posture in the standard library with the smallest difference degree is regarded as the posture category of the image to be analyzed. If the safe difference degree is high, the current posture in the image to be analyzed is determined to be a dangerous state, and the posture category corresponding to the second dangerous posture in the standard library with the smallest difference degree is regarded as the posture category of the image to be analyzed, and a danger signal is issued.

[0044] The present invention also provides a posture recognition system based on a posture recognition neural network, wherein the posture recognition system based on a posture recognition neural network is used to implement the above-mentioned posture recognition method based on a posture recognition neural network, and includes:

[0045] The data acquisition and annotation module is used to acquire an image database containing multiple poses, construct a standard library containing K predefined pose categories, use annotation tools to annotate key points in each image and assign them to pose categories in the standard library to generate corresponding pose category label vectors, and preprocess the annotated images and integrate them into a sample dataset. The key points include head, neck, shoulder, elbow, wrist, hip, knee and ankle.

[0046] The graph structure construction module is used to define each labeled key point as a node of the graph, obtain the feature vector of each node, including the node position coordinates and visibility score, integrate the feature vectors of all nodes on each image into a feature matrix, and then establish the connection relationship of key points to form edges by connecting key points and obtain the adjacency matrix.

[0047] The model building module is used to construct a dual-branch fusion network model, including a graph convolution branch and a global vision branch. The feature matrix and adjacency matrix of the images in the sample dataset are input into the graph convolution branch, and the preprocessed image is input into the global vision branch. The model is trained with the pose category label vectors corresponding to the images in the sample dataset as labels, so that the model outputs a K-dimensional similarity spectrum vector, where each component represents the percentage of similarity between the image and the corresponding pose category in the standard library.

[0048] The pose evaluation module defines the ideal relative distance and ideal angle for each pose category in the standard library; it inputs the image to be analyzed into the trained dual-branch fusion network model to obtain the similarity spectrum vector of the image to be analyzed; it calculates the actual distance and actual angle between each key point in the image to be analyzed, selects the N pose categories with the highest similarity in the similarity spectrum, and calculates the difference between the image to be analyzed and these pose categories to determine whether a danger signal is issued.

[0049] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0050] This invention innovatively combines the powerful modeling capabilities of graph convolutional neural networks (GNNs) for the topological structure of human key points with the ability of GNNs to extract global contextual information from images by constructing a dual-branch fusion network, effectively overcoming the limitations of single-model approaches. This fusion mechanism enables the invention to comprehensively utilize structural and scene information for cross-validation when facing complex postures with key point occlusion, detection bias, or visual similarities. This significantly improves the accuracy and robustness of identifying typical dangerous postures in industrial scenarios, such as waist bending angles exceeding safety and thresholds, and arms entering hazardous areas. More importantly, based on the initial neural network recognition, this invention introduces a post-processing judgment process based on geometric rules for difference. By calculating the quantitative differences between the pose and standard safe and dangerous posture templates, a reliable safety redundancy is constructed. This design not only enhances the ability to judge postures with ambiguous boundaries and significantly reduces the false negative rate, but also makes the decision-making basis of the entire recognition process more transparent and interpretable, providing a highly intelligent and reliable technical guarantee for industrial safety production. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the overall method flow of the present invention;

[0052] Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0054] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0055] Example:

[0056] Please see Figure 1 The present invention provides a technical solution:

[0057] A pose recognition method based on a pose recognition neural network, comprising the following steps:

[0058] Step 1: Obtain an image database containing multiple poses and construct a standard library containing K predefined pose categories; use annotation tools to annotate the key points in each image and assign them to the pose categories in the standard library to generate corresponding pose category label vectors; preprocess the annotated images and integrate them into a sample dataset, where the key points include head, neck, shoulder, elbow, wrist, hip, knee, and ankle.

[0059] In this embodiment, the specific logic for obtaining the sample dataset is as follows:

[0060] Image data containing various poses was downloaded from the publicly available MS-COCO dataset. The MS-COCO dataset boasts rich scene diversity and pose variations, making it suitable for training pose recognition models with strong generalization capabilities. Based on practical application needs (such as industrial safety monitoring), a standard library containing K predefined pose categories was constructed. All pose categories were divided into two categories: Category I safe poses and Category II hazardous poses. Category I safe poses represent operational postures that conform to safety regulations, including at least standard upright observation (body upright, head facing forward, arms hanging naturally or placed at the sides), bent-over lifting (knees bent, waist maintaining a natural curve, arms holding the object with a stable center of gravity), and safe distance operation (body maintaining a safe distance from hazardous equipment, stable posture without tilting).

[0061] The second category of dangerous postures represents typical violations and high-risk postures, including at least the following: bending the waist at an angle exceeding safety and threshold (a bending angle exceeding the safety threshold, such as greater than 45°, easily leads to muscle injury); arms entering mechanically hazardous areas (upper limbs extending into the operating range of equipment, posing a risk of mechanical crush); and loss of balance while working at height (shifting the body's center of gravity, supporting oneself on one foot or without arm support, easily leading to falls). The value of K needs to balance classification granularity and model practicality. If K is too small (e.g., K < 5), it may lead to category confusion (it is difficult to distinguish between bending over and bending the waist at an angle exceeding safety and threshold); if K is too large (e.g., K > 20), it will increase the complexity of annotation and the difficulty of model training. The preferred range for K is 8-15, which can be determined by summarizing typical postures through expert experience or clustering algorithms (such as K-means).

[0062] The core biomechanical characteristic of loss of balance during high-altitude work is that the vertical projection point of the body's center of mass deviates too far from the base of support for both feet, or presents an unstable configuration of single-leg support with a significant body tilt. Specific judgment methods include, for example, determining the support base based on key point coordinates. Typically, the line segment connecting the left and right ankles is used as the primary support base. If one foot leaves the ground (e.g., a key point visibility score of 0), it is determined to be single-leg support, and the support base is the ankle itself. Then, the approximate position of the body's center of mass is estimated. In a simplified model, the center of mass can be approximated as the coordinates of the midpoint of the hip. The horizontal coordinate of the center of mass (…) The midpoint of the x-coordinate of the left and right buttocks, and the y-coordinate of the centroid ( Let the midpoint of the left and right hip coordinates be the point on the horizontal plane. Then, calculate the shortest distance from the vertical projection of the center of mass onto the horizontal plane (considering only the x-coordinate) to the support base. For a two-foot support, the calculation point is ( ,0) to line segment ( ,0)-( The vertical distance of ,0) Represents the x-coordinate of the left ankle. This represents the horizontal coordinate of the right ankle. When the horizontal distance between the center of mass projection point and the center point of the supporting base is greater than one-third of the distance between the two ankles, it is determined that the body's center of gravity is too far off, and there is a risk of imbalance. When testing on a single-leg support, if the angle between the line connecting the head and the landing ankle and the vertical direction is greater than 15 degrees, it is determined that the body is severely tilted and has lost balance.

[0063] For each image in the image database, at least 14 human keypoints were labeled using the CVAT tool, including the head, neck, shoulders, elbows, wrists, hips, knees, and ankles. Shoulders included the left and right shoulders; elbows included the left and right elbows; wrists included the left and right wrists; hips included the left and right hips; knees included the left and right knees; and ankles included the left and right ankles. Experts assigned the human poses in the images to multiple categories in a standard library, and a K-dimensional pose category label vector was generated using soft labeling, represented as follows: ,in, Represents the pose category label vector. This represents the confidence score indicating that the image belongs to the first pose category. This represents the confidence score indicating that the image belongs to the second pose category. This represents the confidence score that the image belongs to the q-th pose category. This represents the confidence score indicating that the image belongs to the Kth pose category; q is the pose category index in the standard library, and K is the number of pose categories in the standard library; the confidence score ranges from [0,1], and The labeled images are normalized and scaled to a fixed size, such as 224×224 pixels. Bilinear interpolation is used to maintain proportional consistency and avoid distortion. Then, the image pixel values ​​are linearly mapped from the original range to... Within the specified range, preprocessed images are integrated into a sample dataset. If an image clearly belongs to a certain category, the confidence score is higher; for other categories, it approaches 0.

[0064] Step 2: Define each labeled key point as a node in the graph, obtain the feature vector of each node, including node position coordinates and visibility score, integrate the feature vectors of all nodes on each image into a feature matrix, and then establish key point connection relationships to form edges by connecting key points, and obtain the adjacency matrix.

[0065] In this embodiment, the specific logic for obtaining the feature matrix and adjacency matrix is ​​as follows:

[0066] If a node's keypoint is detectable in the image, its visibility score is defined as 1; if the corresponding keypoint is not detected in the image, its visibility score is defined as 0. The feature vector of each node is obtained as follows:

[0067]

[0068] in, Indicates the first The feature vector of each node Indicates the first The visibility score of each node, These represent the positions of the node on the X-axis and Y-axis, respectively. Indicates the node index;

[0069] By integrating the feature vectors of all nodes in each image and stacking them sequentially, a graph structure feature matrix for the image is constructed, represented as:

[0070]

[0071] in, For the characteristic matrix, This represents the number of nodes, which is also the number of key points. ;

[0072] Based on the normal physiological connection structure, the edges between nodes are defined, and an adjacency matrix is ​​constructed to represent the topological connection relationship of the graph. The node connection relationship is as follows: the head connects to the neck, the neck connects to the left shoulder, right shoulder, left hip and right hip respectively, the left shoulder connects to the left elbow, the left elbow connects to the left wrist, the right shoulder connects to the right elbow, the right elbow connects to the right wrist, the left hip connects to the left knee, the left knee connects to the left ankle, the right hip connects to the right knee, and the right knee connects to the right ankle.

[0073] The adjacency matrix is ​​one The matrix is ​​represented as:

[0074]

[0075] in, It is an adjacency matrix. Represents a node and nodes The connection relationship between nodes, if nodes and nodes If there is a connection between them, then ,otherwise , It is also the index of the node;

[0076] The adjacency matrix here has 14 nodes, which is one. The adjacency matrix, based on the connectivity relationships, is as follows:

[0077]

[0078] in, This is a specific adjacency matrix.

[0079] By treating pose keypoints as nodes in a graph and defining the connections between them, graph structures can be effectively used to model pose information. This representation captures the relationships between keypoints, including spatial and topological structures, thus providing valuable information for subsequent graph convolutional networks. Each node's feature vector... This includes the spatial coordinates and visibility information of nodes. This structured feature representation facilitates the network's learning of the spatial location and visibility of keypoints, enabling the model to fully consider the detectable states of different keypoints when processing pose recognition tasks, thereby improving the accuracy and robustness of recognition. By clearly defining the connections between keypoints and constructing an adjacency matrix, the interdependencies between nodes can be clearly represented. This not only helps the network capture the relative positions of keypoints but also guides the model to perform effective feature aggregation during graph convolution, improving the model's ability to express pose features.

[0080] Step 3: Construct a dual-branch fusion network model, including a graph convolution branch and a global vision branch. Input the feature matrix and adjacency matrix of the images in the sample dataset into the graph convolution branch, and input the preprocessed image into the global vision branch. Train the model with the pose category label vector corresponding to the image in the sample dataset as the label, so that the model outputs a K-dimensional similarity spectrum vector, where each component represents the percentage of similarity between the image and the corresponding pose category in the standard library.

[0081] In this embodiment, constructing the dual-branch fusion network model specifically includes:

[0082] The sample dataset is divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5. The dual-branch fusion network consists of a parallel graph convolution branch, a global vision branch, a feature fusion module, and an output layer.

[0083] The graph convolution branch is an L-layer graph convolutional network. The feature matrix and adjacency matrix of the images in the training set are used as the output of this branch. After each graph convolutional layer performs neighborhood information aggregation, aggregation information normalization, feature transformation and nonlinear activation processing, the final feature vector of each node contains the structural information in its multi-hop neighborhood. Global average pooling is applied to the final feature vector of all nodes to obtain the feature vector representing the global structure of the image.

[0084] The global vision branch is used to extract global contextual features from images in the training set. The images in the training set are input into the network, and the feature map before the last fully connected layer is taken. After passing through a global average pooling layer, a global vision feature vector is obtained.

[0085] A feature fusion module is used to input the feature necklaces of the global structure from the two branches and the global visual feature vectors into a fully connected layer for dimensionality reduction and nonlinear transformation; then, a K-dimensional vector is output through the output layer, where each component of the vector represents the percentage of similarity with the corresponding pose category in the standard library; the output layer consists of a K-dimensional fully connected layer and a Softmax activation function;

[0086] Pose category label vectors are used as supervision signals, and multi-class cross-entropy is used as the loss function to evaluate the difference between the model output and the true value. The Adam optimizer is used to minimize the loss function, iteratively updating all parameters in the model. The model is considered trained successfully when the change in the loss function over multiple consecutive iterations on the validation set is less than a preset threshold. Here, the threshold is set to a change rate of less than 0.0001 over 10 consecutive iterations. This value serves as a reference, allowing observation of the natural fluctuation range of the loss value near the convergence point; if it is around this value, it is set as the threshold.

[0087] Accurate human pose recognition relies not only on precise spatial relationships between local key points but also on supplementary global visual context information. The parallel structure of graph convolutional branches and global visual branches is precisely designed to collaboratively capture these two different levels of complementary feature representations, thereby enhancing the model's discriminative ability in complex real-world scenarios.

[0088] The rationale behind the graph convolutional branch lies in its ability to explicitly model the inherent topological structure of the human body. By treating keypoints as graph nodes and defining edges based on physiological connections, this branch utilizes Graph Convolutional Network (GCN) layers to aggregate neighborhood information. Each GCN operation enables the features of each node to incorporate information from its directly connected neighbors (e.g., one-hop neighborhoods) and even distant nodes indirectly connected through multiple layers of the network (e.g., multi-hop neighborhoods). This process simulates the cognitive logic of "local joint combinations forming limbs, and limb combinations forming complete postures" in human posture understanding. For example, recognizing a posture where the waist bending angle exceeds a safe threshold requires not only the positional information of the waist keypoints (e.g., hips) themselves, but also their relative positional relationships with keypoints in the upper body (shoulders, neck) and lower body (knees). Finally, the global structural feature vector obtained by global average pooling of all node features effectively encodes the spatial configuration and topological constraints of the entire human skeleton, exhibiting robustness to occluded or partially visible keypoints, as information from visible nodes can be propagated through the graph structure to invisible nodes for a certain degree of reconstruction.

[0089] However, keypoint structural information alone may be insufficient. The introduction of a global vision branch aims to capture the rich visual context that cannot be fully expressed by sparse keypoints. This contextual information includes, but is not limited to: the interaction between a person and the environment (e.g., whether they are near dangerous equipment), the object being held (e.g., whether they are carrying heavy objects), the overall body outline, and scene factors such as lighting and background. This information is crucial for distinguishing visually similar but semantically different poses. For example, a "raised arm" gesture may not be distinguishable on the keypoint map as "operating equipment" or "waving," but the background information of the equipment in the global image can provide decisive clues. This branch typically uses a mature convolutional neural network (CNN) as its backbone because it has proven very effective in extracting global semantic features of images. By extracting the feature map before the last fully connected layer and applying global average pooling, the resulting global visual feature vector contains scene-level semantic information of the image.

[0090] The design of the feature fusion module is crucial, as it effectively integrates heterogeneous features from two branches, combining structured graph features with semantic visual features. A common and effective fusion strategy is to simply concatenate the two feature vectors and then perform dimensionality reduction and nonlinear transformation through a fully connected layer. Fully connected layers can learn the complex mapping relationship between the two feature spaces and automatically adjust their contribution weights to the final classification task. This fusion mechanism allows the model to adaptively determine when to rely more on keypoint structural information (e.g., determining joint bending angles) and when to rely more on global contextual information (e.g., determining whether the work is in a high-altitude working environment).

[0091] The output layer maps the fused high-level features to the same dimension as the number of categories K in the standard library through a K-dimensional fully connected layer and a Softmax activation function, and outputs the similarity percentage (probability distribution) for each category. Employing a multi-class cross-entropy loss function with soft labels as supervision signals better handles potential ambiguity between pose categories, consistent with the soft label concept defined in step 1. Training with the Adam optimizer, combined with early stopping (stopping training when the validation set loss no longer significantly improves), prevents overfitting and ensures the model's generalization ability. In summary, this dual-branch model provides a reasonable and powerful solution for fine-grained pose recognition in complex scenarios through the complementary fusion of structural and contextual information.

[0092] Step 4: Define the ideal relative distance and ideal angle for each pose category in the standard library; input the image to be analyzed into the trained dual-branch fusion network model to obtain the similarity spectrum vector of the image to be analyzed; calculate the actual distance and actual angle between each key point in the image to be analyzed, select the N pose categories with the highest similarity in the similarity spectrum, and calculate the difference between the image to be analyzed and these pose categories respectively to determine whether a danger signal is issued.

[0093] In this embodiment, the specific logic for determining whether a danger signal should be issued is as follows:

[0094] The formula used to calculate the distance between any two key points is:

[0095]

[0096] in, Represents a node and nodes The distance between them and They are nodes exist direction and Components in direction, and They are nodes exist direction and Components in the direction, and ;

[0097] The angles formed by the head, neck, and left shoulder are calculated separately using the following formula:

[0098]

[0099]

[0100]

[0101] in, Indicates the distance from the head to the neck. This indicates the distance from the neck to the left shoulder. This indicates the distance from the head to the left shoulder. and The head node is located at direction and Components in direction, and The neck nodes are respectively direction and Components in direction, and The left shoulder node is respectively direction and Components in direction;

[0102] The formula for calculating angles using the law of cosines is:

[0103]

[0104] in, This indicates the angle formed by the head, neck, and left shoulder;

[0105] The angle formed by the head, neck, and right shoulder is calculated as follows:

[0106]

[0107]

[0108]

[0109]

[0110] in, Indicates the distance from the neck to the right shoulder. Indicates the distance from the head to the right shoulder. and The right shoulder node is located at direction and Components in direction, Indicates the angles of the head, neck, and right shoulder;

[0111] The angle formed by the left shoulder, left elbow, and left wrist is calculated as follows:

[0112]

[0113]

[0114]

[0115]

[0116] in, This indicates the distance from the left shoulder to the left elbow. This indicates the distance from the left elbow to the left wrist. This indicates the distance from the left shoulder to the left wrist. and The left shoulder node is respectively direction and Components in direction, and The left elbow node is located at direction and Components in direction, and The left wrist nodes are respectively direction and Components in direction, This indicates the angle formed by the left shoulder, left elbow, and left wrist;

[0117] The angle formed by the right shoulder, right elbow, and right wrist is calculated as follows:

[0118]

[0119]

[0120]

[0121]

[0122] in, This indicates the distance from the right shoulder to the right elbow. This indicates the distance from the right elbow to the right wrist. Indicates the distance from the right shoulder to the right wrist. and The right shoulder node is located at direction and Components in direction, and The right elbow node is located at direction and Components in direction, and The right wrist nodes are respectively direction and Components in direction, Indicates the angle formed by the right shoulder, right elbow, and right wrist;

[0123] The angle formed by the left hip, left knee, and left ankle is calculated as follows:

[0124]

[0125]

[0126]

[0127]

[0128] in, This indicates the distance from the left hip to the left knee. This indicates the distance from the left knee to the left ankle. This indicates the distance from the left hip to the left ankle. and The left hip node is located at direction and Components in direction, and The left knee node is located at direction and Components in direction, and The left ankle node is located at direction and Components in direction, This indicates the angle formed by the left hip, left knee, and left ankle;

[0129] The angle formed by the right hip, right knee, and right ankle is calculated as follows:

[0130]

[0131]

[0132]

[0133]

[0134] in, This indicates the distance from the right hip to the right knee. This indicates the distance from the right knee to the right ankle. This indicates the distance from the right hip to the right ankle. and The right hip node is located at direction and Components in direction, and The right knee node is located at direction and Components in direction, and The right ankle node is located at direction and Components in direction, Indicates the angle formed by the right hip, right knee, and right ankle;

[0135] The N pose categories with the highest similarity are selected from the similarity spectral vector. The difference between the pose in the image to be analyzed and its corresponding pose category in the standard library is calculated using the following formula:

[0136]

[0137] in, This represents the degree of difference between the image to be analyzed and the q-th pose category in the standard library. This represents the actual relative distance between the i-th node and the j-th node in the image to be analyzed. This represents the set of the m-th actual angles in the image to be analyzed. This represents the ideal relative distance between the i-th node and the j-th node in the q-th pose category of the standard library. This represents the m-th ideal angle set in the standard library, where m is the index of the angle set. These are the weight coefficients for the corresponding terms, and In most cases, priority should be given to ensuring... The fundamental reason is that the danger or abnormality of human posture is more reflected in the degree and direction of joint flexion than in the absolute extension or contraction of limb length. For example, the decisive feature for judging the typical dangerous posture of excessive lumbar flexion is the significant reduction in the angle between the torso (such as the line connecting the neck and hips) and the legs (i.e., excessive bending angle). Even if the person's height varies, resulting in differences in the absolute distance from the neck to the ankle, the flexion angle of the waist is the direct and stable indicator of whether it is "excessive". Similarly, judging whether "the arm has entered the danger zone" involves the concept of distance, but the extension posture of the arm (the angle of the shoulder and elbow joints) constitutes the basis of the movement. A stiffly straightened arm and a naturally bent arm, even if the wrist ultimately reaches the same spatial position, have different postural risks and muscle loads, and this difference is primarily captured by the angle.

[0138] This formula aims to calculate the average absolute error in spatial geometry between the pose to be evaluated and the q-th pose template in the standard library. It measures this error from two dimensions: distance difference and angle difference. The topological structure formed by keypoints is considered. By calculating the absolute differences in keypoint distances and joint angles between the pose to be analyzed and the standard pose template, it transforms the "formal similarity" of the pose into a mathematical "closeness." This comparison based on geometric physical quantities ensures that the judgment process does not rely on the internal black-box computation of the model, but is built upon intuitive and verifiable spatial relationships, greatly enhancing the transparency and credibility of the results.

[0139] The relationship between the parameters and the independent variable in the formula directly determines the sensitivity and focus of the discrimination. Dissimilarity As the dependent variable, its value is directly determined by two core independent variable groups: first, the deviation between the actual and ideal distances of all keypoint pairs; and second, the deviation between the actual and ideal angles of each joint. Any geometric deformation of the actual posture relative to the standard template will directly manifest as an increase in these deviations, thus leading to... The value increased. This means that, It is sensitive to even the slightest changes in posture. More importantly, the weighting coefficients... and As adjustable parameters, they act as a "decision lever." By adjusting their values, the judgment criteria can be flexibly customized: if the range of limb extension (distance) is considered more critical in a specific scenario, it can be increased. If the degree (angle) of joint flexion is considered the primary indicator of risk, then the risk level can be increased. This design allows the formula to adapt to the specific needs of different application scenarios, such as focusing on the spinal angle in handling operations, and focusing on the distance between limbs and dangerous areas in mechanical operations.

[0140] The fundamental rationale behind this difference-based judgment mechanism lies in its implementation of a dual, more flexible verification strategy. It cleverly utilizes the "probability distribution" (similarity spectrum) provided by the neural network in step 3 to filter out the most likely candidate categories, but the final decision is entrusted to more deterministic geometric rules. This design not only compensates for potential ambiguity in boundary cases that may arise with neural networks, but more importantly, it performs a macro-level camp-level judgment by calculating the overall difference with the "safe posture camp" and the "dangerous posture camp," rather than a simple "one-to-one" match. When the "dangerous difference" is significantly lower than the "safe difference," even if the posture's similarity to a dangerous template is not the highest, the system can still issue an alert based on its geometric characteristics. This provides a crucial safety redundancy for high-risk scenarios, minimizing false negatives.

[0141] The sum of the differences between the image to be analyzed and all first-class safe postures is calculated as the safe difference degree, and the sum of the differences between the image to be analyzed and all second-class dangerous postures is calculated as the dangerous difference degree. The magnitudes of the safe difference degree and the dangerous difference degree are determined. If the dangerous difference degree is high, the current posture in the image to be analyzed is determined to be a safe state, and the posture category corresponding to the first safe posture in the standard library with the smallest difference degree is regarded as the posture category of the image to be analyzed. If the safe difference degree is high, the current posture in the image to be analyzed is determined to be a dangerous state, and the posture category corresponding to the second dangerous posture in the standard library with the smallest difference degree is regarded as the posture category of the image to be analyzed, and a danger signal is issued.

[0142] Even after obtaining the similarity spectrum from the neural network output, the system still introduces difference metrics for final judgment. The core purpose is to establish a safer, more reliable, and interpretable decision-making mechanism. In posture recognition systems, especially in industrial safety monitoring, the primary task is to avoid missing genuine hazards. While neural network models are powerful, their decision-making process is relatively opaque and may produce misjudgments in certain boundary cases or rare postures not fully covered by training data. By calculating the geometric difference between a posture and standard safe and dangerous posture templates, a traceable judgment criterion based on explicit physical rules is introduced. If the network considers a posture safe, but the geometric difference (safety difference) between that posture and all safe templates is significantly greater than the difference (danger difference) with dangerous templates, the system will decisively classify it as dangerous and issue an alarm. This rule-based post-judgment logic adds a safety valve to the system, significantly reducing the risk of missed detections due to blindly overestimating model confidence, aligning with functional safety design principles.

[0143] This judgment mechanism enhances the ability to handle ambiguous and mixed postures. Real-world postures are often continuously changing and may simultaneously contain both safe and dangerous characteristics. Soft labeling and similarity spectra have already acknowledged this ambiguity. The difference judgment goes a step further; instead of simply selecting the single category with the highest similarity, it calculates the overall difference from both the safe and dangerous camps. By comparing the "sum of safe differences" with the "sum of dangerous differences," the system can assess, from a more macroscopic perspective, which camp the current posture leans towards. This "camp-based" comparison is more flexible than a binary single classification, better reflects the continuous spectrum characteristics of postures, and thus makes decisions that are more consistent with reality.

[0144] Please see Figure 2 The present invention also provides a posture recognition system based on a posture recognition neural network, which is used to implement the above-mentioned posture recognition method based on a posture recognition neural network, and includes:

[0145] The data acquisition and annotation module is used to acquire an image database containing multiple poses, construct a standard library containing K predefined pose categories, use annotation tools to annotate key points in each image and assign them to pose categories in the standard library to generate corresponding pose category label vectors, and preprocess the annotated images and integrate them into a sample dataset. The key points include head, neck, shoulder, elbow, wrist, hip, knee and ankle.

[0146] The graph structure construction module is used to define each labeled key point as a node of the graph, obtain the feature vector of each node, including the node position coordinates and visibility score, integrate the feature vectors of all nodes on each image into a feature matrix, and then establish the connection relationship of key points to form edges by connecting key points and obtain the adjacency matrix.

[0147] The model building module is used to construct a dual-branch fusion network model, including a graph convolution branch and a global vision branch. The feature matrix and adjacency matrix of the images in the sample dataset are input into the graph convolution branch, and the preprocessed image is input into the global vision branch. The model is trained with the pose category label vectors corresponding to the images in the sample dataset as labels, so that the model outputs a K-dimensional similarity spectrum vector, where each component represents the percentage of similarity between the image and the corresponding pose category in the standard library.

[0148] The pose evaluation module defines the ideal relative distance and ideal angle for each pose category in the standard library; it inputs the image to be analyzed into the trained dual-branch fusion network model to obtain the similarity spectrum vector of the image to be analyzed; it calculates the actual distance and actual angle between each key point in the image to be analyzed, selects the N pose categories with the highest similarity in the similarity spectrum, and calculates the difference between the image to be analyzed and these pose categories to determine whether a danger signal is issued.

[0149] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0150] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A pose recognition method based on a pose recognition neural network, characterized in that, The specific steps include: Step 1: Obtain an image database containing multiple poses and construct a standard library containing K predefined pose categories; use annotation tools to annotate the key points in each image and assign them to the pose categories in the standard library to generate corresponding pose category label vectors; preprocess the annotated images and integrate them into a sample dataset, where the key points include head, neck, shoulder, elbow, wrist, hip, knee, and ankle. Step 2: Define each labeled key point as a node in the graph, obtain the feature vector of each node, including node position coordinates and visibility score, integrate the feature vectors of all nodes on each image into a feature matrix, and then establish key point connection relationships to form edges by connecting key points, and obtain the adjacency matrix. Step 3: Construct a dual-branch fusion network model, including a graph convolution branch and a global vision branch. Input the feature matrix and adjacency matrix of the images in the sample dataset into the graph convolution branch, and input the preprocessed image into the global vision branch. Train the model with the pose category label vector corresponding to the image in the sample dataset as the label, so that the model outputs a K-dimensional similarity spectrum vector, where each component represents the percentage of similarity between the image and the corresponding pose category in the standard library. Step 4: Define the ideal relative distance and ideal angle for each pose category in the standard library; input the image to be analyzed into the trained dual-branch fusion network model to obtain the similarity spectrum vector of the image to be analyzed; calculate the actual distance and actual angle between each key point in the image to be analyzed, select the N pose categories with the highest similarity in the similarity spectrum, and calculate the difference between the image to be analyzed and these pose categories respectively to determine whether a danger signal is issued; The specific logic for obtaining the sample dataset is as follows: Image data containing various postures was downloaded from a public dataset. A standard library containing K predefined posture categories was constructed, and all posture categories were divided into two categories: Category I safe postures and Category II dangerous postures. Category I safe postures represent operating postures that comply with safety regulations, including at least standard upright observation, bending over and carrying, and operating at a safe distance. Category II dangerous postures represent typical violations and high-risk postures, including at least waist bending angles exceeding safety and thresholds, arms entering the mechanical danger zone, and loss of balance during high-altitude operations. For each image in the image database, at least 14 human keypoints are labeled, including the head, neck, shoulders, elbows, wrists, hips, knees, and ankles. Shoulders include left and right shoulders; elbows include left and right elbows; wrists include left and right wrists; hips include left and right hips; knees include left and right knees; and ankles include left and right ankles. Experts assign human poses in the images to multiple categories in a standard library, and a K-dimensional pose category label vector is generated using soft labeling, represented as follows: ,in, Represents the pose category label vector. This represents the confidence score indicating that the image belongs to the first pose category. This represents the confidence score indicating that the image belongs to the second pose category. This represents the confidence score that the image belongs to the q-th pose category. This represents the confidence score indicating that the image belongs to the Kth pose category; q is the pose category index in the standard library, and K is the number of pose categories in the standard library; the confidence score ranges from [0,1], and The labeled image is then normalized in size, and the image pixel values ​​are scaled to [size value missing]. Within the scope, the preprocessed images are integrated into a sample dataset; The specific criteria used to obtain the feature matrix and adjacency matrix include: If a node's keypoint is detectable in the image, its visibility score is defined as 1; if the corresponding keypoint is not detected in the image, its visibility score is defined as 0. The feature vector of each node is obtained as follows: ； in, Indicates the first The feature vector of each node Indicates the first The visibility score of each node, These represent the positions of the node on the X-axis and Y-axis, respectively. Indicates the node index; By integrating the feature vectors of all nodes in each image and stacking them sequentially, a graph structure feature matrix for the image is constructed, represented as: ； in, For the characteristic matrix, This represents the number of nodes, which is also the number of key points. ; Based on the normal physiological connection structure, the edges between nodes are defined, and an adjacency matrix is ​​constructed to represent the topological connection relationship of the graph. The node connection relationship is as follows: the head connects to the neck, the neck connects to the left shoulder, right shoulder, left hip and right hip respectively, the left shoulder connects to the left elbow, the left elbow connects to the left wrist, the right shoulder connects to the right elbow, the right elbow connects to the right wrist, the left hip connects to the left knee, the left knee connects to the left ankle, the right hip connects to the right knee, and the right knee connects to the right ankle. The adjacency matrix is ​​one The symmetric matrix is ​​represented as: ； in, It is an adjacency matrix. Represents a node and nodes The connection relationship between nodes, if nodes and nodes If there is a connection between them, then ,otherwise , It is also the index of the node; Constructing the dual-branch fusion network model specifically includes: The sample dataset is divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.

5. The dual-branch fusion network consists of a parallel graph convolution branch, a global vision branch, a feature fusion module, and an output layer. The graph convolution branch is an L-layer graph convolutional network. The feature matrix and adjacency matrix of the images in the training set are used as the output of this branch. After each graph convolutional layer performs neighborhood information aggregation, aggregation information normalization, feature transformation and nonlinear activation processing, the final feature vector of each node contains the structural information in its multi-hop neighborhood. Global average pooling is applied to the final feature vector of all nodes to obtain the feature vector representing the global structure of the image. The global vision branch is used to extract global contextual features from images in the training set. The images in the training set are input into the network, and the feature map before the last fully connected layer is taken. After passing through a global average pooling layer, a global vision feature vector is obtained. A feature fusion module is used to input the feature necklaces of the global structure from the two branches and the global visual feature vectors into a fully connected layer for dimensionality reduction and nonlinear transformation; then, a K-dimensional vector is output through the output layer, where each component of the vector represents the percentage of similarity with the corresponding pose category in the standard library; the output layer consists of a K-dimensional fully connected layer and a Softmax activation function; The model uses pose category label vectors as supervision signals and multi-class cross-entropy as the loss function to evaluate the difference between the model output and the true value. The Adam optimizer is used to minimize the loss function and iteratively update all parameters in the model. When the change in the loss function on the validation set is less than a preset threshold in multiple consecutive iterations, the model is considered to have completed training.

2. The pose recognition method based on a pose recognition neural network according to claim 1, characterized in that, The specific logic for determining whether to issue a danger signal is as follows: Define the ideal relative distance between all keypoint pairs in each image in the standard library and the ideal set of angles formed by three keypoints. The set of angles includes: head-neck-left shoulder, head-neck-right shoulder, left shoulder-left elbow-left wrist, right shoulder-right elbow-right wrist, left hip-left knee-left wrist, and right hip-right knee-right wrist. The image to be analyzed is input into the trained dual-branch fusion network model to obtain the similarity spectrum vector. Based on the coordinates of key points in the image, the Euclidean distance between all key point pairs is calculated as the actual distance. Then, the cosine theorem is used to calculate the actual angle values ​​of all angles in the angle set. The calculation formula is as follows: ； in, This indicates the angle formed by the head, neck, and left shoulder; Similarly, the actual angles formed between the head-neck-right shoulder, left shoulder-left elbow-left wrist, right shoulder-right elbow-right wrist, left hip-left knee-left ankle, and right hip-right knee-right ankle are obtained and represented as follows: ; The N pose categories with the highest similarity are selected from the similarity spectral vector. The difference between the pose in the image to be analyzed and its corresponding pose category in the standard library is calculated using the following formula: ； in, This represents the degree of difference between the image to be analyzed and the q-th pose category in the standard library. This represents the actual relative distance between the i-th node and the j-th node in the image to be analyzed. This represents the set of the m-th actual angles in the image to be analyzed. This represents the ideal relative distance between the i-th node and the j-th node in the q-th pose category of the standard library. This represents the m-th ideal angle set in the standard library, where m is the index of the angle set. These are the weight coefficients for the corresponding terms, and ; The sum of the differences between the image to be analyzed and all first-class safe postures is calculated as the safe difference degree, and the sum of the differences between the image to be analyzed and all second-class dangerous postures is calculated as the dangerous difference degree. The magnitudes of the safe difference degree and the dangerous difference degree are determined. If the dangerous difference degree is high, the current posture in the image to be analyzed is determined to be a safe state, and the posture category corresponding to the first safe posture in the standard library with the smallest difference degree is regarded as the posture category of the image to be analyzed. If the safe difference degree is high, the current posture in the image to be analyzed is determined to be a dangerous state, and the posture category corresponding to the second dangerous posture in the standard library with the smallest difference degree is regarded as the posture category of the image to be analyzed, and a danger signal is issued.

3. A posture recognition system based on a posture recognition neural network, characterized in that, The posture recognition system based on a posture recognition neural network is used to implement the posture recognition method based on a posture recognition neural network as described in any one of claims 1-2, comprising: The data acquisition and annotation module is used to acquire an image database containing multiple poses, construct a standard library containing K predefined pose categories, use annotation tools to annotate key points in each image and assign them to pose categories in the standard library to generate corresponding pose category label vectors, and preprocess the annotated images and integrate them into a sample dataset. The key points include head, neck, shoulder, elbow, wrist, hip, knee and ankle. The graph structure construction module is used to define each labeled key point as a node of the graph, obtain the feature vector of each node, including the node position coordinates and visibility score, integrate the feature vectors of all nodes on each image into a feature matrix, and then establish the connection relationship of key points to form edges by connecting key points and obtain the adjacency matrix. The model building module is used to construct a dual-branch fusion network model, including a graph convolution branch and a global vision branch. The feature matrix and adjacency matrix of the images in the sample dataset are input into the graph convolution branch, and the preprocessed image is input into the global vision branch. The model is trained with the pose category label vectors corresponding to the images in the sample dataset as labels, so that the model outputs a K-dimensional similarity spectrum vector, where each component represents the percentage of similarity between the image and the corresponding pose category in the standard library. The pose evaluation module defines the ideal relative distance and ideal angle for each pose category in the standard library; it inputs the image to be analyzed into the trained dual-branch fusion network model to obtain the similarity spectrum vector of the image to be analyzed; it calculates the actual distance and actual angle between each key point in the image to be analyzed, selects the N pose categories with the highest similarity in the similarity spectrum, and calculates the difference between the image to be analyzed and these pose categories to determine whether a danger signal is issued.

Images