Chronic pain protection behavior detection method based on hierarchical weighted graph convolution model

Abstract

The invention relates to the field of behavior detection, in particular to a chronic pain protection behavior detection method based on a hierarchical weighted graph convolution model. The method comprises the following steps: acquiring a training set, wherein the training set comprises joint movement data of a plurality of healthy individuals and a plurality of chronic lower back pain individuals in the process of completing daily life tasks; preprocessing the data in the training set, and converting the data into three-dimensional time sequence skeleton data; training a detection model by using the preprocessed training set; the detection model converts the human skeleton image into a layered image structure, and features are extracted by using an image convolutional network; introducing a category-based weighting mechanism, and optimizing a classifier and a weighting network through iterative adversarial training; and after the joint movement data of an individual to be detected in the process of completing the daily life task is preprocessed, the trained detection model is input for detection. According to the method, the receptive field is expanded through a layered structure, the category imbalance problem is optimized through a reweighting mechanism, and the robustness and accuracy of detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of chronic pain protective behavior detection, and particularly to a chronic pain protective behavior detection method based on a hierarchical weighted graph convolutional model. Background Art

[0002] Chronic Pain (CP) is a serious health problem affecting millions of people worldwide, especially Chronic Low Back Pain (CLBP). Clinically, patients often adopt a series of protective behaviors (such as avoiding certain movements or postures) to relieve pain. However, long-term protective behaviors may lead to functional disorders and even exacerbate pain symptoms. Traditional rehabilitation treatments rely on manual observation of patients' movements, but this method cannot monitor the behavior changes of patients in daily life in real time and continuously.

[0003] In recent years, with the development of wearable sensor technology, the skeletal data obtained by sensors has provided new opportunities for the detection of protective behaviors. Methods based on Graph Convolutional Networks (GCN) have been successfully applied to action recognition tasks, but most existing GCN models only focus on the relationships of adjacent nodes, ignoring the interactions of distant joint nodes, resulting in limited modeling ability for complex action patterns. In addition, the problem of class imbalance is also very prominent in the detection of protective behaviors, resulting in the model being unable to accurately identify samples of the minority class (such as protective behaviors).

[0004] Therefore, there is an urgent need for a technology that can automatically detect protective behaviors and solve the above problems. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a chronic pain protective behavior detection method based on a hierarchical weighted graph convolutional model, which can make full use of the spatial and temporal features in the skeletal data, expand the receptive field through a hierarchical structure, capture the complex interactions between distant joints, and optimize the class imbalance problem through a reweighting mechanism, improving the robustness and accuracy of the detection.

[0006] To solve the above technical problems, the technical solution of the present invention is: a chronic pain protective behavior detection method based on a hierarchical weighted graph convolutional model, including:

[0007] Obtain a training data set, where the training data set includes joint movement data of multiple healthy individuals and multiple chronic low back pain individuals during the process of completing daily life tasks;

[0008] Preprocess the data in the training data set and convert it into three-dimensional time series skeletal data;

[0009] The detection model is trained using the preprocessed training dataset to obtain a trained detection model; wherein, the detection model converts the human body bone map of three-dimensional time series bone data into a hierarchical graph structure, expands the receptive field, and extracts features using a graph convolutional network; a category-based weighting mechanism is also introduced, and the classifier and the weighting network are optimized through iterative adversarial training;

[0010] After preprocessing the joint movement data of the individual to be detected during the process of completing daily life tasks, the data is input into the trained detection model to detect whether there are chronic pain protection behaviors.

[0011] Furthermore, the data preprocessing is specifically as follows:

[0012] The continuously captured joint movement data is segmented by a sliding window to obtain multiple segments of time series bone data; wherein, a preset overlapping ratio is introduced during the segmentation process;

[0013] Each segment of time series bone data is converted into three-dimensional time series bone data.

[0014] Furthermore, the strategy for converting the human body bone map of three-dimensional time series bone data into a hierarchical graph structure is as follows:

[0015] First, the human body bone map is decomposed into a rooted tree through the natural connection edges of the human body. The abdominal node and the nodes adjacent to the abdominal node together form the first-level set of the graph decomposition. The nodes adjacent to the nodes in the first-level set together form the second-level set of the graph decomposition, and so on, gradually pushing the graph decomposition outward layer by layer. The nodes in the same-level set are divided into the same semantic space;

[0016] Then, undirected connection edge subgraphs are constructed in adjacent semantic spaces to form the relationship between layers;

[0017] Full connection edges are introduced between any two adjacent layers and between any two separated layers respectively to expand the receptive field.

[0018] Furthermore, after the decomposition of the human body bone map, there are a total of N L semantic spaces. Undirected connection edge subgraphs are constructed in adjacent semantic spaces and divided into N H = N L -1 sub-layers. The adjacency matrix of the sub-layers is expressed as:

[0019]

[0020] represents a group of edges from H K to H K+1 . Considering the identity edges of each layer node set, the adjacency matrix is defined as:

[0021]

[0022] Wherein, ε k is for S = {s id , s ue}, s id represents the equilateral subset of self-connection of each node, and s ue represents the undirected edge subset of the actual interaction relationship between adjacent nodes.

[0023] Furthermore, the graph convolutional network is used to extract features, which is expressed as:

[0024]

[0025] Wherein, represents the output feature map of the k-th layer of graph convolution, and the function Φ represents the linear transformation of the parameter , || represents the concatenation operation, represents the adjacency matrix of the k-th sublayer; represents the point convolution operation of the k-th sublayer, and F in represents the input feature map.

[0026] Furthermore, the detection model introduces a category-based weighting mechanism to optimize the classifier and the weighting network through iterative adversarial training; specifically:

[0027] A weighting network is introduced in the detection model. For each category, an independent weighting network is used to perform weighted learning on the samples of this category;

[0028] In the end-to-end adversarial training process, an adversarial mechanism between the classifier network and the weighting network is constructed. The weighted cross-entropy is used as the loss function of the classifier network, and the weighting network dynamically provides sample weights for the classifier. The classifier network and the weighting network adopt the strategy of two-player min-max adversarial game in the framework of adversarial game to optimize the objective function respectively; wherein, the objective function is:

[0029]

[0030] Wherein, is the total number of labeled samples in category c; |C| represents the total number of categories; represents the true label of sample i for category c (usually represented by one-hot, that is, taking values of 0 or 1); the predicted probability of sample i on category c; w i represents the output of the weighting network corresponding to sample i.

[0031] Furthermore, in the training strategy, a multi-step update strategy is adopted, that is, the classifier network is updated m times first, and then the weighted network is updated once.

[0032] After adopting the above technical solution, the present invention can automatically detect the protective behaviors of patients in daily life and effectively solve the class imbalance problem in biomedical data by combining the graph convolutional network (GCN) with the hierarchical decomposition graph (HD-Graph) structure and the reweighted adversarial mechanism. The invention has wide applications in chronic pain management, rehabilitation intervention and personalized pain treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flowchart of the chronic pain protective behavior detection method based on the hierarchical weighted graph convolutional model of the present invention;

[0034] Figure 2 is an iterative decomposition schematic diagram of the hierarchical graph convolutional network HD-Graph;

[0035] Figure 3 is a framework diagram of the detection model HWGCN of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] In order to make the content of the present invention easier to be clearly understood, the present invention will be further described in detail below according to specific embodiments in conjunction with the accompanying drawings.

[0037] As Figure 1 、 Figure 2 and Figure 3 shown, a chronic pain protective behavior detection method based on a hierarchical weighted graph convolutional model includes:

[0038] Step S1, obtaining a training data set, which includes joint movement data of multiple healthy individuals and multiple individuals with chronic low back pain during the process of completing daily life tasks.

[0039] Among them, these tasks include bending, standing on one foot, sitting up, standing up and sitting down, and reaching forward movements, etc. Data collection is recorded using three-dimensional inertial measurement units (IMUs) distributed throughout the body. Since the model proposed in this study operates on graphs with similar skeletal structures, we directly select the 3D coordinate movement data of 22 joints of 28 healthy individuals and 18 CLBP patients as the training data set.

[0040] Step S2, preprocessing the data in the training data set and converting it into three-dimensional time series skeletal data;

[0041] Step S3: Train the detection model using the preprocessed training dataset to obtain a trained detection model. Among them, the detection model converts the human body bone diagram of three-dimensional time series bone data into a hierarchical graph structure, expands the receptive field, and uses a graph convolutional network to extract features. A category-based weighting mechanism is also introduced to optimize the classifier and the weighting network through iterative adversarial training.

[0042] Step S4: After preprocessing the joint movement data of the individual to be detected during the process of completing daily life tasks, input it into the trained detection model to detect whether there are chronic pain protection behaviors.

[0043] In this embodiment, in step S2 and step S4, the data preprocessing is specifically as follows:

[0044] Segment the continuously captured joint movement data using a sliding window to obtain multiple segments of time series bone data. Among them, a preset overlap ratio is introduced during the segmentation process.

[0045] Convert each segment of time series bone data into three-dimensional time series bone data.

[0046] More specifically, the original data stream is segmented into 3-second windows (i.e., 180 time steps) at a sampling rate of 60 Hz, and a 50% overlap ratio is introduced during the segmentation process. Each window represents a segment of time series data. This time series data has rows of time frames and columns of data collected by multiple sensors. The human body bone diagram has 22 nodes, so there are 22 columns of data. After appropriate preprocessing, this data is converted into three-dimensional time series bone data for subsequent graph convolutional network (GCN) processing. Here, the conversion only changes the dimension of the data, arranging the column data according to the distribution of human nodes to form three-dimensional data.

[0047] In this embodiment, as Figure 2 shown, the strategy for converting the human body bone diagram of three-dimensional time series bone data into a hierarchical graph structure is as follows:

[0048] First, decompose the human body bone diagram into a rooted tree through the natural connection edges of the human body. As Figure 2 shown, there are 22 nodes in total, namely the abdomen, hip, right hip, right knee, right ankle, left hip, left knee, left ankle, chest, left shoulder, left arm, left elbow, left wrist, left hand, right shoulder, right arm, right elbow, right wrist, right hand, neck, head, and top of the head. The abdomen node is used as the centroid node. The abdomen node and the nodes adjacent to the abdomen node together form the first-level set of graph decomposition. The nodes adjacent to the nodes in the first-level set together form the second-level set of graph decomposition, and so on, gradually pushing the graph decomposition outward. The nodes in the same-level set are divided into the same semantic space. After decomposition, there are N L semantic spaces;

[0049] Then, construct an undirected connected edge subgraph in the adjacent semantic space to form the relationship between levels. A total of N H = N L - 1 sub - layers can be divided. The adjacency matrix of the sub - layer is expressed as:

[0050]

[0051] represents a set of edges from H K to Hk +1 . To obtain the characteristics of the node itself, the identity edges of each level node set must be considered. Therefore, the adjacency matrix is defined as:

[0052]

[0053] In the formula, ε k is for S = {s id , s ue}, where s id represents the subset of identity edges of each node's self - connection, and s ue represents the subset of undirected edges of the actual interaction relationship between adjacent nodes. It should be noted that V represents the number of nodes of the human body, and N S represents the number of subsets, which is 2 here; is the adjacency matrix representation containing only undirected connected edges; is the adjacency matrix representation with self - loop identity edges added.

[0054] Then, introduce fully - connected edges to enhance the node relationship between different levels and capture the joint node interaction across levels and regions: To more effectively identify the relationship between distant nodes in the same semantic space, adopt the strategy of connecting all nodes in the adjacent - level node sets to each other. At the same time, introduce a fully - connected operation between the separated semantic spaces to perform a full connection on the distant - level node sets.

[0055] Specifically, introduce fully - connected edges to make the connection of the graph tighter and greatly expand the receptive field. Next, to ensure the stability of training, normalize the adjacency matrix and the degree matrix (the degree refers to the number of other nodes connected by a node, and the degree matrix is the degree matrix obtained according to the degree of each node, which is used in the graph convolution operation and is part of the convolution formula). To further improve the adaptability of the model to data, set all elements in the matrix as learnable parameters so that the model can dynamically adjust the edge weights during the training process.

[0056] In this embodiment, the graph convolution (HD-Graph convolution) includes two parallel branch operations, two graph convolutions performed through the HD-Graph. To reduce the computational complexity, linear transformations are performed on both operations, and the subset GCN operation is executed on each hierarchical edge set with two edge subsets. However, instead of adding the output values of each subset, these output values are concatenated in the channel dimension. Feature extraction is performed using a graph convolutional network, expressed as:

[0057]

[0058] Due to the inherent characteristics of the skeletal data, the number of joint nodes contained in each hierarchical set is different. Therefore, an addition strategy is adopted for the outputs of N H sub-layers.

[0059]

[0060] In the formula, represents the output feature map of the k-th layer of the hierarchical graph HD-graph convolution, the function Φ represents the linear transformation of the parameter , || represents the concatenation operation, represents the adjacency matrix of the k-th sub-layer; represents the point convolution operation of the k-th sub-layer, and F in represents the input feature map.

[0061] The weighted cross-entropy loss function is a commonly used objective function for imbalanced data classification, usually completed by weighting the samples of each class to be proportional to the reciprocal of the class frequency. However, the class weighting method assigns the same weight to misclassified samples and samples that have been correctly classified.

[0062] To solve this problem in this embodiment, the weighting strategy is improved: the sample weights are dynamically adjusted during the training process. When a sample is misclassified, the weight it obtains relative to the correctly classified samples will increase. Through this dynamic weighting strategy, the classifier pays more attention to misclassified samples during the training process, thereby more effectively correcting the deviation of the decision boundary between the minority class and the majority class.

[0063] More specifically, in the model proposed in this embodiment, the classifier network is based on the HD-Graph structure and uses a graph convolutional network (GCN) to learn and classify features. Different from traditional node classification tasks, the goal of this study is to classify a complete time-series data sequence. In this context, GCN extracts spatio-temporal features between nodes by performing convolution on each joint node in the human skeletal data, and finally gives a classification result for the entire batch of data. To obtain a classification prediction in the form of a probability distribution, the softmax activation function is used in the last layer of the classifier network. To address the problem of class imbalance, this embodiment also introduces a weighted network into the model. For each class, there is an independent GCN (i.e., W-GCN) to perform weighted learning on the samples of that class. The weighted network takes the spatio-temporal features extracted from the samples as input and assigns appropriate weight values to each sample according to the learned weight distribution. To ensure that the sample weights are within a reasonable range and the weights of different classes are comparable, the sample weights of each class are normalized by the softmax function, thus maintaining the relative stability and interpretability of the weights.

[0064] That is, in the end-to-end adversarial training process of this embodiment, an adversarial mechanism between the classifier network and the weighted network is constructed to highlight the importance of misclassified samples and correctly classified samples with insufficient confidence. By using weighted cross-entropy as the loss function of the classifier and having the weighted network dynamically provide sample weights for the classifier network, the classifier and the weighted network are continuously optimized within the framework of an adversarial game. Using the strategy of a two-player min-max adversarial game, the classifier and the weighted network respectively optimize the following objectives:

[0065]

[0066] In the formula, is the total number of labeled samples in class c; |C| represents the total number of classes; The true label of sample i for class c (usually represented in one-hot form, i.e., taking values of 0 or 1); represents the predicted probability of sample i for class c; w i represents the output of the weighted network corresponding to sample i.

[0067] What is described here is an adversarial training process: the goal of the classifier network is to minimize its objective function, while the goal of the weighted network is to maximize this objective function. During the training process, the classifier attempts to correctly classify the training samples (especially high-weight samples) to minimize the loss function; while the weighted network adjusts the parameters to increase the weights of those challenging samples (such as samples that are not correctly classified or have low classification confidence), thereby promoting the maximization of the objective function.

[0068] In summary, in the model of this embodiment, in addition to the GCN-based classifier, a class-specific GCN-based weighted network is introduced to assign a weight distribution to the training samples of the corresponding class. These weights will be used in the weighted cross-entropy loss function. By imposing different degrees of penalty on the samples, the objective function can specifically guide the classifier to make more accurate predictions for high-weight samples.

[0069] In addition, in this embodiment, a multi-step update scheme is adopted in the training strategy: first, the classifier network is updated m times, and then the weighted network is updated once. This can avoid the high optimization cost caused by overly frequent parameter updates and ensure the stability of the adversarial game process. As the training continues, the game relationship between the classifier and the weighted network gradually reaches a balance.

[0070] Next, combined with different experiments, the advantages of the technical solutions involved in the above embodiments are verified.

[0071] 1. Performance Analysis of Hierarchical Graph Convolutional Network

[0072] To verify the performance improvement of hierarchical graph convolution (HD-GCN) compared to traditional PC-GCN in protective behavior detection, we conducted the following comparative experiments. The specific experimental results are shown in Table 1. We used the average F1 score and the area under the curve (AUC) as the main metrics. The average F1 score (denoted as Fm) balances the precision and recall, providing an overall view of the precision of the positive and negative class models. The methods included in this analysis will be described below:

[0073] PC-GCN: Only uses the natural human connection as the adjacency matrix, based on the standard graph convolutional network architecture. HD-GCN: Decomposes the natural human connection into a hierarchical graph structure and introduces fully connected edges.

[0074] Table 1

[0075]

[0076] As can be seen from Table 1, HD-GCN has a significant improvement over traditional PC-GCN in all evaluation metrics, with the Fm increasing by about 7.0% and the AUC increasing by about 11.0%. This indicates that hierarchical graph convolution can more effectively capture complex joint interaction relationships, especially the long-distance interactions between different semantic spaces, which helps to improve the detection accuracy of the model for protective behaviors.

[0077] 2. Performance Analysis of Reweighted Adversarial Network

[0078] To further improve the model's performance on class-imbalanced datasets, a reweighted adversarial network is introduced. This mechanism dynamically adjusts the weights of samples from different classes, focusing on the difficult-to-classify minority-class samples, thereby enhancing the model's ability to detect protective behaviors. Here, it will be compared with the traditional weighted cross-entropy method to analyze the performance improvement of the reweighted adversarial network.

[0079] Table 2

[0080]

[0081] As shown in Table 2, the reweighted adversarial network is significantly superior to the traditional weighted cross-entropy in terms of both the mean AUC and the mean F1 score. This indicates that the reweighted adversarial network significantly improves the F1 score, showing a better balance in terms of both precision and recall. In addition, the improvement in the AUC score of the reweighted adversarial network further demonstrates its superiority in distinguishing positive and negative class samples. Therefore, our method is more effective in dealing with class imbalance problems and can more accurately identify minority-class (protective behavior) samples.

[0082] 3. Comparison with Advanced Methods

[0083] To further evaluate the effectiveness of our detection model HWGCN, we compare it with a variety of state-of-the-art methods. These methods include Stacked-LSTM, Dual-Stream LSTM, BANet, etc., and the performance of these models on the EmoPain dataset has been reported in previous studies.

[0084] Table 3

[0085]

[0086] Table 3 shows the performance comparison results of HWGCN and the above methods in terms of two metrics, Fm and AUC, and HWGCN obtains the highest scores in terms of AUC and Fm. Specifically:

[0087] Compared with LSTM-based models that only focus on temporal features, HWGCN effectively integrates spatial and semantic hierarchical information with the help of HD-Graph and has more advantages in the recognition of complex human motion patterns;

[0088] Compared with BANet that incorporates an attention mechanism, HWGCN dynamically adjusts sample weights through a reweighted adversarial training mechanism, solves the model bias problem caused by class imbalance, and further improves the recognition accuracy of minority classes.

[0089] Taking the ideal embodiments of the present invention described above as an inspiration, through the above description, relevant staff can completely make various changes and modifications without departing from the technical idea of this invention. The technical scope of this invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for detecting chronic pain protection behavior based on a hierarchical weighted graph convolutional model, characterized in that: It includes: Obtain a training data set, which includes joint movement data of multiple healthy individuals and multiple individuals with chronic low back pain during the process of completing daily life tasks; Preprocess the data in the training data set and convert it into three-dimensional time series skeletal data; Use the preprocessed training data set to train the detection model to obtain a trained detection model; wherein, the detection model converts the human skeletal graph of the three-dimensional time series skeletal data into a hierarchical graph structure, expands the receptive field, and uses a graph convolutional network to extract features; A category-based weighting mechanism is also introduced to optimize the classifier and the weighting network through iterative adversarial training; After preprocessing the joint movement data of the individual to be detected during the process of completing daily life tasks, input it into the trained detection model to detect whether there is chronic pain protection behavior.

2. The method for detecting chronic pain protection behavior based on a hierarchical weighted graph convolutional model according to claim 1, characterized in that: The preprocessing of the data is specifically: Segment the continuously captured joint movement data by a sliding window to obtain multiple segments of time series skeletal data; wherein, a preset overlap ratio is introduced during the segmentation process; Convert each segment of time series skeletal data into three-dimensional time series skeletal data.

3. The method for detecting chronic pain protection behavior based on a hierarchical weighted graph convolutional model according to claim 1, characterized in that: The strategy for converting the human skeletal graph of the three-dimensional time series skeletal data into a hierarchical graph structure is: First, decompose the human skeletal graph into a rooted tree through the natural connection edges of the human body, form the first-level set of the graph decomposition by the abdominal node and the nodes adjacent to the abdominal node, and form the second-level set of the graph decomposition by the nodes adjacent to the nodes in the first-level set, and so on Push the decomposition of the graph layer by layer, and divide the nodes in the same-level set into the same semantic space; Then, construct an undirected connection edge subgraph in adjacent semantic spaces to form the relationship between levels; Then, fully connected edges are introduced between any two adjacent levels and any two separated levels respectively to expand the receptive field.

4. The method for detecting chronic pain protection behavior based on a hierarchical weighted graph convolutional model according to claim 3, characterized in that: After the human body bone diagram is decomposed, there are N in total L semantic spaces. An undirected connected edge subgraph is constructed in adjacent semantic spaces and divided into N H = N L - 1 sublayers. The adjacency matrix of the sublayers is expressed as: Denote a set of edges from H K to H K+1 Considering the identity edges of each hierarchical node set, the adjacency matrix is defined as: where ε k is for S = {s id , s ue}, s id represents the equilateral subset of each node's self-connection, s ue represents the undirected edge subset of the actual interaction relationship between adjacent nodes, V represents the number of nodes, NS represents the number of subsets.

5. The method for detecting chronic pain protection behavior based on a hierarchical weighted graph convolutional model according to claim 4, characterized in that: The extraction of features using a graph convolutional network is expressed as: wherein, represents the output feature map of the k-th layer of graph convolution, and the function Φ represents the linear transformation of the parameter , || represents the concatenation operation, represents the adjacency matrix of the k-th sublayer; represents the point convolution operation of the k-th sublayer, and F in represents the input feature map.

6. The method for detecting chronic pain protection behavior based on a hierarchical weighted graph convolutional model according to claim 1, characterized in that: The detection model introduces a category-based weighting mechanism to optimize the classifier and the weighting network through iterative adversarial training; specifically: Introduce a weighting network in the detection model, and for each category, an independent weighting network is used to perform weighted learning on the samples of this category; During the end-to-end adversarial training process, an adversarial mechanism between the classifier network and the weighting network is constructed. The weighted cross-entropy is used as the loss function of the classifier network, and the weighting network dynamically provides sample weights for the classifier. Under the framework of the adversarial game, the classifier network and the weighting network adopt the strategy of two-player min-max adversarial game to optimize the objective function respectively. Among them, the objective function is: Wherein, is the total number of labeled samples in category c; |C| represents the total number of categories; represents the true label of sample i for category c; the predicted probability of sample i on category c; w i represents the output of the weighted network corresponding to sample i.

7. The chronic pain protection behavior detection method based on the hierarchical weighted graph convolution model according to claim 6, characterized in that In the training strategy, a multi-step update strategy is adopted, that is, the classifier network is updated m times first, and then the weighting network is updated once.

Images