A lightweight sit-up detection method based on pose estimation

Abstract

The present application relates to a kind of lightweight sit-up detection method based on pose estimation, belong to computer vision field.The method includes: S1: collection sit-up image data, pre-processing, mark human body joint coordinates, obtain data set;S2: construct sit-up detection model, including lightweight human skeleton feature extraction network and attention decoupling coordinate classification prediction module;S3: data set is input to model and is trained, square error loss function is used in the process of model training to supervise training process;S4: the sit-up image sequence to be detected is input to the model trained, obtains the coordinate information of human body joint, then using pose estimation module obtains human posture, and then according to the posture change in image sequence realizes the sit-up counting function of target test personnel and action norm detection function.The present application can improve the accuracy of sit-up detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and relates to a lightweight sit-up detection method based on pose estimation. BACKGROUND

[0002] Sit-up plays an important role in physical fitness testing, mainly used to evaluate the abdominal core muscle strength of individuals, which is crucial for maintaining body stability, improving sports performance and preventing waist injuries. In physical fitness testing, sit-up as a simple and effective test method is widely used in people of different ages and physical fitness levels. Traditional sit-up testing often relies on manual scoring or professional instruments, which has problems of inaccurate evaluation, high cost and poor real-time performance.

[0003] Human pose estimation (HPE) is a fundamental and challenging task in the field of computer vision, which is essential for describing human poses and human behaviors. Currently, pose estimation technology has made significant progress in human motion analysis, virtual reality, intelligent monitoring and other fields. Therefore, the sit-up detection algorithm based on pose estimation becomes a potential improvement scheme for physical testing. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a lightweight sit-up detection method based on pose estimation, which can not only realize automatic and accurate testing, but also improve the real-time performance of testing, and provide more scientific and reliable data support for physical education and health management.

[0005] To achieve the above purpose, the present application provides the following technical solutions:

[0006] A lightweight sit-up detection method based on pose estimation, specifically comprising the following steps:

[0007] S1: Collecting sit-up image data, preprocessing, labeling human joint coordinates, and obtaining a data set for sit-up detection model training;

[0008] S2: Constructing a sit-up detection model, including a lightweight human skeleton feature extraction network and an attention decoupling coordinate classification prediction module;

[0009] The lightweight human skeleton feature extraction network is constructed based on a large-core separable convolution module, and a four-stage multi-scale information fusion is adopted to obtain a high-resolution feature map;

[0010] The attention decoupling coordinate classification prediction module is used to predict the coordinate position of the human joint;

[0011] S3: input the dataset into the sit-up detection model for training, and use the mean square error loss function to supervise the training process during the model training process;

[0012] S4: input the image sequence to be detected into the trained sit-up detection model, obtain the coordinate information of the human body joint points, then use the pose estimation module to obtain the human body pose, and then realize the sit-up counting function and the action standardization detection function of the target test personnel according to the pose change in the image sequence.

[0013] Further, in step S2, the general human skeleton feature extraction network usually reduces the resolution of the image step by step in the feature extraction process. This resolution reduction strategy helps to reduce the amount of calculation and memory consumption, but also causes the loss of detailed information. The light human skeleton feature extraction network includes four stages: the first stage is composed of 4 residual modules in cascade, and after processing, the feature map resolution is reduced to 1 / 4 of the original image; the second, third and fourth stages add a parallel branch in turn, and the added branch is 1 / 2 of the minimum resolution branch of the previous stage; the second stage contains two branches, each branch is composed of 3 large-core separable convolution modules in cascade; the third stage contains three branches, respectively composed of 3, 3 and 12 large-core separable convolution modules; the fourth stage contains four branches, respectively composed of 3, 3, 12 and 3 large-core separable convolution modules; at the same time, except for the first stage, each stage fuses multiple parallel branches, and outputs the feature map of the highest resolution branch in the fourth stage. The detailed information of the high-resolution feature is retained, and the global information of the low-resolution feature is also integrated, which improves the overall feature expression ability of the model. In addition, the large-core separable convolution module can significantly reduce the amount of calculation and the number of parameters while maintaining the performance of the model.

[0014] Further, in step S2, the residual module is specifically: the input first passes through a convolution layer with a convolution kernel size of 3x3, and then flows through a batch normalization layer for standardization processing; then, the input and the output of the second convolution layer are added to form a jump connection after passing through a second convolution layer with a convolution kernel size of 3x3 and a batch normalization layer; and finally, a ReLU activation function is used. The jump connection retains the feature information of the input and adds it to the output of the convolution layer, so that the residual module is more likely to learn and recognize small feature changes, making the network converge faster, and also enabling the model to extract features more finely without losing information. The output of the residual module can be expressed as:

[0015] Output=δ(B2(Conv2(B1(Conv1(x)))+x)

[0016] Wherein, xInput, B1, B2 represent two batch normalization layers, Conv1, Conv2 represent two convolution layers, and δ represents a ReLU activation function.

[0017] In a standard convolution operation, each output channel is obtained by convolving and summing all input channels. In a deep convolution, each input channel is only convolved with its corresponding convolution kernel. For an input feature map of size HxWxC i , a convolution kernel of size KxKxC i , and an output feature map of size HxWxC o , the parameter and computational complexity of the standard convolution operation are as follows, ignoring the bias induction:

[0018] Params = KxKxC i xC o = K 2 xC i xC o

[0019] Flops = KxKxC i xC o xHxW = K 2 xC i xC o HW

[0020] Further, in step S2, the large kernel separable convolution module is specifically: a large kernel convolution is decomposed into multiple deep convolutions and deep dilated convolutions, which can significantly reduce the computational complexity and parameter quantity while retaining a larger effective receptive field and feature extraction capability. The specific decomposition process is: an ordinary convolution with a kernel size of kxk is decomposed into a 1x(2d-1) horizontal deep convolution and a (2d-1)x1 vertical deep convolution, a 1x(k / d) horizontal deep dilated convolution and a (k / d)x1 vertical deep dilated convolution, and a 1x1 pointwise convolution. The input and the output of the 1x1 pointwise convolution are added and output after passing through a ReLU activation function. At this time, the parameter quantity and computational complexity of the large kernel separable convolution module are as follows:

[0021]

[0022] Given the input feature map The output feature of the large kernel separable convolution module is represented as:

[0023]

[0024] Where * and represent convolution operation and Hadamard product, respectively; DW represents the output after horizontal and vertical deep convolution; K represents the convolution kernel, and specifically, K1×1 representing a point-wise kernel of 1x1; d represents the dilation rate; DW_D represents the output after horizontal and vertical depthwise dilated convolution, representing the final output of the large kernel separable convolution feature extraction unit, H, W, C represent the height, width and channel number of the feature map respectively.

[0025] Further, in step S2, the attention decoupling coordinate classification prediction module includes an attention feature fusion module and a decoupling coordinate classification prediction module; first, the attention feature fusion module fuses the deep-level features extracted in stage four and the shallow-level features extracted in stage one, providing more comprehensive feature expression and improving the accuracy of the model in joint point positioning; then the decoupling coordinate classification prediction module decomposes the two-dimensional coordinate classification task into one-dimensional coordinate classification tasks in the horizontal and vertical directions to realize the prediction of the human joint points, and introduces a segmentation coefficient l to uniformly divide the pixels into lxl sub-pixel blocks, thereby reducing the quantization error of the model and improving the accuracy of the model.

[0026] Further, in step S2, the specific fusion process of the attention feature fusion module is as follows:

[0027] S201: The shallow-level features obtained by stage one through the residual module are After multiple stages of feature extraction and fusion, the deep-level features of stage four are represented as First, X and Y are simply added by elements to fuse the initial features; the fused feature Z is represented as:

[0028] S202: The input fusion feature Z generates a channel attention weight; the channel attention includes local channel attention L(Z) and global channel attention G(Z); the local channel attention uses point convolution to capture the correlation between channels at each spatial position; specifically, it passes through a 1x1 convolution layer and a ReLU activation function, then passes through a 1x1 convolution layer, and finally outputs through a batch normalization layer; the global channel attention extracts the attention of the global feature through global average pooling; first, it passes through global average pooling, then passes through a 1x1 convolution layer and a ReLU activation function, then passes through a 1x1 convolution layer, and finally outputs through a batch normalization layer;

[0029] G(Z)=B(K 1×1 (δ(B(K 1×1 (Avg(Z))))))

[0030] L(Z)=B(K 1×1 (δ(B(K 1×1 (Z)))))

[0031]

[0032] where B denotes a batch normalization layer, K 1×1 is a 1x1 point-wise convolution kernel, δ is a ReLU activation function, Avg denotes a global average pooling operation, M(Z) denotes the attention-weighted features, σ denotes a sigmoid function, denotes element-wise addition;

[0033] S203: The original features X and Y are weighted and averaged using the channel attention weight M(Z) obtained above, and the final output feature Q(Z) is represented as:

[0034] Further, in step S2, the specific prediction process of the decoupled coordinate classification prediction module is as follows:

[0035] S211: Map the n real joint coordinates (x p ,y p ) in the continuous coordinate space to the discrete pixel coordinate space, where p∈[1,n]; In order to reduce the quantization error, each pixel is uniformly divided into lxl (l≥1) sub-pixel blocks, and (x p ,y p ) is encoded and mapped to the sub-pixel space as the classification label for model training, and the encoded joint coordinates are represented as:

[0036] p'=(x',y')=(round(x p ×l),round(y p ×l))

[0037] where round(·) is a rounding function;

[0038] S212: Perform 1x1 convolution on the feature map (C,H,W) obtained by the human skeleton feature extraction network to obtain a feature map with size (n,H,W); Then merge the last two dimensions of the feature map to obtain n n joint representations, which n n joint representations are one-dimensional vectors with length d, where d=HWl;

[0039] S213: For the i-th joint representation , the horizontal and vertical coordinate classifiers (two fully connected layers) obtain one-dimensional vectors z xi and z yi in the horizontal and vertical directions respectively, and use the softmax function to convert them into two one-dimensional probability distributions and represent the probability of the joint position on the coordinate axis;

[0040]

[0041] wherein, represents the weight matrix of the horizontal coordinate classifier; represents the weight matrix of the vertical coordinate classifier; represents the probability value of the ith joint node at the jth horizontal coordinate position; represents the probability value of the ith joint node at the qth vertical coordinate position;

[0042] S214: the coordinates of the prediction point are obtained by finding the position of the maximum point of and dividing by the segmentation coefficient l to restore to the original picture scale.

[0043]

[0044] wherein, argmax represents the index of the maximum value.

[0045] Further, in step S4, the posture estimation module obtains the human posture, specifically including the following steps:

[0046] S41: selecting the head, shoulder joint, elbow joint, wrist joint, hip joint, knee joint and ankle joint to monitor the posture of the body;

[0047] S42: calculating the included angle between the joint nodes, selecting two vectors formed by the shoulder-hip and hip-knee, and judging the bending degree of the body by calculating the included angle θ1 between the two vectors; selecting two vectors formed by the hip-knee and knee-ankle joints to calculate the included angle θ2, so as to judge the standard degree of the sit-up leg action; combining the Euclidean distance D of the wrist joint to the head and the included angle θ3 of the shoulder-elbow-wrist to judge the standard degree of the hand action;

[0048] Suppose the coordinates of vectors v1 and v2 are (x 11 ,y 11 ) and (x 12 ,y 12 ) and (x 21 ,y 21 ) and (x 22 ,y 22 ), joint nodes p1 (x1, y1) and p1 (x2, y2), the calculation formula of the included angle θ and the Euclidean distance D is as follows:

[0049]

[0050] wherein,

[0051] v1·v2=(x 11 -x 12 )(x 21 -x 22)+(y 11 -y 12 )(y 21 -y 22 )

[0052]

[0053] S43: Set threshold values to determine the standard degree of counting and sit-up action; set the threshold angle of the sit-up lying state as θ down , set the threshold angle of the sit-up sitting state as θ up , set the threshold angle between the hip-knee-ankle joints as θ hka1 and θ hka2 , set the Euclidean distance threshold of the wrist joint to the head as D threshold , and set the threshold angle between the shoulder-elbow-wrist joints as θ sew ;

[0054] S44: Real-time calculation of the angles θ1, θ2, θ3 and the Euclidean distance D to determine whether the threshold is reached;

[0055] Sit-up posture judgment:

[0056] Leg movement standard action judgment:

[0057] Hand movement standard action judgment:

[0058] In the continuous image sequence, when it is detected that the measured person changes from the lying state to the sitting state and the leg and hand movements are determined to be standard, the count is incremented by one.

[0059] The beneficial effects of the present application are: the present application uses a large kernel separable convolution extraction unit to decompose the original large kernel convolution into multiple small kernel convolutions, which reduces the computational complexity and memory occupation while maintaining a larger receptive field. Secondly, the attention decoupling coordinate classification prediction module is proposed to fuse the shallow features and deep features in the human skeleton feature extraction network, so that the model can utilize both detailed information and semantic information, improving the accuracy and performance of key point detection. Finally, the coordinate prediction problem is reconstructed as a classification task of horizontal and vertical coordinates, evenly dividing each pixel into smaller pixel blocks to achieve higher positioning accuracy and lower quantization error.

[0060] Other advantages, objects, and features of the present application will be apparent to those skilled in the art from the following specification, and it is intended to be covered by the following claims. The objects and other advantages of the present application can be achieved and obtained by the following specification. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to make the objects, technical solutions and advantages of the present application clearer, the preferred embodiments of the present application will be described in detail below with reference to the drawings, in which:

[0062] Figure 1 Flow chart of the posture estimation-based lightweight sit-up detection method of the present application;

[0063] Figure 2 Schematic diagram of key points detected by the method of the present application;

[0064] Figure 3 Structural schematic diagram of the method of the present application;

[0065] Figure 4 Structural schematic diagram of the attention feature fusion module of the method of the present application. DETAILED DESCRIPTION

[0066] The present application will be described in greater detail by way of specific embodiments, and those skilled in the art will readily understand other advantages and purposes of the present application from the disclosure of the present application. The present application can also be implemented or applied in other different specific embodiments, and various modifications or changes can be made to the details of the present application based on different views and applications without departing from the spirit of the present application. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0067] The drawings are only used for illustrative description, and the representation is only a schematic diagram, not a physical diagram, and should not be understood as a limitation on the present application; in order to better illustrate the embodiments of the present application, some components in the drawings may be omitted, enlarged or reduced, and do not represent the actual size of the product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0068] The same or similar reference numerals in the drawings of the embodiments of the present application correspond to the same or similar components; in the description of the present application, it should be understood that the orientations or positional relationships indicated by terms such as "upper", "lower", "left", "right", "front", "back" and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore the terms describing the positional relationship in the drawings are only used for illustrative description, and should not be understood as a limitation on the present application, and for those skilled in the art, the specific meanings of the above terms can be understood according to the specific circumstances.

[0069] Please refer to Figures 1-4The embodiment of the application provides a lightweight sit-up detection method based on pose estimation, and specifically comprises the following steps:

[0070] I. Constructing a sit-up dataset

[0071] In the embodiment, the collection camera is fixed on a tripod at a distance of 0.85 m from the horizontal, and the distance from the collector is 1.5 m. The shooting angle is a side view. Each experimenter collects a 1-minute sit-up video. The video is saved in MP4 format. The collected video is converted into frame images, and 5 frame images are taken per second. In order to increase the diversity of samples and improve the generalization ability of the model, the images are preprocessed: first, the original images are cropped to ensure uniformity; second, the training set and the verification set are divided according to the ratio of 8:2; then, the data enhancement methods such as horizontal random flip, random scale cropping and scaling, and random graying are used to improve the robustness of the model. Finally, the human body joint points in the image are manually labeled to obtain the sit-up dataset.

[0072] II. Constructing a sit-up detection model

[0073] The sit-up detection model comprises a lightweight human skeleton feature extraction network and an attention decoupling coordinate classification prediction module.

[0074] As shown in Figure 3 , in the embodiment, the construction process of the sit-up detection model is as follows:

[0075] 1. Constructing a lightweight human skeleton feature extraction network;

[0076] The network comprises four stages: stage one is composed of four residual modules in cascade, and after processing, the feature map resolution is reduced to 1 / 4 of the original image; stages two, three and four each add a parallel branch, and the added branch is 1 / 2 of the smallest resolution branch of the previous stage; stage two comprises two branches, each branch is composed of three large-kernel separable convolution modules in cascade; stage three comprises three branches, which are composed of 3, 3 and 12 large-kernel separable convolution modules, respectively; stage four comprises four branches, which are composed of 3, 3, 12 and 3 large-kernel separable convolution modules, respectively. At the same time, except for stage one, each stage fuses multiple parallel branches, and outputs the feature map of the highest resolution branch in the fourth stage.

[0077] The residual module is specifically: the input is first processed through a convolution layer with a convolution kernel size of 3x3, and then flows through a batch normalization layer for standardization processing. Subsequently, the input is added to the output of the second convolution layer after being processed through a second convolution layer with a convolution kernel size of 3x3 and a batch normalization layer, forming a jump connection, and finally being processed through a ReLU activation function. The output of the residual module can be expressed as:

[0078] Output = δ(B2(Conv2(B1(Conv1(x)) + x))

[0079] wherein, x denotes the input; B1, B2 denotes two batch normalization layers; similarly, Conv1, Conv2 denotes two convolution layers; δ denotes the ReLU activation function.

[0080] The large-kernel separable convolution module decomposes a large-kernel convolution into multiple depth convolutions and depth dilated convolutions, which can significantly reduce the computational complexity and parameter quantity while retaining a larger effective receptive field and feature extraction capability. The specific decomposition process is as follows: an ordinary convolution with a kernel size of k x k is decomposed into a 1 x (2d-1) horizontal depth convolution and a (2d-1) x 1 vertical depth convolution, a 1 x (k / d) horizontal depth dilated convolution and a (k / d) x 1 vertical depth dilated convolution, and a 1 x 1 pointwise convolution. The input and the output of the 1 x 1 pointwise convolution are added and output after a ReLU activation function.

[0081] Given the input feature map The output feature is represented as:

[0082]

[0083] wherein, * and denote convolution operation and Hadamard product, respectively; DW denotes the output after horizontal and vertical depth convolution; K represents the convolution kernel, in particular, K 1×1 represents a 1 x 1 pointwise convolution kernel; d represents the dilated rate; DW_D denotes the output after horizontal and vertical depth dilated convolution, denotes the final output after the large-kernel separable convolution feature extraction unit.

[0084] 2. Construct an attention decoupling coordinate classification prediction module;

[0085] First, the attention feature fusion module fuses the deep-level features extracted in stage four and the shallow-level features extracted in stage one, and then the decoupling coordinate classification prediction module decomposes the two-dimensional classification task into one-dimensional classification tasks in the horizontal and vertical directions to realize the prediction of the human joint points, while introducing a segmentation coefficient l to uniformly divide the pixels into l x l sub-pixel blocks to reduce the quantization error of the model.

[0086] 2.1. The specific process of the attention feature fusion module is as follows:

[0087] 2.1.1. The shallow-level feature obtained by stage one through the residual module is After multiple stages of feature extraction and fusion, the deep-level feature of stage four is represented as First, X and Y are simply element-wise added to fuse the initial features. The fused feature is represented as Z:

[0088] 2.1.2, the input fused feature Z generates channel attention weight. Channel attention includes local channel attention L(Z) and global channel attention G(Z). Local channel attention uses point convolution to capture the correlation between channels at each spatial position. Specifically, it passes through a 1x1 convolution layer and a ReLU activation function, then through a 1x1 convolution layer, and finally through a batch normalization layer output; global channel attention extracts the attention of global features through global average pooling. First, it passes through global average pooling, then through a 1x1 convolution layer and a ReLU activation function, then through a 1x1 convolution layer, and finally through a batch normalization layer output.

[0089] G(Z) = B(K 1×1 (δ(B(K 1×1 (Avg(Z))))))

[0090] L(Z) = B(K 1×1 (δ(B(K 1×1 (Z)))))

[0091]

[0092] where B represents the batch normalization layer, K 1×1 is a 1x1 pointwise convolution kernel, δ is a ReLU activation function, Avg represents a global average pooling operation, M(Z) represents a channel attention weight, σ represents a sigmoid function, represents element-wise addition.

[0093] 2.1.3, the channel attention weight M(Z) obtained above is used to weight average the original features X and Y, and the final output feature is represented as Q(Z):

[0094]

[0095] 2.2, the specific process of the decoupling coordinate classification prediction module is:

[0096] 2.2.1, map n real joint coordinates (x p ,y p )(p∈[1,n]) in the continuous coordinate space to the discrete pixel coordinate space. In order to reduce quantization error, each pixel is uniformly divided into lxl (l≥1) sub-pixel blocks, and (x p ,y p ) is encoded and mapped to the sub-pixel space as the classification label for model training. The encoded joint coordinates are represented as:

[0097] p' = (x', y') = (round(x p ×l), round(y p ×l))

[0098] where round(·) is a rounding function.

[0099] 2.2.2, 1x1 convolution is performed on the feature map (C, H, W) obtained by the human skeleton feature extraction network to obtain a feature map with a size of (n, H, W). Then the last two dimensions of the feature map are merged to obtain n joint point representation, which n is a one-dimensional vector with a length of d, where d = HWl.

[0100] 2.2.3, for the i-th joint point representation the horizontal and vertical coordinate classifiers (two fully connected layers) obtain one-dimensional vectors and respectively in the horizontal and vertical directions, and use the Softmax function to convert them into two one-dimensional probability distributions and representing the probability of the joint position on the coordinate axis.

[0101]

[0102] where, represents the weight matrix of the horizontal coordinate classifier; represents the weight matrix of the vertical coordinate classifier; represents the probability value of the i-th joint at the j-th horizontal coordinate position; represents the probability value of the i-th joint at the q-th vertical coordinate position.

[0103] 2.2.4, by finding the positions of the maximum points of and and then dividing by the segmentation coefficient l to restore to the original picture scale, the coordinates of the predicted points

[0104]

[0105] where argmax represents the index of the maximum value.

[0106] III. The model is trained using the supine and sit-up data set, and supervised training is performed using the mean square error loss function.

[0107] The mean square loss function calculates the average of the squares of the differences between the predicted value and the true value, and its formula is:

[0108]

[0109] where y i The predicted output of the model is N is the number of samples.

[0110] During the training process, the training set of the sit-up dataset is input into the human skeleton feature extraction network. The predicted position of each joint is calculated by forward propagation, and the mean square loss function value is calculated according to the predicted position and the real position. Then, the gradient descent method is used to propagate the error according to the loss function value, and the gradient of each parameter in the network is calculated. The weights and bias parameters of the network are updated according to the gradient, and the prediction error is reduced until the preset training epoch (epoch = 210) is reached.

[0111] Four, input the image or video into the trained model to complete the detection and posture estimation of the human body, and then use the joint threshold counting detection module to realize the sit-up counting function of the target test personnel.

[0112] 1. Select the head, shoulder joint, elbow joint, wrist joint, hip joint, knee joint, and ankle joint to monitor the posture of the body.

[0113] 2. Calculate the vector between each pair of joints, and then calculate the included angle between the joints according to the two vectors. Select the two vectors formed by the shoulder-hip and hip-knee to calculate the included angle (θ1) between them to judge the bending degree of the body; select the two vectors formed by the hip-knee and knee-ankle joints to calculate the included angle (θ2) to judge the standard degree of the sit-up leg action; combine the Euclidean distance (D) from the wrist joint to the head and the included angle (θ3) formed by the shoulder-elbow-wrist to judge the standard degree of the hand action.

[0114] Assume that the coordinates of vectors v1 and v2 are (x 11 ,y 11 ) and (x 12 ,y 12 ) and (x 21 ,y 21 ) and (x 22 ,y 22 ), and the joint p1 (x1, y1) and the joint p1 (x2, y2), the included angle θ and the Euclidean distance D are calculated as follows:

[0115]

[0116] where,

[0117] v1·v2=(x 11 -x 12 )(x 21 -x 22 )+(y11 - y 12 )(y 21 - y 22 )

[0118]

[0119] 3. Set threshold to determine the standard degree of counting and sit-up action. Set the threshold angle θ down = 130° when lying down, set the threshold angle θ up = 60° when ending sit-up, set the threshold angle θ hka1 = 70° and θ hka2 = 100° between hip-knee-ankle joints, set the threshold Euclidean distance D threshold = 5cm from wrist joint to head, and set the threshold angle θ sew = 35° between shoulder-elbow-wrist joints.

[0120] 4. Real-time calculation of the angles θ 1 , θ 2 , θ 3 and the Euclidean distance D to determine whether the threshold is reached.

[0121] Sit-up posture determination:

[0122]

[0123] Leg action standard action determination:

[0124]

[0125] Hand action standard action determination:

[0126]

[0127] In the continuous image sequence, when the measured person is detected from the lying state to the sitting state and the leg and hand actions are determined to be standard, the count is incremented by one.

[0128] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the purpose and scope of the technical solutions, and they should be covered in the scope of the claims of the present application.

Claims

1. A lightweight sit-up detection method based on posture estimation, characterized in that: The method specifically comprises the following steps: S1: Collect sit-up image data, perform preprocessing, mark the coordinates of human joints, and obtain a dataset for training the sit-up detection model; S2: Build a sit-up detection model, including a lightweight human skeleton feature extraction network and an attention-decoupled coordinate classification prediction module; The lightweight human skeleton feature extraction network is constructed based on the large-core separable convolution module, and adopts four-stage multi-scale information fusion to obtain a high-resolution feature map; the lightweight human skeleton feature extraction network includes four stages: the first stage consists of a cascade of four residual modules, and after processing, the feature map resolution is reduced to 1 / 4 of the original image; the second, third, and fourth stages add a new parallel branch in turn, and the new branch is 1 / 2 of the minimum resolution branch of the previous stage; the second stage contains two branches, each branch consists of a cascade of three large-core separable convolution modules; the third stage contains three branches, which are respectively composed of 3, 3, and 12 large-core separable convolution modules; the fourth stage contains four branches, which are respectively composed of 3, 3, 12, and 3 large-core separable convolution modules; at the same time, except for the first stage, each stage fuses multiple parallel branches, and outputs the feature map of the highest resolution branch in the fourth stage; The attention decoupled coordinate classification prediction module includes an attention feature fusion module and a decoupled coordinate classification prediction module, which is used to predict the coordinate positions of human joints. First, the attention feature fusion module fuses the deep features extracted in stage 4 and the shallow features extracted in stage 1. Then, the decoupled coordinate classification prediction module decomposes the two-dimensional coordinate classification task into one-dimensional coordinate classification tasks in the horizontal and vertical directions to achieve the prediction of human joints. At the same time, the segmentation coefficient l is introduced to evenly divide the pixels into l×l sub-pixel blocks. The specific fusion process of the attention feature fusion module is as follows: S201: The shallow features obtained by the residual module in stage 1 are After multiple stages of feature extraction and fusion, the deep features of stage four are represented as First, add the elements of X and Y to fuse the initial features; the fused feature Z is expressed as: S202: Input fusion feature Z to generate channel attention weight; channel attention includes local channel attention L(Z) and global channel attention G(Z); local channel attention uses point convolution to capture the correlation between channels at each spatial position. Specifically, it passes through a 1×1 convolution layer and a ReLU activation function, then a 1×1 convolution layer, and finally a batch normalization layer output; global channel attention extracts the attention of global features through global average pooling. It first passes through global average pooling, then a 1×1 convolution layer and a ReLU activation function, then a 1×1 convolution layer, and finally a batch normalization layer output; G(Z)=B(K 1×1 (δ(B(K 1×1 (Aug(Z)))))) L(Z)=B(K 1×1 (δ(B(K 1×1 (WITH))))) Among them, B represents the batch normalization layer, K 1×1 is a 1×1 point-by-point convolution kernel, δ is the ReLU activation function, Avg represents the global average pooling operation, M(Z) represents the attention-weighted feature, σ represents the sigmoid function, represents element-wise addition; S203: Use the channel attention weight M(Z) obtained above to perform weighted averaging on the original features X and Y. The final output feature Q(Z) is expressed as: The specific prediction process of the decoupled coordinate classification prediction module is as follows: S211: Convert the n real joint point coordinates (x p ,y p ) is mapped to a discrete pixel coordinate space, where p∈[1,n]; each pixel is evenly divided into l×l sub-pixel blocks, and (x p ,y p ) encoding is mapped to the sub-pixel space as the classification label for model training. The encoded joint point coordinates are expressed as: p′=(x',y')=(round(x p ×l),round(y p ×l)) Among them, round(·) is the rounding function; S212: Perform a 1×1 convolution on the feature map (C, H, W) obtained by the human skeleton feature extraction network to obtain a feature map of size (n, H, W); then merge the last two dimensions of the feature map to obtain n joint point representations, which are one-dimensional vectors of length d, where d = HWl; S213: For the i-th joint point i∈[1,n], the horizontal and vertical coordinate classifiers obtain one-dimensional vectors in the horizontal and vertical directions respectively. and And use the softmax function to convert it into two one-dimensional probability distributions and Indicates the position probability of the joint point on the coordinate axis; in, Represents the weight matrix of the horizontal axis classifier; Represents the weight matrix of the ordinate classifier; Represents the probability value of the i-th joint point at the j-th horizontal coordinate position; Represents the probability value of the i-th joint point at the q-th vertical coordinate position; S214: By finding and The position of the maximum point on the image is then divided by the segmentation coefficient l to restore it to the original image scale, thereby obtaining the coordinates of the predicted point. Among them, argmax represents the index of the maximum value; S3: Input the dataset into the sit-up detection model for training, and use the mean square error loss function to supervise the training process; S4: Input the sit-up image sequence to be tested into the trained sit-up detection model to obtain the coordinate information of the human joint points. Then, use the posture estimation module to obtain the human posture. Then, based on the posture changes in the image sequence, the sit-up counting function and the action standardization detection function of the target test person are implemented. The posture estimation module obtains the human body posture, specifically comprising the following steps: S41: Select the head, shoulder, elbow, wrist, hip, knee, and ankle joints to monitor body posture; S42: Calculate the angles between joint points, select two vectors formed by the shoulder-hip and hip-knee joints, and determine the degree of body bending by calculating the angle θ1 between these two vectors; select two vectors formed by the hip-knee joint and the knee-ankle joint to calculate the angle θ2, and use this to determine the standard degree of the leg movement in the sit-up; and determine the standard degree of the hand movement by combining the Euclidean distance D from the wrist joint to the head and the angle θ3 formed by the shoulder-elbow-wrist joint; Assume that the coordinates of vectors v1 and v2 are (x 11 ,y 11 ) and (x 12 ,y 12 ) and (x 21 ,y 21 ) and (x 22 ,y 22 ), the joint point p1(x1,y1) and the joint point p2(x2,y2), the angle θ and the Euclidean distance D are calculated as follows: in, v1·v2=(x 11 -x 12 )(x 21 -x 22 )+(y 11 -y 12 )(y 21 -y 22 ) S43: Setting a threshold value to judge the standard degree of counting and sit-up action; setting the threshold angle when doing sit-ups lying flat to θ down , set the threshold angle of sit-up to θ up , set the threshold angle between the hip-knee-ankle joint to θ hka1 and θ hka2 , set the Euclidean distance threshold from wrist to head to D threshold , set the threshold angle between the shoulder-elbow-wrist joint to θ sew ; S44: Calculate the angles θ1, θ2, θ3 and the Euclidean distance D in real time to determine whether a threshold is reached; Sit-up posture judgment: Leg movement standard action judgment: Standard hand movement judgment: In a continuous image sequence, when it is detected that the person being tested moves from a lying state to a sitting state and the leg and hand movements are judged to be standard, the count is increased by one.

2. The lightweight sit-up detection method according to claim 1, characterized in that: In step S2, the residual module is specifically as follows: the input first passes through a convolution layer with a convolution kernel size of 3×3, and then flows through a batch normalization layer for normalization; then, it passes through a second convolution layer with a convolution kernel size of 3×3 and a batch normalization layer, and the input is added to the output of the second convolution layer to form a skip connection, and finally passes through the ReLU activation function; the output of the residual module is expressed as: Output=δ(B2(Conv2(B1(Conv1(x))))+x) Among them, x represents the input, B1 and B2 represent two batch normalization layers, Conv1 and Conv2 represent two convolutional layers, and δ represents the ReLU activation function.

3. The lightweight sit-up detection method according to claim 1, characterized in that: In step S2, the large kernel separable convolution module is specifically: decomposing a large kernel convolution into multiple depth convolutions and depth expansion convolutions, and the specific decomposition process is: a normal convolution with a convolution kernel size of k×k is decomposed into a 1×(2d-1) horizontal depth convolution and a (2d-1)×1 vertical depth convolution, and a 1×(k / d) horizontal depth expansion convolution and a (k / d)×1 vertical depth expansion convolution and a 1×1 point-by-point convolution, adding the input and the output of the 1×1 point-by-point convolution and outputting it after a ReLU activation function; Given an input feature map The output feature of the large kernel separable convolution module is expressed as: Among them, * and Represents the convolution operation and Hadamard product respectively; DW represents the output after horizontal and vertical depth convolution; K represents the convolution kernel, especially, K 1×1 represents a 1×1 point-by-point convolution kernel; d represents the dilation rate; DW_D represents the output after horizontal and vertical depth dilation convolution, It represents the final output of the large-kernel separable convolution feature extraction unit, where H, W, and C represent the height, width, and number of channels of the feature map, respectively.

Images