Lightweight sit-up detection method based on attitude estimation

Abstract

The invention relates to a lightweight sit-up detection method based on attitude estimation, and belongs to the field of computer vision. The method comprises the steps that S1, sit-up image data are collected and preprocessed, human body joint point coordinates are marked, and a data set is obtained; s2, constructing a sit-up detection model, wherein the sit-up detection model comprises a lightweight human skeleton feature extraction network and an attention decoupling coordinate classification prediction module; s3, inputting the data set into the model for training, and supervising the training process by using a mean square error loss function in the model training process; and S4, inputting a to-be-detected sit-up image sequence into the trained model, obtaining coordinate information of human body joint points, then obtaining a human body posture by using a posture estimation module, and then realizing a sit-up counting function and an action normative detection function on a target test person according to posture changes in the image sequence. The accuracy of sit-up detection can be improved.

Description

Technical Field

[0001] The invention belongs to the field of computer vision and relates to a lightweight sit-up detection method based on posture estimation. Background Art

[0002] Sit-ups play an important role in physical fitness tests. They are mainly used to assess the strength of an individual's abdominal core muscles, which is essential for maintaining body stability, improving athletic performance, and preventing lumbar injuries. In physical fitness tests, sit-ups are a simple and effective test method that is widely used among people of different ages and physical fitness levels. Traditional sit-up tests often rely on manual scoring or professional instruments, which have problems such as inaccurate assessment, high cost, and poor real-time performance.

[0003] Human pose estimation (HPE) is a fundamental and challenging task in the field of computer vision, which is crucial for describing human posture and human behavior. At present, pose estimation technology has made significant progress in the fields of human motion analysis, virtual reality, and intelligent monitoring. Therefore, the sit-up detection algorithm based on pose estimation has become a potential solution for improving body measurement. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a lightweight sit-up detection method based on posture estimation, which can not only realize automated and precise testing, but also improve the real-time performance of the test, and provide more scientific and reliable data support for physical education and health management.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A lightweight sit-up detection method based on posture estimation specifically comprises the following steps:

[0007] S1: Collect sit-up image data, perform preprocessing, mark the coordinates of human joints, and obtain a data set for sit-up detection model training;

[0008] S2: Build a sit-up detection model, including a lightweight human skeleton feature extraction network and an attention decoupled coordinate classification prediction module;

[0009] The lightweight human skeleton feature extraction network is constructed based on a large-kernel separable convolution module, and uses four-stage multi-scale information fusion to obtain a high-resolution feature map;

[0010] The attention decoupling coordinate classification prediction module is used to predict the coordinate positions of human joint points;

[0011] S3: Input the dataset into the sit-up detection model for training, and use the mean square error loss function to supervise the training process during the model training process;

[0012] S4: Input the sit-up image sequence to be detected into the trained sit-up detection model to obtain the coordinate information of the human joint points, and then use the posture estimation module to get the human posture. Then, according to the posture changes in the image sequence, the sit-up counting function and the action standardization detection function of the target test person are realized.

[0013] Further, in step S2, the ordinary human skeleton feature extraction network usually gradually reduces the resolution of the image during the feature extraction process. This strategy of reducing resolution helps to reduce the amount of calculation and memory consumption, but it also leads to the loss of detail information. The lightweight human skeleton feature extraction network includes four stages: stage one is composed of 4 residual modules in cascade, and after processing, the feature map resolution is reduced to 1 / 4 of the original image; the second, third, and fourth stages add a parallel branch in turn, and the new branch is 1 / 2 of the minimum resolution branch of the previous stage; stage two contains two branches, each of which is composed of 3 large-core separable convolution modules in cascade; stage three contains three branches, which are composed of 3, 3, and 12 large-core separable convolution modules respectively; stage four contains four branches, which are composed of 3, 3, 12, and 3 large-core separable convolution modules respectively; at the same time, except for stage one, each stage fuses multiple parallel branches, and outputs the feature map of the highest resolution branch in the fourth stage. While retaining the detail information of high-resolution features, it also integrates the global information of low-resolution features, improving the overall feature expression ability of the model. In addition, the large-core separable convolution module can significantly reduce the amount of computation and parameters while maintaining the model performance.

[0014] Further, in step S2, the residual module is specifically as follows: the input first passes through a convolution layer with a convolution kernel size of 3×3, and then flows through a batch normalization layer for standardization; then, it passes through a second convolution layer with a convolution kernel size of 3×3 and a batch normalization layer, and the input is added to the output of the second convolution layer to form a jump connection, and finally passes through a ReLU activation function; since the jump connection retains the feature information of the input and adds it to the output of the convolution layer, the residual module is easier to learn to identify small feature changes, making the network converge faster, and also enabling the model to extract features more finely without losing information. The output of the residual module can be expressed as:

[0015] Output=δ(B2(Conv2(B1(Conv1(x))))+x)

[0016] in, xDenotes input, B1 and B2 denote two batch normalization layers, Conv1 and Conv2 denote two convolutional layers, and δ denotes the ReLU activation function.

[0017] In standard convolution operation, each output channel is obtained by convolving and summing all input channels. In depth convolution, each input channel is convolved only with its corresponding convolution kernel. i The input feature map of the convolution kernel size is K×K×C i , the output feature map size is H×W×C o , ignoring the bias induction, the number of parameters and the amount of calculation of the standard convolution operation are as follows:

[0018] Params=K×K×C i ×C o =K 2 C i C o

[0019] Flops = K × K × C i ×C o ×H×W=K 2 C i C o HW

[0020] Further, in step S2, the large-core separable convolution module is specifically: decomposing a large-core convolution into multiple depth convolutions and depth dilated convolutions, which can significantly reduce the amount of calculation and parameters while retaining a large effective receptive field and feature extraction capabilities. The specific decomposition process is: a normal convolution with a convolution kernel size of k×k is decomposed into a 1×(2d-1) horizontal depth convolution and a (2d-1)×1 vertical depth convolution, as well as a 1×(k / d) horizontal depth dilated convolution and a (k / d)×1 vertical depth dilated convolution and a 1×1 point-by-point convolution. The input and the output of the 1×1 point-by-point convolution are added and output after a ReLU activation function. At this time, the parameter amount and calculation amount of the large-core separable convolution module are as follows:

[0021]

[0022] Given an input feature map The output feature of the large-core separable convolution module is expressed as:

[0023]

[0024] Among them, * and Denote the convolution operation and Hadamard product respectively; DW denotes the output after horizontal and vertical depth convolution; K represents the convolution kernel, in particular, K1×1 represents a 1×1 point-by-point convolution kernel; d represents the dilation rate; DW_D represents the output after horizontal and vertical depth dilation convolution, It represents the final output of the large-core separable convolution feature extraction unit. H, W, and C represent the height, width, and number of channels of the feature map, respectively.

[0025] Further, in step S2, the attention decoupled coordinate classification prediction module includes an attention feature fusion module and a decoupled coordinate classification prediction module; first, the attention feature fusion module fuses the deep features extracted in stage four and the shallow features extracted in stage one, providing a more comprehensive feature expression and improving the joint positioning accuracy of the model; then the decoupled coordinate classification prediction module decomposes the two-dimensional coordinate classification task into one-dimensional coordinate classification tasks in the horizontal and vertical directions to realize the prediction of human joints, and at the same time introduces a segmentation coefficient l to evenly divide the pixels into l×l sub-pixel blocks, thereby reducing the quantization error of the model and improving the accuracy of the model.

[0026] Further, in step S2, the specific fusion process of the attention feature fusion module is:

[0027] S201: The shallow features obtained by the residual module in stage 1 are After multiple stages of feature extraction and fusion, the deep features of stage 4 are represented as First, simply add the elements of X and Y to fuse the initial features; the fused feature Z is expressed as:

[0028] S202: Input fusion feature Z to generate channel attention weights; channel attention includes local channel attention L(Z) and global channel attention G(Z); local channel attention uses point convolution to capture the correlation between channels at each spatial position. Specifically, it passes through a 1×1 convolution layer and a ReLU activation function, then a 1×1 convolution layer, and finally a batch normalization layer output; global channel attention extracts the attention of global features through global average pooling, first through global average pooling, then through a 1×1 convolution layer and a ReLU activation function, then through a 1×1 convolution layer, and finally through a batch normalization layer output;

[0029] G(Z)=B(K 1×1 (δ(B(K 1×1 (Avg(Z))))))

[0030] L(Z)=B(K 1×1 (δ(B(K 1×1 (Z)))))

[0031]

[0032] Among them, B represents the batch normalization layer, K 1×1 is a 1×1 point-by-point convolution kernel, δ is the ReLU activation function, Avg represents the global average pooling operation, M(Z) represents the attention-weighted feature, σ represents the sigmoid function, represents element-wise addition;

[0033] S203: Use the channel attention weight M(Z) obtained above to perform weighted averaging on the original features X and Y. The final output feature Q(Z) is expressed as:

[0034] Further, in step S2, the specific prediction process of the decoupled coordinate classification prediction module is:

[0035] S211: Convert the n real joint point coordinates (x p ,y p ) is mapped to a discrete pixel coordinate space, where p∈[1,n]; in order to reduce the quantization error, each pixel is evenly divided into l×l (l≥1) sub-pixel blocks. p ,y p ) is encoded and mapped to the sub-pixel space as the classification label for model training. The encoded joint point coordinates are expressed as:

[0036] p′=(x',y')=(round(x p ×l),round(y p ×l))

[0037] Among them, round(·) is the rounding function;

[0038] S212: Perform 1×1 convolution on the feature map (C, H, W) obtained by the human skeleton feature extraction network to obtain a feature map of size (n, H, W); then merge the last two dimensions of the feature map to obtain n The joint points indicate that n The joint point representation is a one-dimensional vector of length d, where d = HWl;

[0039] S213: For the i-th joint point The horizontal and vertical classifiers (two fully connected layers) obtain the one-dimensional vector z in the horizontal and vertical directions respectively. xi and z yi , and use the softmax function to convert it into two one-dimensional probability distributions and Indicates the position probability of the joint point on the coordinate axis;

[0040]

[0041] in, Represents the weight matrix of the abscissa classifier; Represents the weight matrix of the ordinate classifier; Represents the probability value of the i-th joint point at the j-th horizontal coordinate position; Represents the probability value of the i-th joint point at the q-th ordinate position;

[0042] S214: By finding and The position of the maximum point on the image is then divided by the segmentation coefficient l to restore it to the original image scale, thereby obtaining the coordinates of the predicted point.

[0043]

[0044] Here, argmax represents the index of the maximum value.

[0045] Further, in step S4, the posture estimation module obtains the human body posture, which specifically includes the following steps:

[0046] S41: Select the head, shoulder, elbow, wrist, hip, knee, and ankle to monitor the body posture;

[0047] S42: Calculate the angles between joint points, select two vectors formed by shoulder-hip and hip-knee, and determine the degree of body bending by calculating the angle θ1 between the two vectors; select two vectors of hip-knee and knee-ankle to calculate the angle θ2, and determine the standard degree of leg action in sit-ups; determine the standard degree of hand action by combining the Euclidean distance D from the wrist joint to the head and the angle θ3 formed by the shoulder-elbow-wrist;

[0048] Assume that the coordinates of vectors v1 and v2 are (x 11 ,y 11 ) and (x 12 ,y 12 ) and (x 21 ,y 21 ) and (x 22 ,y 22 ), the calculation formulas of joint point p1(x1,y1) and joint point p1(x2,y2), angle θ and Euclidean distance D are as follows:

[0049]

[0050] in,

[0051] v1·v2=(x 11 -x 12 )(x 21 -x 22)+(y 11 -y 12 )(y 21 -y 22 )

[0052]

[0053] S43: Setting a threshold value to judge the standard degree of counting and sit-up action; setting the threshold angle when lying down for sit-ups as θ down , set the threshold angle of sit-ups to θ up , set the threshold angle between the hip-knee-ankle joint to θ hka1 and θ hka2 , set the Euclidean distance threshold from wrist to head to D threshold , set the threshold angle between the shoulder-elbow-wrist joint to θ sew ;

[0054] S44: Calculate the angles θ1, θ2, θ3 and the Euclidean distance D in real time to determine whether a threshold is reached;

[0055] Sit-up posture judgment:

[0056] Standard leg movement judgment:

[0057] Hand movement standard action judgment:

[0058] In a continuous sequence of images, when it is detected that the person being tested changes from a lying state to a sitting state and the leg and hand movements are judged to be standard, the count is increased by one.

[0059] The beneficial effect of the present invention is that the present invention uses a large-core separable convolution extraction unit to decompose the original large-core convolution into multiple small-core convolutions, reducing the computational complexity and memory usage while maintaining a large receptive field. Secondly, an attention decoupled coordinate classification prediction module is proposed to integrate shallow features and deep features in the human skeleton feature extraction network, so that the model can simultaneously utilize detail information and semantic information to improve the accuracy and performance of key point detection. Finally, the coordinate prediction problem is reconstructed into a classification task of horizontal and vertical coordinates, and each pixel is evenly divided into smaller pixel blocks to achieve higher positioning accuracy and lower quantization error.

[0060] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0062] Figure 1 It is a flow chart of a lightweight sit-up detection method based on posture estimation of the present invention;

[0063] Figure 2 It is a schematic diagram of key points detected by the method of the present invention;

[0064] Figure 3 It is a structural schematic diagram of the method of the present invention;

[0065] Figure 4 Schematic diagram of the structure of the attention feature fusion module of the method of the present invention. DETAILED DESCRIPTION

[0066] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0067] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0068] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0069] See also Figure 1 to Figure 4The embodiment of the present invention provides a lightweight sit-up detection method based on posture estimation, which specifically includes the following steps:

[0070] 1. Construct a sit-up dataset

[0071] In this embodiment, the acquisition camera is fixed on a tripod at a height of 0.85m from the horizontal, 1.5m away from the person being collected, and the shooting angle is taken from the side. Each experimenter collects a 1-minute sit-up video. The video is saved in MP4 format. The collected video is converted into frame images, and 5 frames of images are sampled per second. In order to increase the diversity of samples and improve the generalization ability of the model, the image is preprocessed: first, the original image is cropped to ensure the same size; secondly, the training set and the verification set are divided according to the ratio of 8:2; then, data enhancement methods such as horizontal random flipping, random ratio cropping and scaling, and random graying are used to improve the robustness of the model. Finally, the human joints in the image are manually annotated to obtain the sit-up data set.

[0072] 2. Build a sit-up detection model

[0073] The sit-up detection model includes a lightweight human skeleton feature extraction network and an attention-decoupled coordinate classification prediction module.

[0074] like Figure 3 As shown, in this embodiment, the construction process of the sit-up detection model is:

[0075] 1. Build a lightweight human skeleton feature extraction network;

[0076] The network consists of four stages: stage one consists of 4 residual modules in cascade, and after processing, the feature map resolution is reduced to 1 / 4 of the original image; the second, third, and fourth stages add a new parallel branch, and the new branch is 1 / 2 of the minimum resolution branch in the previous stage; stage two contains two branches, each of which is composed of 3 large-core separable convolution modules in cascade; stage three contains three branches, which are composed of 3, 3, and 12 large-core separable convolution modules respectively; stage four contains four branches, which are composed of 3, 3, 12, and 3 large-core separable convolution modules respectively. At the same time, except for stage one, each stage fuses multiple parallel branches, and outputs the feature map of the highest resolution branch in the fourth stage.

[0077] The specific residual module is as follows: the input first passes through a convolution layer with a convolution kernel size of 3×3, and then flows through a batch normalization layer for standardization. Subsequently, it passes through a second convolution layer with a convolution kernel size of 3×3 and a batch normalization layer, and the input is added to the output of the second convolution layer to form a skip connection, and finally passes through the ReLU activation function. The output of the residual module can be expressed as:

[0078] Output=δ(B2(Conv2(B1(Conv1(x))))+x)

[0079] in, x Represents input; B1 and B2 represent two batch normalization layers; similarly, Conv1 and Conv2 represent two convolutional layers; δ represents the ReLU activation function.

[0080] The large-core separable convolution module decomposes a large-core convolution into multiple depth convolutions and depth dilated convolutions, which can significantly reduce the amount of calculation and parameters while retaining a large effective receptive field and feature extraction capabilities. The specific decomposition process is: a normal convolution with a convolution kernel size of k×k is decomposed into a 1×(2d-1) horizontal depth convolution and a (2d-1)×1 vertical depth convolution, a 1×(k / d) horizontal depth dilated convolution and a (k / d)×1 vertical depth dilated convolution and a 1×1 point-by-point convolution. The input and the output of the 1×1 point-by-point convolution are added and output after a ReLU activation function.

[0081] Given an input feature map The output features are expressed as:

[0082]

[0083] Among them, * and Denote the convolution operation and Hadamard product respectively; DW denotes the output after horizontal and vertical depth convolution; K represents the convolution kernel, in particular, K 1×1 represents a 1×1 point-by-point convolution kernel; d represents the dilation rate; DW_D represents the output after horizontal and vertical depth dilation convolution, Represents the final output of the large-kernel separable convolution feature extraction unit.

[0084] 2. Construct an attention decoupled coordinate classification prediction module;

[0085] First, the attention feature fusion module fuses the deep features extracted in stage 4 and the shallow features extracted in stage 1. Then, the decoupled coordinate classification prediction module decomposes the two-dimensional classification task into one-dimensional classification tasks in the horizontal and vertical directions to realize the prediction of human joints. At the same time, the segmentation coefficient l is introduced to evenly divide the pixels into l×l sub-pixel blocks to reduce the quantization error of the model.

[0086] 2.1. The specific process of the attention feature fusion module is as follows:

[0087] 2.1.1. The shallow features obtained by the residual module in stage 1 are After multiple stages of feature extraction and fusion, the deep features of stage 4 are represented as First, simply add the elements of X and Y to fuse the initial features. The fused features are represented as Z:

[0088] 2.1.2. Input fusion feature Z to generate channel attention weights. Channel attention includes local channel attention L(Z) and global channel attention G(Z). Local channel attention uses point convolution to capture the correlation between channels at each spatial position. Specifically, it passes through a 1×1 convolution layer and a ReLU activation function, then a 1×1 convolution layer, and finally a batch normalization layer output; global channel attention extracts the attention of global features through global average pooling. It first passes through global average pooling, then a 1×1 convolution layer and a ReLU activation function, then a 1×1 convolution layer, and finally a batch normalization layer output.

[0089] G(Z)=B(K 1×1 (δ(B(K 1×1 (Avg(Z))))))

[0090] L(Z)=B(K 1×1 (δ(B(K 1×1 (Z)))))

[0091]

[0092] Among them, B represents the batch normalization layer, K 1×1 is a 1×1 point-by-point convolution kernel, δ is the ReLU activation function, Avg represents the global average pooling operation, M(Z) represents the channel attention weight, σ represents the sigmoid function, Represents element-wise addition.

[0093] 2.1.3. Use the channel attention weight M(Z) obtained above to perform weighted averaging on the original features X and Y. The final output feature is expressed as Q(Z):

[0094]

[0095] 2.2. The specific process of the decoupled coordinate classification prediction module is as follows:

[0096] 2.2.1、The n real joint point coordinates (x p ,y p )(p∈[1,n]) is mapped to a discrete pixel coordinate space. In order to reduce the quantization error, each pixel is evenly divided into l×l (l≥1) sub-pixel blocks. p ,y p ) is encoded and mapped to the sub-pixel space as the classification label for model training. The encoded joint point coordinates are expressed as:

[0097] p′=(x',y')=(round(x p ×l),round(y p ×l))

[0098] Where round(·) is the rounding function.

[0099] 2.2.2. Perform 1×1 convolution on the feature map (C, H, W) obtained by the human skeleton feature extraction network to obtain a feature map of size (n, H, W). Then merge the last two dimensions of the feature map to obtain n The joint points indicate that n The joint point representation is a one-dimensional vector of length d, where d = HWl.

[0100] 2.2.3. For the i-th joint point, The horizontal and vertical classifiers (two fully connected layers) obtain one-dimensional vectors in the horizontal and vertical directions respectively. and And use the Softmax function to convert it into two one-dimensional probability distributions and Represents the position probability of the joint point on this coordinate axis.

[0101]

[0102] in, Represents the weight matrix of the abscissa classifier; Represents the weight matrix of the ordinate classifier; Represents the probability value of the i-th joint point at the j-th horizontal coordinate position; Represents the probability value of the i-th joint point at the q-th ordinate position.

[0103] 2.2.4, by finding and The position of the maximum point on the image is then divided by the segmentation coefficient l to restore it to the original image scale, thereby obtaining the coordinates of the predicted point.

[0104]

[0105] Here, argmax represents the index of the maximum value.

[0106] 3. Use the sit-up dataset to train the model and use the mean square error loss function for supervised training.

[0107] The mean squared loss function calculates the average of the squares of the differences between the predicted values ​​and the true values. Its formula is:

[0108]

[0109] Among them, y i Corresponding to the true output, the model's predicted output is N is the sample size.

[0110] During the training process, the training set of the sit-up data set is input into the human skeleton feature extraction network. The predicted position of each joint point is calculated by forward propagation, and the mean square loss function value is calculated based on the predicted position and the true position. Then, the gradient descent method is used to back-propagate the error based on the loss function value and calculate the gradient of each parameter in the network. The weight and bias parameters of the network are updated according to the gradient to reduce the prediction error until the preset number of training rounds (epoch = 210) is reached.

[0111] Fourth, input the image or video into the trained model to complete the detection of the human body and the estimation of the posture, and then use the joint point threshold counting detection module to realize the sit-up counting function of the target test person.

[0112] 1. Select the head, shoulder, elbow, wrist, hip, knee and ankle to monitor the body posture;

[0113] 2. Calculate the vectors between the two joints, and then calculate the angles between the joints based on the two vectors. Select the two vectors formed by the shoulder-hip and hip-knee, and determine the degree of body bending by calculating the angle between the two vectors (θ1); select the two vectors of the hip-knee and knee-ankle joints to calculate the angle (θ2) to determine the standard degree of leg movements in sit-ups; combine the Euclidean distance from the wrist joint to the head (D) and the angle formed by the shoulder-elbow-wrist (θ3) to determine the standard degree of hand movements.

[0114] Assume that the coordinates of vectors v1 and v2 are (x 11 ,y 11 ) and (x 12 ,y 12 ) and (x 21 ,y 21 ) and (x 22 ,y 22 ), the calculation formulas of joint point p1(x1,y1) and joint point p1(x2,y2), angle θ and Euclidean distance D are as follows:

[0115]

[0116] in,

[0117] v1·v2=(x 11 -x 12 )(x 21 -x 22 )+(y11 -y 12 )(y 21 -y 22 )

[0118]

[0119] 3. Set the threshold value to judge the standard degree of counting and sit-up action. Set the threshold angle θ when lying down for sit-ups down =130°, set the threshold angle θ for the end of sit-ups up = 60°, setting the threshold angle θ between the hip-knee-ankle joint hka1 =70° and θ hka2 = 100°, set the Euclidean distance threshold D from wrist to head threshold =5cm, set the threshold angle between the shoulder-elbow-wrist joint to θ sew =35°.

[0120] 4. Calculate the angles θ1, θ2, θ3 and the Euclidean distance D in real time to determine whether the threshold is reached.

[0121] Sit-up posture judgment:

[0122]

[0123] Standard leg movement judgment:

[0124]

[0125] Hand movement standard action judgment:

[0126]

[0127] In a continuous sequence of images, when it is detected that the person being tested changes from a lying state to a sitting state and the leg and hand movements are judged to be standard, the count is increased by one.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A lightweight sit-up detection method based on posture estimation, characterized in that: The method specifically comprises the following steps: S1: Collect sit-up image data, perform preprocessing, mark the coordinates of human joints, and obtain a data set for sit-up detection model training; S2: Build a sit-up detection model, including a lightweight human skeleton feature extraction network and an attention decoupled coordinate classification prediction module; The lightweight human skeleton feature extraction network is constructed based on a large-kernel separable convolution module, and uses four-stage multi-scale information fusion to obtain a high-resolution feature map; The attention decoupling coordinate classification prediction module is used to predict the coordinate positions of human joint points; S3: Input the dataset into the sit-up detection model for training, and use the mean square error loss function to supervise the training process during the model training process; S4: Input the sit-up image sequence to be detected into the trained sit-up detection model to obtain the coordinate information of the human joint points, and then use the posture estimation module to get the human posture. Then, according to the posture changes in the image sequence, the sit-up counting function and the action standardization detection function of the target test person are realized.

2. The lightweight sit-up detection method according to claim 1, characterized in that: In step S2, the lightweight human skeleton feature extraction network includes four stages: stage 1 is composed of 4 residual modules cascaded, and after processing, the feature map resolution is reduced to 1 / 4 of the original image; A new parallel branch is added in the second, third, and fourth stages respectively, and the new branch is 1 / 2 of the minimum resolution branch in the previous stage; stage two contains two branches, each branch is composed of 3 large-core separable convolution modules in cascade; stage three contains three branches, which are composed of 3, 3, and 12 large-core separable convolution modules respectively; stage four contains four branches, which are composed of 3, 3, 12, and 3 large-core separable convolution modules respectively; at the same time, except for stage one, each stage fuses multiple parallel branches, and outputs the feature map of the highest resolution branch in the fourth stage.

3. The lightweight sit-up detection method according to claim 2, characterized in that: In step S2, the residual module is specifically as follows: the input first passes through a convolution layer with a convolution kernel size of 3×3, and then flows through a batch normalization layer for standardization; then, it passes through a second convolution layer with a convolution kernel size of 3×3 and a batch normalization layer, and the input is added to the output of the second convolution layer to form a skip connection, and finally passes through a ReLU activation function; the output of the residual module is expressed as: Output=δ(B2(Conv2(B1(Conv1(x))))+x) in, x Denotes input, B1 and B2 denote two batch normalization layers, Conv1 and Conv2 denote two convolutional layers, and δ denotes the ReLU activation function.

4. The lightweight sit-up detection method according to claim 2, characterized in that: In step S2, the large-core separable convolution module is specifically: decomposing a large-core convolution into multiple depth convolutions and depth dilated convolutions, and the specific decomposition process is: a common convolution with a convolution kernel size of k×k is decomposed into a 1×(2d-1) horizontal depth convolution and a (2d-1)×1 vertical depth convolution, and a 1×(k / d) horizontal depth dilated convolution and a (k / d)×1 vertical depth dilated convolution and a 1×1 point-by-point convolution, and the input and the output of the 1×1 point-by-point convolution are added and output after a ReLU activation function; Given an input feature map The output feature of the large-core separable convolution module is expressed as: Among them, * and Denote the convolution operation and Hadamard product respectively; DW denotes the output after horizontal and vertical depth convolution; K represents the convolution kernel, in particular, K 1×1 represents a 1×1 point-by-point convolution kernel; d represents the dilation rate; DW_D represents the output after horizontal and vertical depth dilation convolution, It represents the final output of the large-core separable convolution feature extraction unit. H, W, and C represent the height, width, and number of channels of the feature map, respectively.

5. The lightweight sit-up detection method according to claim 2, characterized in that: In step S2, the attention decoupled coordinate classification prediction module includes an attention feature fusion module and a decoupled coordinate classification prediction module; first, the attention feature fusion module fuses the deep features extracted in stage four and the shallow features extracted in stage one; then the decoupled coordinate classification prediction module decomposes the two-dimensional coordinate classification task into one-dimensional coordinate classification tasks in the horizontal and vertical directions to realize the prediction of human joint points, and at the same time introduces the segmentation coefficient l to evenly divide the pixels into l×l sub-pixel blocks.

6. The lightweight sit-up detection method according to claim 5, characterized in that: In step S2, the specific fusion process of the attention feature fusion module is: S201: The shallow features obtained by the residual module in stage 1 are After multiple stages of feature extraction and fusion, the deep features of stage 4 are represented as First, add the elements of X and Y to fuse the initial features; the fused feature Z is expressed as: S202: Input fusion feature Z to generate channel attention weights; channel attention includes local channel attention L(Z) and global channel attention G(Z); local channel attention uses point convolution to capture the correlation between channels at each spatial position. Specifically, it passes through a 1×1 convolution layer and a ReLU activation function, then a 1×1 convolution layer, and finally a batch normalization layer output; global channel attention extracts the attention of global features through global average pooling, first through global average pooling, then through a 1×1 convolution layer and a ReLU activation function, then through a 1×1 convolution layer, and finally through a batch normalization layer output; G(Z)=B(K 1×1 (δ(B(K 1×1 (Aug(Z)))))) L(Z)=B(K 1×1 (δ(B(K 1×1 (WITH))))) Among them, B represents the batch normalization layer, K 1×1 is a 1×1 point-by-point convolution kernel, δ is the ReLU activation function, Avg represents the global average pooling operation, M(Z) represents the attention-weighted feature, σ represents the sigmoid function, represents element-wise addition; S203: Use the channel attention weight M(Z) obtained above to perform weighted averaging on the original features X and Y. The final output feature Q(Z) is expressed as:

7. The lightweight sit-up detection method according to claim 5, characterized in that: In step S2, the specific prediction process of the decoupled coordinate classification prediction module is: S211: Convert the n real joint point coordinates (x p ,y p ) is mapped to a discrete pixel coordinate space, where p∈[1,n]; each pixel is evenly divided into l×l sub-pixel blocks, and (x p ,y p ) is encoded and mapped to the sub-pixel space as the classification label for model training. The encoded joint point coordinates are expressed as: p′=(x',y')=(round(x p ×l),round(y p ×l)) Among them, round(·) is the rounding function; S212: Perform 1×1 convolution on the feature map (C, H, W) obtained by the human skeleton feature extraction network to obtain a feature map of size (n, H, W); then merge the last two dimensions of the feature map to obtain n The joint points indicate that n The joint point representation is a one-dimensional vector of length d, where d = HWl; S213: For the i-th joint point The horizontal and vertical classifiers obtain one-dimensional vectors in the horizontal and vertical directions respectively. and And use the softmax function to convert it into two one-dimensional probability distributions and Indicates the position probability of the joint point on the coordinate axis; in, Represents the weight matrix of the abscissa classifier; Represents the weight matrix of the ordinate classifier; Represents the probability value of the i-th joint point at the j-th horizontal coordinate position; Represents the probability value of the i-th joint point at the q-th ordinate position; S214: By finding and The position of the maximum point on the image is then divided by the segmentation coefficient l to restore it to the original image scale, thereby obtaining the coordinates of the predicted point. Here, argmax represents the index of the maximum value.

8. The lightweight sit-up detection method according to claim 1, characterized in that: In step S4, the posture estimation module obtains the human body posture, which specifically includes the following steps: S41: Select the head, shoulder, elbow, wrist, hip, knee, and ankle to monitor the body posture; S42: Calculate the angles between joint points, select two vectors formed by shoulder-hip and hip-knee, and determine the degree of body bending by calculating the angle θ1 between the two vectors; select two vectors of hip-knee and knee-ankle to calculate the angle θ2, and determine the standard degree of leg action in sit-ups; determine the standard degree of hand action by combining the Euclidean distance D from the wrist joint to the head and the angle θ3 formed by the shoulder-elbow-wrist; Assume that the coordinates of vectors v1 and v2 are (x 11 ,y 11 ) and (x 12 ,y 12 ) and (x 21 ,y 21 ) and (x 22 ,y 22 ), the calculation formulas of joint point p1(x1,y1) and joint point p1(x2,y2), angle θ and Euclidean distance D are as follows: in, v1·v2=(x 11 -x 12 )(x 21 -x 22 )+(y 11 -y 12 )(y 21 -y 22 ) S43: Setting a threshold value to judge the standard degree of counting and sit-up action; setting the threshold angle when lying down for sit-ups as θ down , set the threshold angle of sit-ups to θ up , set the threshold angle between the hip-knee-ankle joint to θ hka1 and θ hka2 , set the Euclidean distance threshold from wrist to head to D threshold , set the threshold angle between the shoulder-elbow-wrist joint to θ sew ; S44: Calculate the angles θ1, θ2, θ3 and the Euclidean distance D in real time to determine whether a threshold is reached; Sit-up posture judgment: Standard leg movement judgment: Hand movement standard action judgment: In a continuous sequence of images, when it is detected that the person being tested moves from a lying state to a sitting state and the leg and hand movements are judged to be standard, the count is increased by one.

Images