A hand posture assessment method based on RGB images
Through the MSIPA-HandNet algorithm, combined with multi-scale information perception and distribution perception technology, the problem of difficult balance between speed and accuracy of hand posture estimation in the prior art is solved, and more efficient model optimization and application effects are achieved.
Patent Information
- Application Number
- CN202410762874.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-13
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-06-13
AI Technical Summary
The existing hand posture estimation method based on RGB images is difficult to balance between speed and accuracy, and lacks quantitative evaluation standards, resulting in waste of resources and difficulty in model optimization.
A MSIPA-HandNet algorithm is proposed, which uses multi-scale information perception structure and distribution perception coordinate representation, combines attention mechanism and deep learning model to achieve the speed and accuracy balance of hand pose estimation, and proposes a comprehensive metric standard for SAT in evaluating and optimizing models.
A better balance of speed and accuracy is achieved, quantitative evaluation standards are provided, the hand posture estimation model is optimized, and the application efficiency and resource utilization of the model are improved.
Smart Images

Figure CN118658180B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision, and in particular to a hand posture evaluation method based on RGB images. Background Art
[0002] In recent years, the use of convolutional neural networks (CNNs) has made significant progress in hand pose estimation. With the development of depth sensors, hand pose estimation methods have shifted from hardware synchronization methods based on data gloves to computer vision-based methods.
[0003] Vision-based methods are divided into two categories, one is to estimate hand pose based on depth images, and the other is to estimate hand pose using RGB images. The increasing depth image data has promoted the research of hand pose estimation technology. It should be noted that although depth sensors have limited resolution and range, they are also affected by ambient lighting conditions. Compared with ordinary cameras, depth sensors are relatively rare. Current hand pose estimation research is mainly based on RGB images, which is also one of the hot spots in this field. Some of the main reasons are that it is more representative of actual application scenarios.
[0004] In the prior art, the standard coordinate decoding method designed based on experience when estimating hand posture based on RGB images cannot accurately correspond to the position of the tube joint, but only corresponds to the rough position. In addition, the models of the prior art sometimes sacrifice accuracy in exchange for reaction speed, and sometimes sacrifice reaction speed in exchange for reaction accuracy. In a large number of academic studies, the reasoning speed of the model is often ignored, and the accuracy of the model is emphasized. For the industrial implementation of deep learning models, there are often different considerations for speed-accuracy based on different deployment hardware platforms. At present, there is no quantitative evaluation standard for the speed-accuracy trade-off, and only a rough trade-off can be made for different hardware platforms, which will cause a certain waste of resources. Summary of the invention
[0005] The purpose of the present invention is to provide a hand posture evaluation method based on RGB images, and a MSIPA-HandNet algorithm is proposed, which uses multi-scale information perception structure and distribution-aware coordinate representation to achieve a better balance between speed and accuracy for hand posture estimation. Based on the speed / accuracy trade-off problem, a comprehensive metric is proposed to evaluate hand posture estimation, and valuable opinions are provided for optimizing the hand posture estimation model, using the position information of gesture key points for easy application.
[0006] To achieve the above object, the present invention provides a hand posture assessment method based on RGB images, comprising the following steps:
[0007] S1. Propose a multi-scale information perception model based on attention mechanism to obtain gesture key point information;
[0008] S2, more accurate estimation of hand posture through distribution-aware key point coordinate representation method;
[0009] S3. Choose the appropriate balance of speed and accuracy for a given application and platform based on the deep learning model.
[0010] Preferably, the multi-scale information perception model based on the attention mechanism in S1 is an encoder-decoder structure, and the encoder-decoder structure is composed of downsampling and upsampling. In downsampling, 3×3 depth-separable convolution is used as the basic unit, and in upsampling, 4×4 deconvolution block is used as the basic unit.
[0011] Preferably, the downsampling includes a multi-scale information perception structure MSIPA based on the attention mechanism, which is composed of an improved hole convolution pyramid and a randomly disrupted attention structure SA. A 1×1 convolution jump connection from input to output is added to the MSIPA structure to ensure the gradient transfer of the network.
[0012] F′=f3(F) (1)
[0013]
[0014] F represents the feature map of the input MSIPA structure, F′ and F″ both represent the feature maps generated during the MSIPA structure process, f3(·) represents the convolution function with a convolution kernel of 3×3, and f n,m (·) represents the convolution function, n represents the size of the n×n convolution kernel, m is the expansion coefficient of the convolution, AvgP(·) represents average pooling, ⊙ represents channel concatenation,
[0015] F″ is input into the SA structure, assuming that the input feature is F″∈R C×H×W , this step will split the input F″ into g groups along the channel dimension: F″=[F″1,…,F″ g ],R C / g×H×W , for each set of features, the features F″ of any set of the above 1-g groups k Will be split into two branches along the channel dimension: F″ k1 ,F″ k2 ∈R C / 2g×H×W , one branch is used to learn channel attention features, and the other branch is used to learn spatial attention features.
[0016] Preferably, the SA structure uses two types of attention mechanisms, space and channel, in parallel in blocks, so that the two types of attention mechanisms are efficiently combined, wherein the channel attention mechanism adopts the simplest combination of GAP+Scale+Sigmoid, and the process is described as follows:
[0017]
[0018]
[0019] F gp (·) represents global average pooling, W1 and b1 represent the parameters used to scale and shift s respectively, σ represents the Sigmoid activation function, s represents global average pooling, Indicates the accumulation of pixel values at each position (i, j), i represents the index in the spatial height dimension, and j represents the index in the width dimension. Represents the feature map after channel attention, F c Represents the mapping function of the fully connected layer in the channel attention mechanism;
[0020] The spatial attention mechanism first uses Group Norm (GN) to k2 Processing is performed to obtain statistical information at the spatial level, and then F c (·) is enhanced, and the process is described as follows:
[0021]
[0022] W2 and b2 indicate the shape is R C / 2g×H×W Parameters, represents the feature map after spatial attention, GN represents group normalization;
[0023] Then concatenate the two branches so that the number of channels is the same as the number of inputs, F″ k =[F″ k1 ,F″ k2 ]∈R C / g×H×W ,
[0024]
[0025] δ(·) represents the Hardswish activation function, BN(·) represents batch normalization, and F s (·) represents the channel shuffle operation, ⊙ represents channel concatenation, f1(·) represents the convolution function with a convolution kernel size of 1×1, F" g1 and F" g2 It means that the g-th group of features is divided into equal feature map subsets, which are then used for channel and spatial attention mechanism processing respectively.
[0026] Preferably, the distribution-aware key point coordinate representation method in S2 comprises the following steps:
[0027] S2.1, heat map distribution modulation;
[0028] In order to smooth the influence of multiple peaks in the heat map H, the Gaussian kernel K is used to have the same variation as the training data, and the formula is expressed as:
[0029]
[0030] in, represents the convolution operation, H' represents the smoothed heat map after convolution with Gaussian kernel K;
[0031] Using the following transformation, we scale Make its maximum activation equal to H, retaining the size of the original heat map:
[0032]
[0033] Among them, max(·) and min(·) return the maximum and minimum values of the input matrix, Represents the heat map after thermal scaling;
[0034] S2.2, distribution-aware maximum relocalization;
[0035] To obtain accurate location information at the sub-pixel level, assuming that the predicted heat map follows the same two-dimensional Gaussian distribution as the true heat map, it means that the predicted heat map is:
[0036]
[0037] where x is a pixel location in the prediction heatmap, μ represents the Gaussian mean of the joint location to be estimated, T represents the transpose of the matrix, and the covariance Σ is a diagonal matrix, the same as the coordinate encoding,
[0038]
[0039] Where σ is the standard deviation in both directions,
[0040] While keeping the original position of the maximum activation, h(x,μ,Σ) is log-transformed.
[0041]
[0042] μ represents the Gaussian mean of the joint position to be estimated. As an extreme point of the distribution, the first-order derivative of the Gaussian mean μ meets the following conditions:
[0043]
[0044] Using Taylor's theorem, the Taylor series up to the quadratic term is used to approximate the maximum activation of G(μ) in the predicted heat map, where the Taylor series is estimated at the maximum activation m in the predicted heat map,
[0045]
[0046] Where G(μ) represents the value of the function G at the Gaussian mean μ, G(m) represents the value of the function G at μ=m, D'(m) represents the first-order derivative of the function G at μ=m, and D″(m) is the second-order derivative of G evaluated at m, defined as:
[0047] D″(m)=D″(x)| x=m =-Σ -1 (14)
[0048] To approximate μ, m should represent a good rough joint prediction close to μ;
[0049] According to formula (12), formula (13) and formula (14), we can get:
[0050] μ=m-(D″(m)) -1 D′(m) (15)
[0051] Among them, D″(m) and D′(m) are estimated from the prediction heat map;
[0052] S2.3, resolution recovery;
[0053] According to μ obtained in S2.2, the coordinates of the original image space are predicted by the following formula:
[0054]
[0055] Where λ represents the resolution reduction rate, represents the corresponding coordinate point in the image space after resolution reduction, and p represents the coordinate point in the original image space.
[0056] Preferably, the evaluation index SAT for balancing speed and accuracy in S3 is expressed as:
[0057]
[0058] l cpu =p cpu ×n pcc ×2 (18)
[0059] l cpu Indicates logical CPU, p cpu Indicates the number of physical CPUs, n pccIt represents the number of physical CPU cores, EPE represents the average node error on pixels, GFLOPs is used to evaluate the computing power of the model, and Params represents the model parameters.
[0060] Therefore, the present invention adopts the above-mentioned hand posture assessment method based on RGB image, which has the following beneficial effects:
[0061] 1) A MSIPA-HandNet algorithm is proposed to achieve a better balance between speed and accuracy for hand pose estimation by exploiting multi-scale information-aware structure and distribution-aware coordinate representation;
[0062] 2) Based on the speed / accuracy trade-off problem, a comprehensive metric is proposed to evaluate hand pose estimation and provide valuable insights for optimizing the hand pose estimation model;
[0063] 3) Use gesture key point location information for easy application.
[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] Figure 1 is a flow chart of an embodiment of a method for hand posture assessment based on RGB images of the present invention;
[0066] Figure 2 It is a flow chart of an embodiment of MSIPA-HandNet, a hand posture assessment method based on RGB images, of the present invention;
[0067] Figure 3 It is a flow chart of an embodiment of MSIPA of a hand posture assessment method based on RGB images of the present invention. DETAILED DESCRIPTION
[0068] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.
[0069] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.
[0070] Embodiment 1
[0071] As shown in the figure, the present invention provides a hand posture assessment method based on RGB image, comprising the following steps:
[0072] S1. Propose a multi-scale information perception model based on attention mechanism to obtain gesture key point information;
[0073] The multi-scale information perception model based on the attention mechanism is an encoder-decoder structure. The encoder-decoder structure consists of downsampling and upsampling. In downsampling, 3×3 depth-separable convolution is used as the basic unit, and in upsampling, 4×4 deconvolution blocks are used as the basic unit.
[0074] Downsampling includes the multi-scale information perception structure MSIPA based on the attention mechanism. MSIPA consists of an improved hole convolution pyramid and a randomly shuffled attention structure SA. A 1×1 convolution jump connection from input to output is added to the MSIPA structure to ensure the gradient transfer of the network.
[0075] F′=f3(F) (1)
[0076]
[0077] F represents the feature map of the input MSIPA structure, F′ and F″ both represent the feature maps generated during the MSIPA structure process, f3(·) represents the convolution function with a convolution kernel of 3×3, and f n,m (·) represents the convolution function, n represents the size of the n×n convolution kernel, m is the expansion coefficient of the convolution, AvgP(·) represents average pooling, ⊙ represents channel concatenation,
[0078] F″ is input into the SA structure, assuming that the input feature is F″∈R C×H×W , this step will split the input F″ into g groups along the channel dimension: F″=[F″1,...,F″ g ],R C / g×H×W , for each set of features, the features F″ of any set of the above 1-g groups k Will be split into two branches along the channel dimension: F″ k1 ,F″ k2 ∈R C / 2g×H×W , one branch is used to learn channel attention features, and the other branch is used to learn spatial attention features.
[0079] The SA structure captures the dependencies between channels and the pixel-level relationships in space in channel attention and spatial attention respectively, while effectively controlling the amount of computation. The SA attention mechanism inherits the design concept of the SGE attention mechanism, introduces channel random mixing operations, and uses the spatial and channel attention mechanisms in parallel in blocks, so that the two types of attention mechanisms are efficiently combined. The channel attention mechanism uses the simplest combination of GAP+Scale+Sigmoid. The process is described as follows:
[0080]
[0081]
[0082] Fgp (·) represents global average pooling, W1 and b1 represent the parameters used to scale and shift s respectively, σ represents the Sigmoid activation function, s represents global average pooling, Indicates the accumulation of pixel values at each position (i, j), i represents the index in the spatial height dimension, and j represents the index in the width dimension. Represents the feature map after channel attention, F c Represents the mapping function of the fully connected layer in the channel attention mechanism;
[0083] The spatial attention mechanism first uses Group Norm (GN) to k2 Processing is performed to obtain statistical information at the spatial level, and then F c (·) is enhanced, and the process is described as follows:
[0084]
[0085] W2 and b2 indicate the shape is R C / 2g×H×W Parameters, represents the feature map after spatial attention, GN represents group normalization,
[0086] Then concatenate the two branches so that the number of channels is the same as the number of inputs, F″ k =[F″ k1 ,F″ k2 ]∈R C / g×H×W ,
[0087]
[0088] δ(·) represents the Hardswish activation function, BN(·) represents batch normalization, and F s (·) represents the channel shuffle operation, ⊙ represents channel concatenation, f1(·) represents the convolution function with a convolution kernel size of 1×1, F" g1 and F" g2 It means that the g-th group of features is divided into equal feature map subsets, which are then used for channel and spatial attention mechanism processing respectively.
[0089] S2, more accurate estimation of hand posture through distribution-aware key point coordinate representation method;
[0090] The distribution-aware key point coordinate representation method in S2 includes the following steps:
[0091] S2.1, heat map distribution modulation;
[0092] Since the proposed coordinate decoding method is based on the Gaussian distribution assumption, compared with the training heat map data, the predicted heat map usually does not show a good Gaussian structure and presents multiple peaks near the maximum activation, which may have a negative impact on the performance of our decoding method. To solve this problem, it solves this problem by regulating the heat map distribution in advance. In order to smooth the influence of multiple peaks in the heat map H, the Gaussian kernel K is used to have the same changes as the training data, and the formula is expressed as:
[0093]
[0094] in, represents the convolution operation, H' represents the smoothed heat map after convolution with Gaussian kernel K;
[0095] Using the following transformation, we scale Make its maximum activation equal to H, retaining the size of the original heat map:
[0096]
[0097] Among them, max(·) and min(·) return the maximum and minimum values of the input matrix, Represents the heat map after thermal scaling;
[0098] S2.2, distribution-aware maximum relocalization;
[0099] To obtain accurate location information at the sub-pixel level, assuming that the predicted heat map follows the same two-dimensional Gaussian distribution as the true heat map, it means that the predicted heat map is:
[0100]
[0101] where x is a pixel location in the prediction heatmap, μ represents the Gaussian mean of the joint location to be estimated, T represents the transpose of the matrix, and the covariance Σ is a diagonal matrix, the same as the coordinate encoding,
[0102]
[0103] Where σ is the standard deviation in both directions,
[0104] While keeping the original position of the maximum activation, h(x,μ,∑) is log-transformed.
[0105]
[0106] μ represents the Gaussian mean of the joint position to be estimated. As an extreme point of the distribution, the first-order derivative of the Gaussian mean μ meets the following conditions:
[0107]
[0108] Using Taylor's theorem, the Taylor series up to the quadratic term is used to approximate the maximum activation of G(μ) in the predicted heat map, where the Taylor series is estimated at the maximum activation m in the predicted heat map,
[0109]
[0110] Where G(μ) represents the value of the function G at the Gaussian mean μ, G(m) represents the value of the function G at μ=m, D'(m) represents the first-order derivative of the function G at μ=m, and D″(m) is the second-order derivative of G evaluated at m, defined as:
[0111] D″(m)=D″(x)| x=m =-Σ -1 (14)
[0112] To approximate μ, m should represent a good rough joint prediction close to μ;
[0113] According to formula (12), formula (13) and formula (14), we can get:
[0114] μ=m-(D″(m)) -1 D′(m) (15)
[0115] Among them, D″(m) and D′(m) are estimated from the prediction heat map;
[0116] S2.3, resolution recovery;
[0117] According to μ obtained in S2.2, the coordinates of the original image space are predicted by the following formula:
[0118]
[0119] Where λ represents the resolution reduction rate, represents the corresponding coordinate point in the image space after resolution reduction, and p represents the coordinate point in the original image space.
[0120] S3. Choose the right balance of speed and accuracy for a given application and platform based on the deep learning model;
[0121] If time is used directly to measure the speed of the model during inference, it must be measured on the device. This also involves optimizations on the device side such as ARM / NOEN, fixed-point floating-point, and quantization, which is very complex. Therefore, in general, GFLOPs are used to estimate computing power. Since the model inference speed is most related to the computing power of the hardware platform, computing power is often used to judge the model inference speed capability, and the number of logical CPUs of the hardware platform plays an important role in the selection.
[0122] The evaluation index SAT that balances speed and accuracy is expressed as:
[0123]
[0124] l cpu =p cpu ×n pcc ×2 (18)
[0125] l cpu Indicates logical CPU, p cpu Indicates the number of physical CPUs, n pcc It represents the number of physical CPU cores, EPE represents the average node error on pixels, GFLOPs is used to evaluate the computing power of the model, and Params represents the model parameters.
[0126] The hardware platform parameters based on the SAT evaluation index are shown in Table 3-1:
[0127] Table 3-1 Parameters of each hardware platform
[0128]
[0129] Hand gesture estimation has been widely used in motion recognition, animation, games, and motion capture systems. Unfortunately, hand gesture estimation is often seen in academic research, and it is difficult for ordinary readers to experience its effects in real life. There are two main reasons: 1. There are insufficient applications for hand gesture estimation on edge computing devices; 2. The hand gesture estimation model in academic research is large and difficult to deploy on edge computing devices. Based on the above problems, a hand gesture evaluation method based on RGB images is used to implement PPT page turning applications, PPT blackboard applications, etc.
[0130] Therefore, the present invention adopts the above-mentioned hand posture evaluation method based on RGB images and proposes a MSIPA-HandNet algorithm, which uses multi-scale information perception structure and distribution-aware coordinate representation to achieve a better balance between speed and accuracy for hand posture estimation. Based on the speed / accuracy trade-off problem, a comprehensive metric is proposed to evaluate hand posture estimation, and valuable opinions are provided for optimizing the hand posture estimation model. The position information of gesture key points is used for easy application.
[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A hand posture assessment method based on RGB images, characterized in that: The following steps are involved: S1. Propose a multi-scale information perception model based on attention mechanism to obtain gesture key point information; The multi-scale information perception model based on the attention mechanism is an encoder-decoder structure, which is composed of downsampling and upsampling. The 3×3 depth-separable convolution is used as the basic unit in downsampling, and the 4×4 deconvolution block is used as the basic unit in upsampling. The downsampling includes a multi-scale information perception structure MSIPA based on the attention mechanism. MSIPA consists of an improved hole convolution pyramid and a randomly shuffled attention structure SA. A 1×1 convolution jump connection from input to output is added to the MSIPA structure to ensure the gradient transfer of the network. F′=f3(F) (1) F represents the feature map of the input MSIPA structure, F′ and F″ both represent the feature maps generated during the MSIPA structure process, f3(·) represents the convolution function with a convolution kernel of 3×3, and f n,m (·) represents the convolution function, n represents the size of the n×n convolution kernel, and m is the expansion coefficient of the convolution. AvgP(·) represents average pooling, ⊙ represents channel concatenation, F″ is input into the SA structure, assuming that the input feature is F″∈R C×H×W , this step will split the input F″ into g groups along the channel dimension: F″=[F″1,...,F″ g ],R C / g×H×W , for each set of features, the features F of any set of the above 1-g groups k ″Will be split into two branches along the channel dimension: F″ k1 ,F″ k2 ∈R C / 2g×H×W , one branch is used to learn channel attention features, and the other branch is used to learn spatial attention features; S2, more accurate estimation of hand posture through distribution-aware key point coordinate representation method; The distribution-aware key point coordinate representation method comprises the following steps: S2.1, heat map distribution modulation; In order to smooth the influence of multiple peaks in the heat map H, the Gaussian kernel K is used to have the same variation as the training data, and the formula is expressed as: in, represents the convolution operation, H′ represents the smoothed heat map after convolution with Gaussian kernel K; Using the following transformation, we scale Make its maximum activation equal to H, retaining the size of the original heat map: Among them, max(·) and min(·) return the maximum and minimum values of the input matrix, Represents the heat map after thermal scaling; S2.2, distribution-aware maximum relocalization; To obtain accurate location information at the sub-pixel level, assuming that the predicted heat map follows the same two-dimensional Gaussian distribution as the true heat map, it means that the predicted heat map is: where x is a pixel location in the prediction heatmap, μ represents the Gaussian mean of the joint location to be estimated, T represents the transpose of the matrix, and the covariance Σ is a diagonal matrix, the same as the coordinate encoding, Where σ is the standard deviation in both directions, While keeping the original position of the maximum activation, h(x; μ, Σ) is log-transformed. μ represents the Gaussian mean of the joint position to be estimated. As an extreme point of the distribution, the first-order derivative of the Gaussian mean μ meets the following conditions: Using Taylor's theorem, the Taylor series up to the quadratic term is used to approximate the maximum activation of G(μ) in the predicted heat map, where the Taylor series is estimated at the maximum activation m in the predicted heat map, Among them, G(μ) represents the value of function G at the Gaussian mean μ, G(m) represents the value of function G at μ-m, D′(m) represents the first-order derivative of function G at μ-m, and D″(m) is the second-order derivative of G evaluated at m, which is defined as: D″(m)=D″(x) x=m =-Σ -1 (14) To approximate μ, m should represent a good rough joint prediction close to μ; According to formula (12), formula (13) and formula (14), we can get: μ=m-(D″(m)) -1 D′(m) (15) Among them, D″(m) and D′(m) are estimated from the prediction heat map; S2.3, resolution recovery; According to μ obtained in S2.2, the coordinates of the original image space are predicted by the following formula: Where λ represents the resolution reduction rate, represents the corresponding coordinate point in the image space after the resolution is reduced, and p represents the coordinate point in the original image space; S3. Choose the appropriate balance of speed and accuracy for a given application and platform based on the deep learning model.
2. The hand posture assessment method based on RGB image according to claim 1, characterized in that: The SA structure uses two types of attention mechanisms, space and channel, in parallel, so that the two types of attention mechanisms are efficiently combined. The channel attention mechanism uses the simplest combination of GAP+Scale+Sigmoid. The process is described as follows: F gp (·) represents global average pooling, W1 and b1 represent the parameters used to scale and shift s respectively, σ represents the Sigmoid activation function, s represents global average pooling, Indicates the accumulation of pixel values at each position (i, j), i represents the index in the spatial height dimension, and j represents the index in the width dimension. Represents the feature map after channel attention, F c Represents the mapping function of the fully connected layer in the channel attention mechanism; The spatial attention mechanism first uses Group Norm (GN) to k2 Processing is performed to obtain statistical information at the spatial level, and then F c (·) is enhanced, and the process is described as follows: W2 and b2 indicate the shape is R C / 2g×H×W Parameters, represents the feature map after spatial attention, GN represents group normalization; Then concatenate the two branches so that the number of channels is the same as the number of inputs, F″ k =[F″ k1 ,F″ k2 ]∈R C / g×H×W , δ(·) represents the Hardswish activation function, BN(·) represents batch normalization, and F s (·) represents the channel shuffle operation, ⊙ represents channel concatenation, f1(·) represents the convolution function with a convolution kernel size of 1×1, and F″ g-1 and F″ g It means that the g-1th group and the gth group features are divided into equal feature map subsets, which are then used for channel and spatial attention mechanism processing respectively.
3. The hand posture assessment method based on RGB image according to claim 1, characterized in that: The evaluation index SAT for balancing speed and accuracy in S3 is expressed as: l cpu =p cpu ×n pcc ×2 (18) l cpu Indicates logical CPU, p cpu Indicates the number of physical CPUs, n pcc It represents the number of physical CPU cores, EPE represents the average node error on the pixel, GFLOPs is used to evaluate the computing power of the model, and Params represents the model parameters.
Citation Information
Patent Citations
Light-weight three-dimensional hand posture estimation method based on RGB image
CN115880724A
3D hand posture estimation method based on global topological structure information learning
CN117423164A
