Method for detecting hand and foot postures and hand and foot DR auxiliary positioning method

Abstract

The invention relates to a hand and foot posture detection method and a hand and foot DR auxiliary positioning method. The hand and foot posture detection method comprises the following steps: step 1, acquiring an RGB-D image containing a hand or a foot; step 2, intercepting an area where a hand or a foot is located in the RGB-D image; step 3, acquiring a hand and foot posture estimation model, wherein the hand and foot posture estimation model is of a Transform architecture; 4, taking the image intercepted in the step 2 as the input of the hand and foot posture estimation model, and outputting the key point position of the hand or the foot by using the hand and foot posture estimation model; and step 5, obtaining the hand and foot postures based on the key point positions and the depth information of the RGB-D image. According to the method, the hand and foot postures can be accurately predicted by utilizing the trained hand and foot posture estimation model of the Transform architecture, whether the current posture of the patient is appropriate or not is evaluated in combination with specific positioning requirements, and then the personalized exposure parameters are calculated, so that the workload of a radiotechnician is relieved, and the error probability is reduced.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a method for detecting the posture of hands and feet and a DR-assisted positioning method of hands and feet. Background Art

[0002] Hand DR radiography (Direct Digit Radiography) is a key medical imaging technology that is widely used to evaluate the condition of the wrist and related bones and joints, and to diagnose fractures, dislocations, arthritis, osteoporosis and other diseases. Foot DR radiography is used to examine the bone structure of the foot in detail, and is mainly used to diagnose fractures, sprains, arthritis, foot deformities and other diseases. Although hand and foot DR radiography target different parts of the human body, they have significant similarities in technical application and diagnostic purposes.

[0003] When performing DR photography of hands and feet, since the hands and feet contain complex structures such as multiple small bones, joints, tendons and soft tissues, accurately capturing and clearly displaying all key structures faces multiple challenges, such as fine cracks in bones, slight changes between small bones and tiny joint gaps. Take the hand as an example: first, unlike the human body or face, the hand has a uniform shape and lacks obvious local features; second, the hand has a complex structure, many joints, and a high degree of freedom of movement; in addition, mutual occlusion between fingers and interaction between hands and objects often make certain key points difficult to observe directly; finally, changing lighting conditions and complex backgrounds can also affect the detection accuracy of key points.

[0004] Therefore, how to provide a method that can accurately capture and clearly display the postures of hands and feet is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] The present invention provides a method for detecting the posture of hands and feet and a DR-assisted positioning method of hands and feet to solve the above-mentioned technical problems.

[0006] In order to solve the above technical problems, the present invention provides a method for detecting hand and foot postures, comprising the following steps:

[0007] Step 1: Get an RGB-D image containing hands or feet;

[0008] Step 2: intercepting the area where the hand or foot is located in the RGB-D image;

[0009] Step 3: Obtain a hand and foot posture estimation model, wherein the hand and foot posture estimation model is a Transformer architecture;

[0010] Step 4: using the image captured in step 2 as the input of the hand and foot posture estimation model, and using the hand and foot posture estimation model to output the key point positions of the hand or foot;

[0011] Step 5: Based on the key point positions and the depth information of the RGB-D image, obtain the hand and foot postures.

[0012] Preferably, in step 2, a positioning module is used to preliminarily locate the area where the hands and feet are located, and a tracking module is used to determine the interception area according to the changing trend of the positions of the hands and feet.

[0013] Preferably, the hand and foot posture estimation model includes a CNN backbone network, a hand posture estimation module and a foot posture estimation module, and the hand posture estimation module and the foot posture estimation module share the CNN backbone network.

[0014] Preferably, the hand posture estimation module and the foot posture estimation module respectively include a Transformer encoder layer and a head network, which are used to output heat maps of hand key points and foot key points respectively.

[0015] Preferably, in step 3, the training process of the hand and foot posture estimation model includes:

[0016] Construct hand key point dataset and foot key point dataset;

[0017] Using the samples in the hand key point data set and their corresponding labels to train the hand and foot posture estimation model, and obtain the network parameters of the CNN backbone network and the hand posture estimation module;

[0018] Freeze the network parameters of the CNN backbone network, use the samples in the foot key point data set and their corresponding labels to train the foot posture estimation module, and obtain the network parameters of the Transformer encoder layer and the head network in the foot posture estimation module.

[0019] Preferably, the Transformer encoder layer includes a multi-head attention layer and a feedforward network layer.

[0020] Preferably, the head network adopts a single-layer deconvolution.

[0021] Preferably, in step 5, the hand and foot postures are 2.5D coordinates of key points of the hands or feet, including 2D coordinates of all key points and 3D coordinates of visible key points.

[0022] The present invention also provides a DR-assisted positioning method for hands and feet, comprising: acquiring the postures of the hands and feet based on the method described above, and estimating exposure parameters according to the positioning requirements of the shooting position.

[0023] Preferably, the exposure parameters at least include a region of interest and an exposure area for exposure control.

[0024] Compared with the prior art, the method for detecting the posture of hands and feet and the DR-assisted positioning method of hands and feet provided by the present invention have the following advantages:

[0025] 1. The present invention constructs a hand and foot posture estimation model based on a Transformer architecture and uses samples to train and optimize the model, thereby accurately predicting the hand and foot posture;

[0026] 2. The present invention can use the predicted hand and foot postures, combined with specific positioning requirements, to evaluate whether the patient's current posture is appropriate, and then calculate personalized exposure parameters, which helps to reduce the workload of radiographers and reduce the probability of errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1a This is an X-ray image of the hand;

[0028] Figure 1b Schematic diagram of the key points of the hand;

[0029] Figure 2a This is an X-ray image of the foot;

[0030] Figure 2b This is a schematic diagram of the key points of the foot;

[0031] Figure 3 This is a schematic diagram of the positioning for DR photography of some hands and feet;

[0032] Figure 4 It is a framework diagram of a DR-assisted positioning method for hands and feet in a specific embodiment of the present invention;

[0033] Figure 5 It is a schematic diagram of the tracking effect of the tracking module in a specific embodiment of the present invention;

[0034] Figure 6 This is an example of some hand key point datasets;

[0035] Figure 7 This is an example of some foot key point datasets;

[0036] Figure 8 A schematic diagram of the effects of HFPET provided by the present invention and the existing MediaPipe on the CMU Panoptic dataset;

[0037] Fig. 9 A performance comparison chart of HFPET provided by the present invention and the existing MediaPipe on the DHGCD dataset;

[0038] Fig.10 A performance comparison chart of HFPET provided by the present invention and the existing MediaPipe on the CMU Panoptic dataset;

[0039] Fig.11 This is an example of hand positioning output obtained by using the hand and foot DR-assisted positioning method proposed in the present invention;

[0040] Fig.12 This is an example of foot positioning output obtained by using the hand and foot DR-assisted positioning method proposed in the present invention. DETAILED DESCRIPTION

[0041] In order to describe the technical solution of the above invention in more detail, specific embodiments are listed below to demonstrate the technical effects; it should be emphasized that these embodiments are used to illustrate the present invention but not to limit the scope of the present invention.

[0042] In order to complete the hand and foot posture detection, first of all, we need to understand the basic structure of the hand and foot, such as Figure 1a As shown in the figure, the hand is composed of 27 bones, which can be divided into three groups: 1) The most distal bones of the hand are the phalanges, which have a total of 14 bones and constitute the fingers; 2) The second group of bones are the metacarpal bones, which have a total of 5 bones and constitute the palm of each hand; 3) The third group of bones are the carpal bones, which have a total of 8 bones and constitute the wrist. Figure 2a As shown in the figure, the bones of the foot are very similar to those of the wrist, with a total of 26 bones, which can also be divided into three groups: 1) The most distal bones of the foot are the phalanges, a total of 14 bones, which make up the toes; 2) The 5 bones on the instep are the metatarsals; 3) The 7 large bones in the proximal foot are called tarsal bones. In the hand and foot structure, the key points of the hand and foot are defined as follows: Figure 1b and 2b shown.

[0043] In addition, according to different diagnostic needs, patients need to take specific postures for DR imaging, such as Figure 3 As shown, the lesion site can be accurately captured and clearly displayed.

[0044] Based on the above technical background, the present invention provides a method for detecting hand and foot postures, such as Figure 4 As shown, the following steps are included:

[0045] Step 1: Get an RGB-D image containing hands or feet;

[0046] Step 2: intercepting the area where the hand or foot is located in the RGB-D image;

[0047] Step 3: Obtain a hand and foot posture estimation model, wherein the hand and foot posture estimation model is a Transformer architecture;

[0048] Step 4: using the image captured in step 2 as the input of the hand and foot posture estimation model, and using the hand and foot posture estimation model to output the key point positions of the hand or foot;

[0049] Step 5: Based on the key point positions and the depth information of the RGB-D image, obtain the hand and foot postures.

[0050] The above method can be used to accurately predict the posture of hands and feet.

[0051] In some embodiments, in step 2, a positioning module is used to preliminarily locate the area where the hands and feet are located, a tracking module is used to determine the interception area using the changing trend of the hand and foot positions, and the intercepted image of appropriate size is used as the input of the hand and foot posture estimation model.

[0052] Specifically, the positioning module preliminarily locates the position of the hands and feet in the initial stage of the system or after tracking fails; the tracking module estimates the interception position of the current frame using the key point position estimated in the previous frame. Taking the hand application as an example, the positioning module based on human posture estimation can find the approximate position of the hand, but the intercepted image may not completely cover the hand area, which in turn affects the result of posture estimation, as shown in Figure 5(a). The tracking module can gradually optimize the positioning result based on the hand and foot posture estimation result of the previous frame, thereby ensuring the result of posture estimation, as shown in Figure 5(a). Figure 5 As shown in (b)-(e).

[0053] The localization module uses a 2.5D human pose estimation model to estimate the position of the hand through the position of the wrist and elbow. Assuming that the hand extends from the elbow to the wrist and the hand length is the same as the forearm length, the interception area is defined as a rectangular area with a length of 1.5 times the forearm length and an aspect ratio of 256 / 192. Then, the position of the rectangular area is adjusted so that the part to be photographed is at the center of the intercepted image. For example, when photographing the wrist, the center of the rectangular area is placed at the position of the wrist key point; when photographing the finger, the interception area is moved from the elbow to the wrist to the wrist key point at one-quarter of its length. In the tracking module, the hand size and center are determined by the high-confidence position of some hand key points obtained in the previous frame, and then the interception area is determined. In the foot pose estimation task, the same strategy is adopted. First, the coordinates of the knee and ankle key points given by the human pose estimation model are used to preliminarily locate the foot position. Then, the result of the foot pose estimation is used as the basis for intercepting the next frame of the image. The positioning module makes full use of the existing human posture estimation module to achieve preliminary positioning, and the introduction of the tracking module can not only reduce the call to the human posture estimation model, thereby reducing the amount of calculation, but also locate the hands and feet more accurately, ensuring the effect of subsequent posture estimation.

[0054] In some embodiments, the hand-foot pose estimation model (Hand-Foot Pose Estimation Transformer, HFPET) includes a CNN backbone network, a hand pose estimation module and a foot pose estimation module, and the hand pose estimation module and the foot pose estimation module share the CNN backbone network.

[0055] In some embodiments, the hand pose estimation module and the foot pose estimation module respectively include a Transformer encoder layer and a head network, which are used to output heat maps of hand key points and foot key points respectively. The CNN backbone network is used to extract low-level image features; the Transformer encoder layer is used to capture long-range spatial interactions of feature vectors between different positions; and the head network is used to predict the heat map of key points.

[0056] Specifically, the CNN backbone network can be selected from many common CNN networks. For example, ResNet50 can be selected as the backbone network. In this embodiment, EfficientNet is selected as the backbone network. The size of the input image is 256×192×3 (H I ×W I ×C I ), the feature tensor size obtained after passing through the backbone network is 32×24×256 (H×W×d), and the tensor is expanded along the spatial dimension to form a size of 768×256 as the input of the Transformer encoder layer. The Transformer encoder layer is composed of three stacked encoders, each of which includes a multi-head (8 heads in this embodiment) self-attention layer and a feedforward network layer. The output size of the Transformer encoder layer is the same as the input size. In the head network, the output of the Transformer encoder layer is first restored to a size of 32×24×256, and then upsampled using a 4×4 transposed convolution layer to obtain a feature map of size 64×48×256. Finally, a 1×1 convolution is used to reduce the channel dimension to 21, the number of hand key points, to obtain a heat map of the key points.

[0057] The position of the key point can be determined by finding the maximum value point in each heat map. In order to improve the accuracy of the position, sub-pixel refinement can be achieved through Gaussian fitting, thereby correcting the positioning error caused by the reduction in resolution. At the same time, the original value of the maximum value point in the heat map is used as the confidence level to judge the credibility of the key point position estimate.

[0058] As mentioned earlier, the bone structure of the foot is very similar to that of the hand, so a similar network structure can be used to achieve foot pose estimation. Compared with the hand pose estimation network, the foot pose estimation network only differs from the last 1×1 convolutional layer in the head network, and its output channel dimension is the number of foot key points, that is, 12.

[0059] By combining the 2D key point coordinates provided by the hand and foot posture estimation model with the corresponding depth image, the 2.5D hand and foot posture can be obtained, that is, the 2D coordinates of all key points and the 3D coordinate information of some visible key points.

[0060] In some embodiments, in step 3, the training process of the hand and foot posture estimation model may include:

[0061] Construct hand key point dataset and foot key point dataset;

[0062] Using the samples in the hand key point data set and their corresponding labels to train the hand and foot posture estimation model, and obtain the network parameters of the CNN backbone network and the hand posture estimation module;

[0063] Freeze the network parameters of the CNN backbone network, use the samples in the foot key point data set and their corresponding labels to train the foot posture estimation module, and obtain the network parameters of the Transformer encoder layer and the head network in the foot posture estimation module.

[0064] Specifically, this application uses the percentage of correct keypoints (PCK) as a metric for evaluating the pose estimation task. It measures the accuracy between the keypoints predicted by the model (such as the joint positions of the human body) and the actual keypoints, and is specifically defined as:

[0065]

[0066] Where i represents the index of the key point, i = [1, ..., 21] in the hand pose estimation task, s represents the sample index, D represents the number of samples in the dataset, and y si represents the real coordinates of key point i, is the coordinate predicted by the system. w, h are the width and height of the original image, respectively. σ is the normalization threshold, and l(·) is the indicator function. If the L2 distance between the predicted keypoint coordinates and the true keypoint coordinates is less than σ after normalization, the output value of the indicator function is 1, otherwise it is 0. is the PCK of the i-th key point, PCK σ It represents the average PCK of all key points.

[0067] Currently, there are multiple public datasets that can be used to train hand pose estimation models, but these datasets usually have limited shooting scenes or limited hand changes. For example, CMU Panoptic Hand Dataset is an important dataset launched by Carnegie Mellon University, focusing on the field of gesture recognition research. The dataset was collected in a panoramic studio (Panoptic Studio) equipped with more than 500 synchronized cameras, providing rich perspectives and detailed action annotations. However, the image background is relatively simple and cannot be used alone as a training set for hand pose estimation. In order to meet the diverse application scenarios and hand poses, this application constructs a hand key point dataset. The original image comes from complex and varied images collected on the Internet, and only the image of the hand area is captured. In the annotation process, MediaPipe is first used to obtain the initial key point estimate, and then the possible deviations are manually corrected. The dataset contains a total of 46,500 images, of which 80% are randomly selected as training sets and the rest are used as validation sets. The dataset is called DHGCD, i.e. Diverse Hand Gesture and ContextDataset. Figure 6 Shows some examples in DHGCD.

[0068] The network is implemented in PyTorch, with a batch size of 64, an Adam optimizer, and a dropout probability of 0.1. The entire network is trained for 300 cycles, and the cosine annealing strategy is applied to decay the learning rate from an initial value of 0.002 to a final value of 0.0001.

[0069] Regarding the training of the foot pose estimation module, since there is currently no available public dataset for the foot pose estimation task, this application has collected and organized a dataset for the foot DR scenario. In view of the high similarity between foot pose estimation and hand pose estimation tasks, as well as the existing large-scale hand datasets, the size of the foot dataset required is relatively small. This application uses an RGB-D camera to capture video images of 8 people during foot DR shooting, from which images of the foot area are captured, the similarity between frames is calculated, and some frames with certain differences are extracted as original images. Two annotation personnel follow the Figure 2b The foot key points are annotated according to the rules shown, and the outputs of the two are averaged as the final annotated data. The final dataset consists of 1283 images, of which 1032 are used as training sets and the rest are used as validation sets. This dataset is called DRFD, which stands for DR Foot Dataset. Figure 7 Some examples in DRFD are shown.

[0070] This application uses transfer learning technology to train the foot posture estimation module. The specific method is to use the weights of the trained hand posture estimation module to initialize the network parameters except the head network, and completely freeze the parameters of the backbone network during the training process. Different parts of the Transformer encoder are tuned, and the parameters of the head network are completely unfrozen and trained. This application sets the batch size to 8 and uses the Adam optimizer for optimization, with the dropout probability set to 0.1. The initial learning rate for the tuning part is set to 0.001, and the initial learning rate for the head network is set to 0.002. The entire network was trained for 150 cycles, and the cosine annealing strategy was used to gradually decay the learning rate to the final value of 0.0001.

[0071] In the process of transfer learning from hand pose estimation to foot pose estimation, this application conducts experiments to better understand the role of each part in the network and achieve the best performance. First, the deconvolution layer in the head network amplifies the output of the Transformer encoder to generate an output of the size of a heat map. The increase in the number of deconvolution layers can increase the size of the heat map accordingly, thereby improving the spatial resolution of the heat map. 0 lists the effects of single-layer deconvolution and double-layer deconvolution. It can be seen that the double-layer deconvolution doubles the size of the heat map, PCK σ＝0.03 The improvement is 0.4%, but the number of parameters increases from 4,207,948 to 5,257,036, an increase of 25%. Considering the computational cost and benefits, using a single-layer deconvolution is a better choice.

[0072] Table 1-1 The impact of the number of deconvolution layers

[0073] Number of deconvolution layers Heatmap size Parameter quantity <![CDATA[PCK σ＝0.03 / %]]> 1 64x48 4,207,948 93.4 2 128x96 5,257,036 93.8

[0074] Second, the Transformer encoder plays a crucial role in the entire network. During the tuning process, by freezing different parts of the encoder, we can better understand its importance. Figure 0 shows the results of different tuning strategies. It can be seen that when the encoder is completely frozen and only the head network is tuned, PCK σ＝0.03 Only 75%; when unfreezing the multi-head attention layer and the feed-forward network layer, PCK σ＝0.03 The PCK σ＝0.03 The highest is 94%. The results show that the Transformer encoder layer is very important in capturing the relationship between key points, and the multi-head attention layer and feedforward network layer cannot be ignored.

[0075] Table 1-2 Comparison of results obtained by different tuning strategies

[0076] Multi-Head Attention Layer Feedforward Network Layer Head Network <![CDATA[PCK σ＝0.03 / %]]> × × √ 75 √ × √ 92 × √ √ 93 √ √ √ 94

[0077] In the table: “×” indicates freezing, and “√” indicates participation in training.

[0078] In addition, to measure the performance of HFPET, this application conducted a validation on the hand pose estimation task on the DHGCD and CMU Panoptic datasets and compared it with MediaPipe. Figure 8 The effects of HFPET and MediaPipe on the CMU Panoptic dataset are qualitatively demonstrated. The first image in each group of images is the labeled data of the dataset, the second image is the estimated result of MediaPipe, and the third image is the estimated result of HFPET. It can be seen that HFPET and MediaPipe can give accurate results in most cases, and in some cases are even more reasonable than the labeled data of the dataset, as shown in the (a) group of images. In some cases, the positions of some key points of HFPET and MediaPipe will deviate, as shown in the (g) and (i) groups of images. In addition, there are cases where MediaPipe cannot be recognized, as shown in the last two groups of images.

[0079] In order to quantitatively measure the performance of HFPET, the PCK of HFPET and MediaPipe under different normalization thresholds σ on the DHGCD and CMU Panoptic datasets is calculated. The results are shown in Fig. 9 and Fig.10 As shown. It should be noted that since MediaPipe has unrecognizable situations, that is, it is impossible to give any key point coordinates, for the sake of fairness, two strategies are used when calculating MediaPipe's PCK. One is to set the predicted coordinate position to the center of the image and participate in the PCK calculation when the estimated value cannot be given. The data is marked as "MediaPipe" in the figure; the other is to exclude this situation from the PCK calculation, that is, "MediaPipe (filtered)" in the figure. The results show that on the DHGCD dataset, HFPET outperforms MediaPipe at all σ thresholds, which may be related to the training of HFPET on this dataset. On the CMU Panoptic dataset, after excluding unrecognizable situations, MediaPipe outperforms HFPET at smaller σ thresholds, and begins to be inferior to HFPET after σ>0.07, and the gap is getting bigger and bigger. In addition, there is no unrecognizable situation for HFPET. Overall, HFPET performs better than MediaPipe.

[0080] The present invention also provides a DR-assisted positioning method for hands and feet, comprising: acquiring the postures of the hands and feet based on the method described above, and estimating exposure parameters according to the positioning requirements of the shooting position.

[0081] Specifically, after obtaining the 2.5D key point coordinates of the hand or foot, the exposure parameters can be further estimated according to the specific requirements of each shooting position. The exposure parameters at least include the exposure control region of interest (DAEC ROIs) and the position of the exposure area. Taking the AP position of the hand as an example, the calculation method of the exposure area and DAEC ROI is as follows:

[0082]

[0083] This application realizes auxiliary positioning applications for 18 body positions of the hands and wrists and 16 body positions of the feet. Fig.11 and Fig.12 Some application examples of hand and foot assisted positioning are qualitatively shown. As can be seen from the figure, the posture estimation module accurately identifies the positions of the key points of the hands and feet, and on this basis the system calculates the exposure area and DAEC ROIs required for the corresponding projection. Three experienced radiographers evaluated these results and agreed that the system's estimation results are very accurate.

[0084] In summary, as a part of the intelligent DR system, the present invention locates the position of the hand or foot with the help of the human posture estimation module, and tracks it through the hand or foot posture estimation results; uses the independently collected and sorted highly diverse hand dataset DHGCD and the foot dataset DRFD dedicated to the DR shooting process; based on the Transformer architecture, a hand and foot posture estimation model DFPET is constructed and trained. The model includes two branches for hand and foot posture estimation. The two branches have almost the same network structure and share the same CNN backbone network. DFPET shows better performance than MediaPipe in the hand posture estimation task, and successfully transfers this capability to the foot posture estimation task. Combined with the key point coordinates of the hands and feet and the specific positioning requirements, an auxiliary positioning algorithm that can calculate the exposure area and DAEC ROIs is further developed, and the recommended information given is accurate and effective.

[0085] Obviously, those skilled in the art can make various changes and modifications to the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the invention fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A method for detecting hand and foot postures, characterized in that: The steps include: Step 1: Get an RGB-D image containing hands or feet; Step 2: intercepting the area where the hand or foot is located in the RGB-D image; Step 3: Obtain a hand and foot posture estimation model, wherein the hand and foot posture estimation model is a Transformer architecture; Step 4: using the image captured in step 2 as the input of the hand and foot posture estimation model, and using the hand and foot posture estimation model to output the key point positions of the hand or foot; Step 5: Based on the key point positions and the depth information of the RGB-D image, obtain the hand and foot postures.

2. The method for detecting hand and foot postures according to claim 1, characterized in that: In step 2, the positioning module is used to preliminarily locate the area where the hands and feet are located, and the tracking module is used to determine the interception area based on the changing trend of the positions of the hands and feet.

3. The method for detecting hand and foot postures according to claim 1, characterized in that: The hand and foot posture estimation model includes a CNN backbone network, a hand posture estimation module and a foot posture estimation module, and the hand posture estimation module and the foot posture estimation module share the CNN backbone network.

4. The method for detecting hand and foot postures as claimed in claim 3, characterized in that: The hand posture estimation module and the foot posture estimation module respectively include a Transformer encoder layer and a head network, which are used to output heat maps of hand key points and foot key points respectively.

5. The method for detecting hand and foot postures as claimed in claim 4, characterized in that: In step 3, the training process of the hand and foot posture estimation model includes: Construct hand key point dataset and foot key point dataset; Using the samples in the hand key point data set and their corresponding labels to train the hand and foot posture estimation model, and obtain the network parameters of the CNN backbone network and the hand posture estimation module; Freeze the network parameters of the CNN backbone network, use the samples in the foot key point data set and their corresponding labels to train the foot posture estimation module, and obtain the network parameters of the Transformer encoder layer and the head network in the foot posture estimation module.

6. The method for detecting hand and foot postures as claimed in claim 4, characterized in that: The Transformer encoder layer includes a multi-head attention layer and a feed-forward network layer.

7. The method for detecting hand and foot postures as claimed in claim 4, characterized in that: The head network adopts a single layer of inverse convolution.

8. The method for detecting hand and foot postures as claimed in claim 1, characterized in that: In step 5, the hand and foot postures are 2.5D coordinates of key points of the hands or feet, including 2D coordinates of all key points and 3D coordinates of visible key points.

9. A DR-assisted positioning method for hands and feet, characterized in that: include: The hand and foot postures are obtained based on the method according to any one of claims 1 to 8, and exposure parameters are estimated according to the positioning requirements of the shooting position.

10. The DR-assisted positioning method for hands and feet according to claim 9, characterized in that: The exposure parameters include at least a region of interest and an exposure area for exposure control.

Images