A method for accurately locating acupoints on the human hand based on target and key point detection model

Abstract

The present invention provides a method for accurately locating acupoints on a human hand based on target detection and key point detection models. The acquired hand image is input into a palm and back of hand classification model 1 to determine whether the image is the palm or the back of the hand. The palm image is further input into a palm acupoint detection model 2 to predict the coordinate information of the acupoints; the back of the hand image is input into a back of the hand acupoint detection model 3 for prediction to predict the coordinate information of the acupoints. Finally, the system maps all the predicted acupoint coordinate information to the corresponding original human hand image and displays it visually. The present invention utilizes a deep learning model to improve the accuracy and efficiency of acupoint positioning, reduces the subjective error and experience dependence in traditional methods, and can adapt to the hand characteristics and diverse hand postures of different individuals by combining a variety of image processing technologies and machine learning algorithms, thereby providing an intelligent and standardized technical means for TCM acupoint treatment.

Description

Technical Field

[0001] The present invention belongs to the technical field of human hand acupoint positioning, and specifically relates to a human hand acupoint accurate positioning method based on a target and key point detection model. Background Art

[0002] By stimulating different areas or acupoints on the hands through massage, acupuncture and other methods, diseases of the corresponding organs, organs and various body systems can be improved or even treated. In recent years, the rapid development of deep neural network technology has provided new opportunities for the automatic identification and positioning of TCM acupoints. Traditional Chinese medicine is based mainly on body surface landmarks, combined with the "bone degree and inch" method, using this "inch" to measure distance, and locating acupoints through touch, press, and pinch. "Cun" needs to be based on the patient's own body shape. The physiological differences between individuals such as men and women, old and young, tall and short, fat and thin will lead to great differences in the size of "inch" and the location of acupuncture. Therefore, the accuracy of traditional hand acupoint positioning depends on the doctor's experience and skills, and there are also problems such as strong subjectivity and lack of accuracy.

[0003] The existing human hand acupoint localization method divides the localization work into two parts: "Using deep learning training technology to locate 21 key points on the human hand" and "Based on the 21 key points, combined with the traditional Chinese medicine 'inch' theory to complete the acupoint localization". However, this method has many problems: first, the error of key point localization will accumulate in the process of acupoint localization; second, the localization of each acupoint requires a separate formula designed based on experience, which not only increases the complexity and cost of development, but also reduces the flexibility and generalization ability of the system; finally, there is currently no public high-quality human hand acupoint dataset, which limits the effectiveness and accuracy of model training and makes it difficult to maintain high accuracy under diverse hand postures.

[0004] Therefore, developing an efficient, accurate and adaptable acupoint location method for various hand postures has become the key to improving the intelligent level of TCM acupoint treatment. We proposed a method for accurate location of human hand acupoints based on target and key point detection model to solve the problems existing in the existing technology, so that it can quickly and accurately locate human hand acupoints through computers. Summary of the invention

[0005] The purpose of the present invention is to provide a method for accurately locating acupoints on the human hand based on a target and key point detection model, so as to solve the problem that the existing technology mentioned in the above background technology not only has low work efficiency but also wastes a lot of manpower.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for accurately locating acupoints on a human hand based on a target and key point detection model comprises the following steps:

[0008] S1. Collect hand video data, extract frames from the video to obtain palm and back image data, and annotate them to obtain palm and back dataset 1;

[0009] S2. Use dataset 1 for model training and verification to obtain palm and back classification model 1;

[0010] S3, marking acupoint information on the palm and back of the hand, then collecting the human hand video after acupoint marking, extracting frames from the video to obtain the human hand and acupoint marking information, and obtaining palm acupoint dataset 2 and back acupoint dataset 3 respectively;

[0011] S4, using palm acupoint dataset 2 and back of hand acupoint dataset 3 for model training and verification, to obtain palm acupoint detection model 2 and back of hand acupoint detection model 3 respectively;

[0012] S5. Obtain a hand image for acupoint location, input it into model 1 for prediction to determine the palm or back of the hand, and select model 2 or model 3 for prediction based on the judgment result to obtain acupoint coordinate information.

[0013] Preferably, the palm and back of hand classification model training comprises the following steps:

[0014] Step 1: Collect hand videos of volunteers of different ages and genders, and then extract and filter the videos;

[0015] Step 2: Label the palm and back of the hand on the filtered images to obtain the palm and back of the hand dataset 1;

[0016] Step 3: Divide the dataset 1 into training set and validation set in proportion;

[0017] Step 4: Use the training set and validation set divided by data set 1 to train and validate the YOLOv8 model to obtain palm and back classification model 1.

[0018] Preferably, the palm acupoint detection model 2 training comprises the following steps:

[0019] Step 1: The Chinese medicine practitioner used a 0.5 mm black marker pen to mark the acupuncture points on the hands of volunteers of different ages and genders;

[0020] Step 2: Collect the hand-held video after the acupoints are marked, and then extract and filter the video;

[0021] Step 3: Label the hands and acupoints on the screened images to obtain the palm acupoint dataset 2;

[0022] Step 4: Divide the palm acupoint dataset 2 into a training set and a validation set in proportion;

[0023] Step 5: Use the training set and validation set divided by the palm acupoint dataset 2 to train and validate the RTMDet+ improved MobileNetV2 model to obtain the palm acupoint detection model 2.

[0024] Preferably, the third training of the hand back acupoint detection model comprises the following steps:

[0025] Step 1: The Chinese medicine practitioner used a 0.5 mm black marker pen to mark the acupuncture points on the hands of volunteers of different ages and genders;

[0026] Step 2: Collect the hand-held video after the acupoints are marked, and then extract and filter the video;

[0027] Step 3: Label the hands and acupoints on the screened images to obtain the hand back acupoints dataset 3;

[0028] Step 4: Divide the hand back acupoint dataset 3 into a training set and a validation set according to the proportion;

[0029] Step 5: Use the training set and validation set divided by the back of hand acupoint dataset 3 to train and validate the RTMDet+ improved MobileNetV2 model to obtain the back of hand acupoint detection model 3.

[0030] Preferably, the acquisition of acupoint coordinate information includes the following steps:

[0031] Step 1: Use the camera to obtain a human hand image and input it into model 1 for prediction;

[0032] Step 2: If the prediction is a palm image, the image is sent to model 2 for prediction; if the prediction is a back of the hand image, the image is sent to model 3 for prediction;

[0033] Step 3: Obtain the predicted acupoint coordinate information and visualize it on the hand image.

[0034] Preferably, during the video collection process, volunteers need to continuously open and close and turn their hands to collect as many postures and angles as possible.

[0035] Preferably, when dividing the training set and the validation set into the data set one, the data set two and the data set three, the division is performed in a ratio of 8:2.

[0036] Preferably, during the process of collecting the hand video data and collecting the human hand video data after acupoint marking, the main background of the video data collection is white to ensure sufficient lighting to obtain cleansed video data information.

[0037] Technical effects and advantages of the present invention: Compared with the prior art, the present invention proposes a method for accurately locating acupoints on the human hand based on a target and key point detection model, which has the following advantages:

[0038] The present invention sends the acquired image into the palm and back classification model one, sends the image classified as the palm into the palm acupoint detection model two, and predicts the acupoint coordinate information; sends the image classified as the back of the hand into the back of the hand acupoint detection model three, and predicts the acupoint information; finally, the predicted acupoint coordinate information is mapped to the human hand image and visualized. The method can quickly and accurately realize the positioning of human hand acupoints through a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of the method for accurately locating acupuncture points on the human hand based on the target and key point detection model of the present invention;

[0040] Figure 2 This is a schematic diagram of the prediction results of Model 1 of the present invention;

[0041] Figure 3 This is a visualization diagram of the prediction results of model 2 of the present invention;

[0042] Figure 4 This is a visualization diagram of the prediction results of model three of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] See also Figure 1-Figure 4 The present invention discloses a method for accurately locating acupoints on a human hand based on a target and key point detection model, comprising the following steps:

[0045] S1. Collect hand video data, extract frames from the video to obtain palm and back image data, and annotate them to obtain palm and back dataset 1;

[0046] S2. Use dataset 1 for model training and verification to obtain palm and back classification model 1;

[0047] Exemplarily, the palm and back of hand classification model training includes the following steps:

[0048] Step 1: Collect hand videos of volunteers of different ages and genders, and then extract and filter the videos;

[0049] Step 2: Label the palm and back of the hand on the filtered images to obtain the palm and back of the hand dataset 1;

[0050] Step 3: Divide the dataset 1 into training set and validation set in proportion;

[0051] Step 4: Use the training set and validation set divided by data set 1 to train and validate the YOLOv8 model to obtain palm and back classification model 1.

[0052] S3, marking acupoint information on the palm and back of the hand, then collecting the human hand video after acupoint marking, extracting frames from the video to obtain the human hand and acupoint marking information, and obtaining palm acupoint dataset 2 and back acupoint dataset 3 respectively;

[0053] S4, using palm acupoint dataset 2 and back of hand acupoint dataset 3 for model training and verification, to obtain palm acupoint detection model 2 and back of hand acupoint detection model 3 respectively;

[0054] Exemplarily, the palm acupoint detection model 2 training includes the following steps:

[0055] Step 1: The Chinese medicine practitioner used a 0.5 mm black marker pen to mark the acupuncture points on the hands of volunteers of different ages and genders;

[0056] Step 2: Collect the hand-held video after the acupoints are marked, and then extract and filter the video;

[0057] Step 3: Label the hands and acupoints on the screened images to obtain the palm acupoint dataset 2;

[0058] Step 4: Divide the palm acupoint dataset 2 into a training set and a validation set in proportion;

[0059] Step 5: Use the training set and validation set divided by the palm acupoint dataset 2 to train and validate the RTMDet+ improved MobileNetV2 model to obtain the palm acupoint detection model 2.

[0060] Exemplarily, the third training of the hand back acupoint detection model includes the following steps:

[0061] Step 1: The Chinese medicine practitioner used a 0.5 mm black marker pen to mark the acupuncture points on the hands of volunteers of different ages and genders;

[0062] Step 2: Collect the hand-held video after the acupoints are marked, and then extract and filter the video;

[0063] Step 3: Label the hands and acupoints on the screened images to obtain the hand back acupoints dataset 3;

[0064] Step 4: Divide the hand back acupoint dataset 3 into a training set and a validation set according to the proportion;

[0065] Step 5: Use the training set and validation set divided by the back of hand acupoint dataset 3 to train and validate the RTMDet+ improved MobileNetV2 model to obtain the back of hand acupoint detection model 3.

[0066] S5. Obtain a hand image for acupoint location, input it into model 1 for prediction to determine the palm or back of the hand, and select model 2 or model 3 for prediction based on the judgment result to obtain acupoint coordinate information.

[0067] Exemplarily, the acquisition of acupoint coordinate information includes the following steps:

[0068] Step 1: Use the camera to obtain a human hand image and input it into model 1 for prediction;

[0069] Step 2: If the prediction is a palm image, the image is sent to model 2 for prediction; if the prediction is a back of the hand image, the image is sent to model 3 for prediction;

[0070] Step 3: Obtain the predicted acupoint coordinate information and visualize it on the hand image.

[0071] It is worth mentioning that during the video collection process, volunteers need to constantly open and close and turn their hands to capture as many postures and angles as possible.

[0072] Furthermore, when the data set 1, data set 2 and data set 3 are divided into training sets and validation sets, they are divided according to the ratio of 8:2.

[0073] Furthermore, during the process of collecting the hand video data and collecting the human hand video data after acupoint marking, the main background of the video data collection is white to ensure sufficient lighting to obtain clean video data information.

[0074] The above is the general process of the method for accurately locating acupoints on the human hand based on the target and key point detection model in this embodiment. The specific implementation method also includes the following contents:

[0075] Volunteers of different ages and genders were recruited, and then videos of human hands were collected under conditions of sufficient lighting and a white background. When collecting videos, volunteers were required to open and close their hands and flip their hands to collect as many postures and angles as possible. The collected videos were framed and screened, and then the labeling software labelme was used to annotate the palm and back of the hand, where the palm label was represented as "in" and the back of the hand label was represented as "out", and Dataset 1 was obtained.

[0076] The dataset 1 is divided into a training set and a validation set in a ratio of 8:2. The training set and validation set from the dataset 1 are used to train and validate the YOLOv8 model, and the palm and back classification model 1 is obtained.

[0077] The specific parameters for training the YOLOv8 model are as follows:

[0078] The training weights (weights) of the YOLOv8 model are selected from the official (YOLOv8n), the number of iterations (epochs) is set to 300, the batch size batch_size is set to 8, the model file save interval is set to 10, the image input size is 640*640, and the Stochastic Gradient Descent (SGD) optimizer is used to reduce the possibility of the model falling into the local optimum. The initial learning rate of the network is set to 0.01, the momentum factor is set to 0.937, and the Mosaic technology is used for data enhancement.

[0079] Traditional Chinese medicine uses a 0.5mm black marker pen to mark acupoints on the hands of volunteers of different ages and genders, paying attention to the uniformity and clarity of the marking points. Then collect videos of the volunteers' hands. When collecting videos, volunteers need to constantly open and close and flip their hands to collect as many postures and angles as possible. The collected videos are framed and screened, and then the acupoints are annotated using the annotation software labelme. For the palm pictures, the annotated acupoints and their labels are: Yuji "yuJi", Laogong "laoGong", Shaofu "shaoFu", the four seams on the index finger "sfL", the four seams on the middle finger "sfM", the four seams on the ring finger "sfR", and the four seams on the little finger "sfP", and the data set 2 is obtained;

[0080] The data set 2 is divided into a training set and a validation set in a ratio of 8:2. The training set and validation set from the data set 2 are used to train and validate the RTMDet+ improved MobileNetV2 model to obtain the palm acupoint detection model 2.

[0081] For the back of the hand image, the acupoints and their labels are marked as: Shao Chong "shaoChong", Xiao Gu Kong "xiaoGuKong", Zhong Kui "zhongKui", Shang Yang "shangYang", He Gu "heGu", and the data set 3 is obtained;

[0082] The data set 3 is divided into a training set and a validation set in a ratio of 8:2. The training set and validation set from the data set 3 are used to train and validate the RTMDet+ improved MobileNetV2 model to obtain the back of hand acupoint detection model 3.

[0083] The specific parameters for training the RTMDet+ improved MobileNetV2 model are as follows:

[0084] The training weights of the RTMDet+ improved MobileNetV2 model are set to default, the number of epochs is set to 300, and the image input size is 640*640; the number of epochs is set to 300, the batch size batch_size is set to 8, the model file save interval is set to 10, the image input size is 640*640, and the Stochastic Gradient Descent (SGD) optimizer is used to reduce the possibility of the model falling into the local optimum. The initial learning rate of the network is set to 0.01, the momentum factor is set to 0.937, and the data enhancement uses Mosaic technology.

[0085] The specific parameters of the improved MobileNetV2 model are as follows:

[0086] Input:256x 256x 3

[0087] 1.Conv2d:3x3,32filters,stride 2,padding 1

[0088] 2. BatchNorm2d

[0089] 3. ReLU

[0090] 4.Bottleneck:t=1,c=16,n=1,s=1

[0091] 5.Bottleneck:t=6,c=24,n=2,s=2

[0092] 6.Bottleneck:t=6,c=32,n=3,s=2

[0093] 7.Bottleneck:t=6,c=64,n=4,s=2

[0094] 8.Bottleneck:t=6,c=96,n=3,s=1

[0095] 9.Bottleneck:t=6,c=160,n=3,s=2

[0096] 10.Bottleneck:t=6,c=320,n=1,s=1

[0097] 11.Conv2d:1x1,1280filters,stride 1

[0098] 12. BatchNorm2d

[0099] 13. ReLU

[0100] 14.AdaptiveAvgPool2d:1x1

[0101] 15. Flatten

[0102] 16.Linear

[0103] Output: list of coordinate points.

[0104] In addition, as shown in the following table:

[0105] Table 1 Video acquisition requirements

[0106]

[0107] Table 2 Optimal values ​​of key point model evaluation indicators

[0108]

[0109] mAP (mean Average Precision) is the mean of the average precision, which is used to evaluate the average precision of the model in multiple categories, providing a more comprehensive evaluation of the overall performance of the model. PCK (Percentage of Correct Keypoint) is used for human posture estimation tasks to measure the accuracy of key point prediction. AUC (Area Under the Curve) is the area under the ROC curve and the coordinate axis, with a value range of 0.5 to 1.0. The closer the value is to 1.0, the higher the authenticity of the detection method. It is usually used to evaluate the performance of binary classification models. NME (Normalized Mean Error) is an indicator commonly used in facial key point location tasks. It is used to measure the normalized mean error between the predicted key points and the actual key points, thereby providing an evaluation of the key point location accuracy.

[0110] After the MobileNetV2 model was improved, indicators such as mAP, PCK, and AUC were improved, and the NME indicator indicating the error decreased. This shows that the improved MobileNetV2 model can better complete the task of locating acupuncture points on the human hand.

[0111] Among them, Input indicates that the size of the input data is 256x256 pixels, with 3 channels (RGB color channels); Conv2d represents a convolution layer, which extracts features on the input data by sliding the convolution kernel; BatchNorm2d is a batch normalization layer used to accelerate the training process of the neural network and improve the convergence speed; ReLU activation function is used to increase the nonlinear characteristics of the model, which helps the model learn complex data patterns; Bottleneck module is usually composed of depthwise separable convolutional layers, which is used to maintain the effect of the model while reducing the computational cost; AdaptiveAvgPool2d is an adaptive average pooling operation used to reduce the spatial size of the feature map, thereby reducing the computational burden of the model and enhancing the model's robustness to position changes; Flatten is used to flatten the multi-dimensional convolution layer output into a one-dimensional vector for input into the fully connected layer for further processing and classification; Liner is a fully connected layer used to output predictions; Output The final output is a list of coordinate points. After all levels of processing and feature extraction, the model outputs a list of coordinate points to represent the predicted coordinate positions of the acupoints.

[0112] Get a hand image and use the trained model one for classification prediction. If the prediction result of model one is "in" - palm, the image is sent to the trained palm acupoint detection model two for acupoint coordinate information prediction; if the prediction result of model one is "out" - back of hand, the image is sent to the trained back of hand acupoint detection model three for acupoint coordinate information prediction; finally, use opencv to predict the obtained acupoint coordinate information and visualize it on the hand image.

[0113] It should be noted that the present invention is not limited to this, and corresponding adjustments can be made according to actual conditions. For example, the number of acupoint marks on the palm and back of the hand can be increased or decreased, as long as the technical functions are achieved according to the above-mentioned technical methods.

[0114] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for accurately locating acupuncture points on a human hand based on a target and key point detection model, characterized in that: The following steps are involved: S1. Collect hand video data, extract frames from the video to obtain palm and back image data, and annotate them to obtain palm and back dataset 1; S2. Use dataset 1 for model training and verification to obtain palm and back classification model 1; S3, marking acupoint information on the palm and back of the hand, then collecting the human hand video after acupoint marking, extracting frames from the video to obtain the human hand and acupoint marking information, and obtaining palm acupoint dataset 2 and back acupoint dataset 3 respectively; S4, using palm acupoint dataset 2 and back of hand acupoint dataset 3 for model training and verification, to obtain palm acupoint detection model 2 and back of hand acupoint detection model 3 respectively; S5. Obtain a hand image for acupoint location, input it into model 1 for prediction to determine the palm or back of the hand, and select model 2 or model 3 for prediction based on the judgment result to obtain acupoint coordinate information.

2. According to claim 1, a method for accurately locating acupoints on a human hand based on a target and key point detection model is characterized in that: The palm and back of hand classification model training includes the following steps: Step 1: Collect hand videos of volunteers of different ages and genders, and then extract and filter the videos; Step 2: Label the palm and back of the hand on the filtered images to obtain the palm and back of the hand dataset 1; Step 3: Divide the dataset 1 into training set and validation set in proportion; Step 4: Use the training set and validation set divided by data set 1 to train and validate the YOLOv8 model to obtain palm and back classification model 1.

3. The method for accurately locating acupoints on a human hand based on a target and key point detection model according to claim 2, characterized in that: The palm acupoint detection model 2 training comprises the following steps: Step 1: The Chinese medicine practitioner used a 0.5 mm black marker pen to mark the acupuncture points on the hands of volunteers of different ages and genders; Step 2: Collect the hand-held video after the acupoints are marked, and then extract and filter the video; Step 3: Label the hands and acupoints on the screened images to obtain the palm acupoint dataset 2; Step 4: Divide the palm acupoint dataset 2 into a training set and a validation set in proportion; Step 5: Use the training set and validation set divided by the palm acupoint dataset 2 to train and validate the RTMDet+ improved MobileNetV2 model to obtain the palm acupoint detection model 2.

4. The method for accurately locating acupuncture points on a human hand based on a target and key point detection model according to claim 3, characterized in that: The third training of the hand back acupoint detection model includes the following steps: Step 1: The Chinese medicine practitioner used a 0.5 mm black marker pen to mark the acupuncture points on the hands of volunteers of different ages and genders; Step 2: Collect the hand-held video after the acupoints are marked, and then extract and filter the video; Step 3: Label the hands and acupoints on the screened images to obtain the hand back acupoints dataset 3; Step 4: Divide the hand back acupoint dataset 3 into a training set and a validation set according to the proportion; Step 5: Use the training set and validation set divided by the back of hand acupoint dataset 3 to train and validate the RTMDet+ improved MobileNetV2 model to obtain the back of hand acupoint detection model 3.

5. The method for accurately locating acupuncture points on a human hand based on a target and key point detection model according to claim 4, characterized in that: The acupoint coordinate information acquisition comprises the following steps: Step 1: Use the camera to obtain a human hand image and input it into model 1 for prediction; Step 2: If the prediction is a palm image, the image is sent to model 2 for prediction; if the prediction is a back of the hand image, the image is sent to model 3 for prediction; Step 3: Obtain the predicted acupoint coordinate information and visualize it on the hand image.

6. The method for accurately locating acupuncture points on a human hand based on a target and key point detection model according to claim 5, characterized in that: During the video collection process, volunteers need to constantly open and close and turn their hands to capture as many postures and angles as possible.

7. The method for accurately locating acupuncture points on a human hand based on a target and key point detection model according to claim 6, characterized in that: When the data sets 1, 2 and 3 are divided into training sets and validation sets, they are divided according to the ratio of 8:

2.

8. The method for accurately locating acupuncture points on a human hand based on a target and key point detection model according to claim 7, characterized in that: During the process of collecting the hand video data and collecting the human hand video data after acupoint marking, the main background of the video data collection is white to ensure sufficient lighting to obtain cleansed video data information.

Images