A method for precise acupoint localization based on cascaded deep neural networks

Abstract

The present invention discloses an acupoint precise positioning method based on a cascaded deep neural network. Existing deep learning technologies for small target detection have problems such as low recognition accuracy, slow recognition speed, and stroboscopic recognition. The steps of the present invention are as follows: using a cascaded neural network that combines hand position detection and hand key point detection for the obtained hand image, and outputting the hand region and the information of 21 hand key points; finally, according to the geometric relationship between the 21 hand key points and "cun", the acupoint coordinates are calculated and mapped into the hand region. The present invention improves both the accuracy and robustness of detecting hand acupoints on the premise of ensuring the real-time performance and detection speed requirements of the detection process.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, specifically relating to a method for precise acupoint localization using a cascaded deep neural network. Background Technology

[0002] In traditional Chinese medicine, acupoints on the hands are an important component. Stimulating these points through acupuncture, moxibustion, and massage can promote blood circulation, improve complexion, and treat diseases. However, due to the dense distribution of acupoints on the hands and the close proximity of adjacent points, inexperienced individuals or those without specialized equipment often struggle to accurately locate them, reducing the effectiveness of massage. With the development of artificial intelligence, using deep learning technology for medical assistance has become a popular research direction in recent years, especially in the detection and classification of medical images. However, for detecting small targets like hand acupoints, problems such as low recognition accuracy, slow recognition speed, and flickering may occur. Summary of the Invention

[0003] To address the aforementioned problems, this invention provides a method for precise acupoint localization using a cascaded deep neural network. The aim is to improve the accuracy and robustness of hand acupoint detection by building a deep learning algorithm framework while ensuring the real-time performance and speed requirements of the detection process.

[0004] This invention discloses a method for precise acupoint localization based on cascaded deep neural networks, comprising the following steps:

[0005] S1: Collect hand data under different lighting conditions and skin tones to obtain Class I samples;

[0006] S2: Collect hand data under different gestures to obtain Class II samples;

[0007] S3: Perform data preprocessing on Class I samples and divide the training set and validation set proportionally;

[0008] S4: Perform data preprocessing on Class II samples and divide the training set and validation set proportionally;

[0009] S5: Use object detection algorithms to construct a Class I model, and use the training set and validation set obtained based on Class I samples to train and validate the Class I model to obtain an optimized Class I model;

[0010] S6: Use the key point detection algorithm to construct a Class II model, and use the training set and validation set obtained based on Class II samples to train and validate the Class II model to obtain the Class II optimized model;

[0011] S7: Acquire real-time hand images via camera, locate the hand region using a Type I optimization model, and then input the hand region into a Type II optimization model to obtain information on 21 key points of the hand; determine the palm or back of the hand based on the information of the 21 key points of the hand.

[0012] S8: Based on the information of 21 key points on the hand, the acupoints on the hand are located using the middle finger measurement method, and then the acupoint information is mapped to the hand area.

[0013] Preferably, in steps S1 and S2, the Ha-GRID open-source hand dataset and the 11k Hands open-source palm dataset are used as initial samples. These datasets contain hand data with different gestures, different lighting conditions, and different skin colors, but do not contain hand data with physiological defects.

[0014] Preferably, the data preprocessing for Class I samples is as follows: use annotation software to label the positions containing hands in the Class I samples, where the left hand is labeled as "L-Hand" and the right hand is labeled as "R-Hand".

[0015] Preferably, the data preprocessing for Class II samples is as follows: Using annotation software, the coordinates of 21 key points on the hand in the Class II samples are annotated. The 21 key points and their labels are set as follows: wrist "0.WRIST", first wrist-palmar "1.THUMB_CMC", thumb palm-finger "2.THUMB_MCP", thumb interphalangeal joint "3.THUMB_IP", thumb fingertip "4.THUMB_TIP", index finger palm-finger "5.INDEX_FINGER_MCP", index finger proximal interphalangeal joint "6.INDEX_FINGER_PIP", index finger distal interphalangeal joint "7.INDEX_FINGER_DIP", index fingertip "8.INDEX_FINGER_TIP", middle finger palm-finger "9.MIDDLE_FINGER_MCP", middle finger... The following are the pins for the fingertip: proximal interphalangeal joint "10.MIDDLE_FINGER_PIP", distal interphalangeal joint of middle finger "11.MIDDLE_FINGER_DIP", fingertip of middle finger "12.MIDDLE_FINGER_TIP", palmar interphalangeal joint of ring finger "13.RING_FINGER_MCP", proximal interphalangeal joint of ring finger "14.RING_FINGER_PIP", distal interphalangeal joint of ring finger "15.RING_FINGER_DIP", fingertip of ring finger "16.RING_FINGER_TIP", palmar interphalangeal joint of little finger "17.PINKY_MCP", proximal interphalangeal joint of little finger "18.PINKY_PIP", distal interphalangeal joint of little finger "19.PINKY_DIP", and fingertip of little finger "20.PINKY_TIP".

[0016] Preferably, after data preprocessing in steps S3 and S4, a format conversion is performed, specifically converting the preprocessed XML format to YOLO format.

[0017] Preferably, the training set and validation set are divided in an 8:2 ratio.

[0018] Preferably, in step S5, an improved YOLOv5 object detection algorithm is used to construct a Class I model. The training weights are selected from the official weights (yolov5s6), and the number of epochs is 300. The improved YOLOv5 object detection algorithm adds a 160x160 detection feature map to the original detection head framework of the YOLOv5 object detection algorithm for detecting targets larger than 4x4. An attention mechanism is added to the original backbone framework, and some of the original C3 modules are replaced with C3SE modules. The Anchors parameters are recalculated using the K-means clustering algorithm.

[0019] Preferably, in step S6, a heatmap regression method is used to construct a Class II model. The heatmap regression method uses HRNet as the backbone network, and the hyperparameters set are: 300 iterations, Adam optimizer, and ReLU activation function.

[0020] Preferably, in step S8, the "cun" in the middle finger measurement method is calculated using Euclidean distance:

[0021] X c =X 11 -X 10

[0022] Y c =Y 11 -Y 10

[0023]

[0024] Where X c X is the x-coordinate of the distal interphalangeal joint of the middle finger. 11 x-coordinate of the proximal interphalangeal joint of the middle finger 10 The distance between them, Y c Y is the ordinate of the distal interphalangeal joint of the middle finger. 11 The ordinate Y of the proximal interphalangeal joint of the middle finger 10 The distance between them, L c The distance is measured in inches.

[0025] The beneficial effects of this invention are as follows:

[0026] The present invention collects hand information in real time through a camera, cascades the Class I optimization model and the Class II optimization model using a deep learning framework (PyTorch), and combines geometric principles to accurately locate hand acupoints and map them to the hand positions. Among them, since there will be interference information when the camera collects hand information in real time, if the key point algorithm is directly used for hand key point recognition, when the interference information is close to the hand data sample, misdetection and key point drift will occur. Therefore, the present invention cascades the deep learning algorithm. First, the hand position is located and the left and right hands are distinguished through the classification model (Class I optimization model), and then the regression model (Class II optimization model) is used to recognize the hand key points to obtain 21 hand key points. For the distinction between the palm and the back of the hand, it is determined by using the left and right hand information determined by the classification model and the geometric relationship information of the 21 hand key points. The accurate positioning of hand acupoints adopts the middle finger same body inch method in traditional Chinese medicine theory, and the acupoint information of the hand is determined by using the relationship between "inch" and hand key points. On the premise of ensuring the real-time performance and detection speed requirements of the detection process, the present invention improves both the accuracy and robustness of detecting hand acupoints. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flowchart of the present invention.

[0028] Figure 2 is a schematic diagram of the hand key point label parameters of the present invention.

[0029] Figure 3 is a schematic diagram of the hand area located by the Class I optimization model of the present invention.

[0030] Figure 4 is the heat map of each hand key point obtained by the present invention using the heat map regression method.

[0031] Figure 5 2]is a schematic diagram of the combined result of the dual models of the present invention.

[0032] Figure 6 is a schematic diagram of the acupoint positioning result of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] The following combines examples and drawings to further elaborate on the present invention, but the implementation manners of the present invention are not limited thereto.

[0034] The present invention aims to provide a hand acupoint detection method with high accuracy and high scalability, enabling the operation of massaging acupoints to be completed by oneself under the conditions of "no experience" and "no professional equipment".

[0035] Example 1

[0036] As Figure 1As shown in the figure, a method for accurate acupoint positioning based on a cascaded deep neural network uses a cascaded neural network that combines hand position detection and hand key point detection for the acquired hand images, and outputs the hand region and information of 21 hand key points. Finally, according to the geometric relationship between the 21 hand key points and "cun", the acupoint coordinates are calculated and mapped into the hand region. The specific steps are as follows:

[0037] S01. Collect hand data with different gestures, different lighting levels, and different skin colors in the Ha-GRID open-source hand dataset and the 11k Hands open-source palm dataset to obtain type-I samples and type-II samples. Among them, type-I samples contain hand data under different lighting levels and different skin colors, and type-II samples contain hand data under different gestures. Neither type-I samples nor type-II samples contain hand data of the physiological defect type. Then, perform data preprocessing on type-I samples and type-II samples respectively, and perform format conversion on the data samples obtained after data preprocessing. Finally, divide the type-I samples after data preprocessing and format conversion into a training set and a validation set according to a ratio of 8:2, and divide the type-II samples after data preprocessing and format conversion into a training set and a validation set according to a ratio of 8:2.

[0038] Specifically, the data preprocessing of type-I samples is: use the annotation software (LabelImg) to annotate the positions of the hands in type-I samples, where the left hand label is set as "L-Hand" and the right hand label is set as "R-Hand".

[0039] Specifically, as Figure 2As shown, the data preprocessing for Class II samples is as follows: The coordinates of 21 key points on the hand in the Class II samples were labeled using the labeling software (Labelme). The 21 key points and their labels are set as follows: wrist "0.WRIST", first wrist-palmar "1.THUMB_CMC", thumb palm-finger "2.THUMB_MCP", thumb interphalangeal joint "3.THUMB_IP", thumb fingertip "4.THUMB_TIP", index finger palm-finger "5.INDEX_FINGER_MCP", index finger proximal interphalangeal joint "6.INDEX_FINGER_PIP", index finger distal interphalangeal joint "7.INDEX_FINGER_DIP", index fingertip "8.INDEX_FINGER_TIP", middle finger palm-finger "9.MIDDLE_FINGER_MCP". P", middle finger proximal interphalangeal joint "10.MIDDLE_FINGER_PIP", middle finger distal interphalangeal joint "11.MIDDLE_FINGER_DIP", middle fingertip "12.MIDDLE_FINGER_TIP", ring finger palmar finger "13.RING_FINGER_MCP", ring finger proximal interphalangeal joint "14.RING_FINGER_PIP", ring finger distal interphalangeal joint "15.RING_FINGER_DIP", ring fingertip "16.RING_FINGER_TIP", little finger palmar finger "17.PINKY_MCP", little finger proximal interphalangeal joint "18.PINKY_PIP", little finger distal interphalangeal joint "19.PINKY_DIP", little fingertip "20.PINKY_TIP".

[0040] S02. An improved YOLOv5 object detection algorithm is adopted to construct a Class I model. The Class I model is trained and validated using the training set and validation set obtained based on Class I samples to obtain an optimized Class I model.

[0041] The improvements to the YOLOv5 object detection algorithm are as follows:

[0042] The YOLOv5 object detection algorithm uses the official weights (yolov5s6) for training, with 300 epochs. A 160x160 detection feature map is added to the original detection head framework for detecting objects larger than 4x4. An attention mechanism is added to the original backbone framework, and some C3 modules are replaced with C3SE modules. Anchor parameters are recalculated using the K-means clustering algorithm. Specific parameters are as follows:

[0043] Anchors:

[0044] [7,10,19,12,25,30]

[0045] [14.931,20.439,39.648,25.53,53.371,62.35]

[0046] [129.07,84.774,95.719,175.08,142.69,286.95]

[0047] [256.46,239.83,369.9,378.3,581.87,418.56]Backbone:

[0048] [[-1,1,Conv,[64,6,2,2]],

[0049] [-1,1,Conv,[128,3,2]],

[0050] [-1,3,C3SE,

[128] ],

[0051] [-1,1,Conv,[256,3,2]],

[0052] [-1,9,C3SE,

[256] ],

[0053] [-1,1,Conv,[512,3,2]],

[0054] [-1,9,C3SE,

[512] ],

[0055] [-1,1,Conv,[1024,3,2]],

[0056] [-1,1,SPPF,[1024,5]],

[0057] [-1,3,C3,[1024,False]], ]

[0059] Head:

[0060] [[-1,1,Conv,[512,1,1]],

[0061] [-1,1,nn.Upsample,[None,2,’nearest’]],[[-1,6],1,Concat,[1]],

[0062] [-1,3,C3,[512,False]],

[0063] [-1,1,Conv,[512,1,1]],

[0064] [-1,1,nn.Upsample,[None,2,’nearest’]],[[-1,4],1,Concat,[1]],

[0065] [-1,3,C3,[512,False]],

[0066] [-1,1,Conv,[256,1,1]],

[0067] [-1,1,nn.Upsample,[None,2,’nearest’]],[[-1,2],1,Concat,[1]],

[0068] [-1,3,C3,[256,False]],

[0069] [-1,1,Conv,[256,3,2]],

[0070] [[-1,18],1,Concat,[1]],

[0071] [-1,3,C3,[256,False]],

[0072] [-1,1,Conv,[256,3,2]],

[0073] [[-1,14],1,Concat,[1]],

[0074] [-1,3,C3,[512,False]],

[0075] [-1,1,Conv,[512,3,2]],

[0076] [[-1,10],1,Concat,[1]],

[0077] [-1,3,C3,[1024,False]],

[0078] [[21,24,27,30],1,Detect,[nc,Anchors]], ]

[0080] Conv is the convolutional module, including normalized forward propagation and ordinary forward propagation; C3SE is the attention mechanism module; SPPF is the spatial pyramid pooling module, which can convert feature maps of arbitrary size into fixed feature vectors; C3 is a module composed of ConvBNSiLU, BottleNeck and Concat modules. The ConvBNSiLU module consists of Conv convolution, BatchNormalization and SILU activation functions; BottleNeck is a residual module; Concat is feature fusion, which mainly superimposes features with the same channels; nn.Upsample is upsampling; False indicates that the number of channels remains unchanged, for example, [256,False] means that the output channels are 256; nc is the classification category, since left and right hands are detected, nc=2; Detect is the detection layer, which applies convolution to the detection layer.

[0081] S03. Use heatmap regression to construct a Class II model. Use the training set and validation set obtained from Class II samples to train and validate the Class II model, and obtain the optimized Class II model.

[0082] Specifically, the heatmap regression method uses HRNet as the backbone network, with the following hyperparameters: 300 iterations, Adam optimizer, and ReLU activation function. During keypoint calculation, the ArgMax function is used to activate the pixels corresponding to each keypoint in the input Class I optimization model, such as... Figure 4 As shown, a fully convolutional neural network is then used to obtain the output. The transformation module is connected after the fully convolutional feature map to convert the spatial Gaussian heatmap into a set of keypoint coordinates. The final output, which regresses the position of each keypoint, is obtained by taking the average of the set. Figure 5 As shown.

[0083] S04. Acquire real-time hand image, and use a type I optimization model to locate the hand region (i.e., the region of interest, ROI). The result is as follows. Figure 3 As shown; then the hand region is input into the Class II optimization model to obtain information on 21 key points of the hand.

[0084] S05. Based on the information from the 21 key points of the hand obtained in S04, determine whether it is the palm or the back of the hand.

[0085] Specifically, the information for the 21 key points of the hand is set as follows: (0, X0, Y0), (1, X1, Y1), ..., (20, X... 20 Y 20) Since the coordinate changes of the thumb fingertip on the palm or back of the hand are relatively obvious, and the coordinate changes of the little finger fingertip are also relatively obvious, but a reference object is needed for the same-direction changes. Therefore, by comparing the thumb fingertip coordinate with the thumb interphalangeal joint coordinate, and the little finger fingertip coordinate with the little finger distal interphalangeal joint coordinate, to determine whether the hand area is the palm or the back of the hand. Specifically:

[0086] When the hand area is the right hand, if the following inequality is satisfied, it is the back of the hand, otherwise it is the palm:

[0087] X4 - X3 < 0, X 20 -X 19 > 0

[0088] Where X4 is the abscissa of the thumb fingertip, X3 is the abscissa of the thumb interphalangeal joint, X 20 is the abscissa of the little finger fingertip, X 19 is the abscissa of the little finger distal interphalangeal joint;

[0089] When the hand area is the right hand, if the following inequality is satisfied, it is the back of the hand, otherwise it is the palm:

[0090] X4 - X3 > 0, X 20 -X 19 <0

[0091] S06. According to the information of the 21 key points of the hand obtained in S04, combined with the middle finger cun method to accurately locate the acupoints, and then map the acupoint information to the hand area.

[0092] Specifically, according to the information of the 21 key points of the hand, the distance between the middle finger distal interphalangeal joint and the middle finger proximal interphalangeal joint is determined as one "cun", and then the acupoint points are determined in combination with the relationship between each key point and the "cun". The effect is as Figure 6 shown.

[0093] Specifically, the "cun" is calculated using the Euclidean distance:

[0094] X c =X 11 -X 10

[0095] Y c =Y 11 -Y 10

[0096]

[0097] Where X c is the distance between the abscissa X 11 of the middle finger distal interphalangeal joint and the abscissa X 10 of the middle finger proximal interphalangeal joint, Y cY is the ordinate of the distal interphalangeal joint of the middle finger. 11 The ordinate Y of the proximal interphalangeal joint of the middle finger 10 The distance between them, L c The distance is measured in inches.

[0098] For example, the Shaoze acupoint is located on the back of the hand, on the ulnar side of the distal phalanx of the little finger, 0.1 cun above the corner of the nail root.

[0099] X shaoze =(X 20 +X 19 )×0.5±0.1×L c

[0100] Y shaoze =(Y 20 +Y 19 )×0.5+0.1×L c

[0101] Where X shaoze Let X be the x-coordinate of the Shaoze acupoint. 20 Let X be the x-coordinate of the little fingertip. 19 Y is the x-coordinate of the distal interphalangeal joint of the little finger; sha0ze Y is the vertical coordinate of the Shaoze acupoint. 20 Y is the ordinate of the little fingertip. 19 The coordinate of the distal interphalangeal joint of the little finger is ordinate; when the hand region is the right hand, the ± sign is +, otherwise -.

[0102] For example, the Zhongkui acupoint is located on the back of the hand, at the midpoint of the transverse crease of the proximal interphalangeal joint on the dorsal side of the middle finger.

[0103] X zhongkui =X 10

[0104] Y zhongkui =Y 10

[0105] Where X zhongkui Let X be the x-coordinate of the Zhongkui acupoint. 10 Y is the x-coordinate of the proximal interphalangeal joint of the middle finger; where Y zhongkui Y is the vertical coordinate of the Zhongkui acupoint. 10 The vertical coordinate is the ordinate of the proximal interphalangeal joint of the middle finger.

[0106] For example, the Shaochong acupoint is located on the back of the hand, on the radial side of the distal phalanx of the little finger, 0.1 cun above the corner of the nail root.

[0107] X shaochong =(X 20 +X 19 )×0.5±0.1×L c

[0108] Y shaochong =(Y 20 +Y 19 )×0.5+0.1×L c

[0109] Where X shaochong Y is the x-coordinate of the Shaochong acupoint. shaochong The vertical coordinate of the Shaochong acupoint is shown; when the hand area is the left hand, the ± sign is +, otherwise -.

[0110] For example, the Shangyang acupoint is located on the back of the hand, on the radial side of the distal phalanx of the index finger, 0.1 cun from the corner of the nail.

[0111] X shangyang = (X8 + X7) × 0.5 ± 0.1 × L c

[0112] Y shangyang = (Y8 + Y7) × 0.5 + 0.1 × L c

[0113] Where X shangyang X is the x-coordinate of the Shangyang acupoint, X8 is the x-coordinate of the fingertip of the index finger, and X7 is the x-coordinate of the distal interphalangeal joint of the index finger; Y shangyang Y1 is the ordinate of Shangyang acupoint, Y2 is the ordinate of the fingertip of the index finger, and Y3 is the ordinate of the distal interphalangeal joint of the index finger. When the hand area is the left hand, the ± sign is +, otherwise -.

[0114] For example, the Sanjian acupoint is located on the back of the hand, one inch in the depression on the radial side of the second metacarpophalangeal joint.

[0115] X sanjian =X5±Lc

[0116] Y sdnjian =Y5

[0117] Where X sanjian X is the x-coordinate of the Sanjian acupoint, and X5 is the x-coordinate of the palmar index finger; Y sanjian Y1 represents the vertical coordinate of the Sanjian acupoint, and Y5 represents the vertical coordinate of the index finger's palmar axis. When the hand area is the left hand, the ± sign is +, otherwise -.

[0118] For example, the Hegu acupoint is located on the back of the hand, at the midpoint of the radial side of the second metacarpal bone.

[0119] X hegu = (X5 + X0) × 0.5

[0120] Y hegu = (Y5 + Y0) × 0.5

[0121] Where Xhegu X is the x-coordinate of the Hegu acupoint, and X0 is the x-coordinate of the wrist; Y hegu Y is the vertical coordinate of the Hegu acupoint, and Y0 is the vertical coordinate of the wrist.

[0122] For example, the Yimen acupoint is located on the back of the hand, between the fourth and fifth fingers.

[0123] X yemen =(X 13 +X 17 )×0.5

[0124] Y yemen =(Y 13 +Y 17 )×0.5

[0125] Where X yemen Let X be the x-coordinate of the Yimen acupoint. 17 Let X be the x-coordinate of the little finger's palm. 13 Y is the x-coordinate of the palmar index of the ring finger; yemen Y is the ordinate of the Yimen acupoint. 17 Y is the ordinate of the palmar coordinate of the little finger. 13 The vertical coordinate is the palmar coordinate of the ring finger.

[0126] For example, the Lung Point is located in the palm of the hand, at the midpoint of the transverse crease of the distal interphalangeal joint of the ring finger on the palmar surface.

[0127] X feixue =X 15

[0128] Y feixue =Y 15

[0129] Where X feixue X represents the x-coordinate of the lung acupoint. 15 Y is the x-coordinate of the distal interphalangeal joint of the ring finger; where Y feixue Y is the vertical coordinate of the lung acupoint. 15 The vertical coordinate is the ordinate of the distal interphalangeal joint of the ring finger.

[0130] For example, the kidney point (kidney acupoint) is located in the palm of the hand, on the palm side, at the midpoint of the transverse crease of the distal interphalangeal joint of the little finger.

[0131] X shenxue =X 19

[0132] Y shenxue =Y 19

[0133] Where X shenxue Y is the x-coordinate of the kidney acupoint. shenxue The vertical coordinate is the kidney acupoint.

[0134] For example, the Liver Point is located in the palm of the hand, on the side of the hand, at the midpoint of the transverse crease of the proximal interphalangeal joint of the ring finger.

[0135] X ganxue =X 14

[0136] Y ganxue =Y 14

[0137] Where X ganxue X represents the x-coordinate of the liver acupoint. 14 Y is the x-coordinate of the proximal interphalangeal joint of the ring finger; ganxue Y is the vertical coordinate of the liver acupoint. 14 The vertical coordinate is the ordinate of the proximal interphalangeal joint of the ring finger.

[0138] For example, the Laogong acupoint is located in the palm of the hand, between the second and third metacarpal bones.

[0139] X laogong =[(X5+X0)×0.5+(X9+X0)×0.5]×0.5

[0140] Y laogong =[(Y5+Y0)×0.5+(Y9+Y0)×0.5]×0.5

[0141] Where X laogong X is the x-coordinate of the Laogong acupoint, and X9 is the x-coordinate of the palmar index finger of the middle finger; Y laogong Y1 represents the vertical coordinate of the Laogong acupoint, and Y2 represents the vertical coordinate of the palmar index finger of the middle finger.

[0142] In particular, the Shaofu acupoint is located in the palm of the hand, usually between the 4th and 5th metacarpal bones.

[0143] X shaofu =[(X 13 +X0)×0.5+(X 17 +X0)×0.5]×0.5

[0144] Y shaofu =[(Y 13 +Y0)×0.5+(Y 17 +Y0)×0.5]×0.5

[0145] Where X shaofu Let X be the x-coordinate of the Shaofu acupoint. 13 Let X be the x-coordinate of the palmar index of the ring finger. 17 Y is the x-coordinate of the little finger palm. shaofu Y is the vertical coordinate of the Shaofu acupoint. 13 Y is the ordinate of the palmar coordinate of the ring finger. 17 The vertical coordinate is the palmar coordinate of the little finger.

[0146] For the location of other acupoints, refer to the national standard GB12346-1990 for acupoint locations. The coordinates of other acupoints can be calculated according to the calculation rules for acupoints in similar locations.

[0147] Example 2

[0148] This invention is used for facial acupoint detection. Specifically, it involves: acquiring facial image data to form Class I and Class II samples, performing data preprocessing and format conversion, and dividing the data into training and validation sets; constructing a Class I model using an improved YOLOv5 object detection algorithm, and training and validating the Class I model using the training and validation sets obtained from the Class I samples to obtain an optimized Class I model; constructing a Class II model using heatmap regression, and training and validating the Class II model using the training and validation sets obtained from the Class II samples to obtain an optimized Class II model; using the improved YOLOv5 object detection algorithm to initially locate the face, obtaining the facial region, and inputting the facial region into the optimized Class II model to obtain information on 15 facial feature points, including the corners of the eyes, pen tips, corners of the mouth, and brow bones; calculating the corresponding acupoints on the face using the finger-inch positioning method in traditional Chinese medicine combined with facial key points, and mapping them to the facial region.

[0149] It should be noted that the present invention is not limited thereto, and can be adjusted accordingly based on the actual situation. For example, the parameter settings for Anchors can be obtained based on other clustering algorithms, and the improvements to the detection head can also be changed depending on the target object to be detected, as long as the technical function is achieved in accordance with the above technical methods.

[0150] The above descriptions of various embodiments of the present invention are exemplary and not limited to the disclosed embodiments. Any modifications or additions made within the scope of the principles of the present invention should be within the protection scope of the present invention.

Claims

1. A method for precise acupoint localization based on cascaded deep neural networks, characterized in that: Includes the following steps: S1: Collect hand data under different lighting conditions and different skin tones to obtain Class I samples; S2: Collect hand data under different gestures to obtain Class II samples; S3: Perform data preprocessing on Class I samples and divide the training set and validation set proportionally; S4: Perform data preprocessing on Class II samples and divide the training set and validation set proportionally; S5: Use object detection algorithms to construct a Class I model, and use the training set and validation set obtained based on Class I samples to train and validate the Class I model, thus obtaining an optimized Class I model; S6: Use the key point detection algorithm to construct a Class II model, and use the training set and validation set obtained based on Class II samples to train and validate the Class II model to obtain the Class II optimized model; S7: Acquire real-time hand images via camera, locate the hand region using a Type I optimization model, and then input the hand region into a Type II optimization model to obtain information on 21 key points of the hand; determine the palm or back of the hand based on the information of the 21 key points of the hand. S8: Based on the information of 21 key points on the hand, the acupoints on the hand are located by combining the middle finger measurement method, and then the acupoint information is mapped to the hand area. The data preprocessing for Class I samples is as follows: the positions containing hands in the Class I samples are labeled using annotation software, with the left hand labeled as "L-Hand" and the right hand labeled as "R-Hand". The data preprocessing for Class II samples was as follows: Labeling software was used to annotate the coordinates of 21 key points on the hand in the Class II samples. The 21 key points and their labels were set as follows: wrist "0.WRIST", first wrist-palmar "1.THUMB_CMC", thumb palmar-digital "2.THUMB_MCP", thumb interphalangeal joint "3.THUMB_IP", thumb fingertip "4.THUMB_TIP", index finger palmar-digital "5.INDEX_FINGER_MCP", index finger proximal interphalangeal joint "6.INDEX_FINGER_PIP", index finger distal interphalangeal joint "7.INDEX_FINGER_DIP", index fingertip "8.INDEX_FINGER_TIP", middle finger palmar-digital "9.MIDDLE_FINGER_MCP", middle finger proximal interphalangeal joint "9.MIDDLE_FINGER_MCP", middle finger proximal interphalangeal joint "10.INDEX_FINGER_C ... Interphalangeal joint "10.MIDDLE_FINGER_PIP", distal interphalangeal joint of middle finger "11.MIDDLE_FINGER_DIP", fingertip of middle finger "12.MIDDLE_FINGER_TIP", palmar interphalangeal joint of ring finger "13.RING_FINGER_MCP", proximal interphalangeal joint of ring finger "14.RING_FINGER_PIP", distal interphalangeal joint of ring finger "15.RING_FINGER_DIP", fingertip of ring finger "16.RING_FINGER_TIP", palmar interphalangeal joint of little finger "17.PINKY_MCP", proximal interphalangeal joint of little finger "18.PINKY_PIP", distal interphalangeal joint of little finger "19.PINKY_DIP", fingertip of little finger "20.PINKY_TIP"; After data preprocessing in steps S3 and S4, format conversion is performed, specifically converting the preprocessed XML format to YOLO format. In step S5, an improved YOLOv5 object detection algorithm is used to construct a Class I model. The training weights are selected from the official weights, and the number of iterations is 300. The improved YOLOv5 object detection algorithm adds a 160x160 detection feature map to the original detection head framework of the YOLOv5 object detection algorithm for detecting targets larger than 4x4. An attention mechanism is added to the original backbone network framework, and some of the original C3 modules are replaced with C3SE modules. The Anchors parameters are recalculated using the K-means clustering algorithm. In step S6, a type II model is constructed using the heatmap regression method. The heatmap regression method uses HRNet as the backbone network, and the hyperparameters set are: 300 iterations, Adam optimizer, and ReLU activation function. In step S8, the "cun" in the middle finger measurement method is calculated using Euclidean distance: in The x-coordinate of the distal interphalangeal joint of the middle finger x-axis of the proximal interphalangeal joint of the middle finger The distance between them The ordinate of the distal interphalangeal joint of the middle finger The ordinate of the proximal interphalangeal joint of the middle finger The distance between them The distance is measured in inches.

2. The method for precise acupoint localization based on cascaded deep neural networks according to claim 1, characterized in that: In steps S1 and S2, the Ha-GRID open-source hand dataset and the 11k Hands open-source palm dataset are used as initial samples. They contain hand data with different gestures, different lighting conditions and different skin colors, but do not contain hand data with physiological defects.

3. The method for precise acupoint localization based on cascaded deep neural networks according to claim 1, characterized in that: The training set and validation set are divided in an 8:2 ratio.

Images