Back acupoint positioning method based on prior information and deep learning

Abstract

The application discloses a back acupoint positioning method based on prior information and deep learning, which is good in real-time performance, high in accuracy and convenient to apply. The application comprises the following steps: (10) collecting back pictures, marking back acupoint points, and establishing a back data set; (20) improving a Keypomt Rcnn network and training an acupoint positioning model; (30) using the positioning model to preliminarily position the back pictures; (40) setting a posterior median line positioning algorithm, correcting the preliminary positioning result according to the posterior median line algorithm and setting a correction equation; (50) summarizing acupoint prior information, proposing an acupoint positioning evaluation index, and expanding the positioning result according to the prior information, and finally positioning 23 back acupoints.

Description

Technical Field

[0001] This invention belongs to the field of automatic acupoint localization, specifically relating to a method for locating back acupoints based on prior information and deep learning. Background Technology

[0002] Acupoints are specific points on the meridians of the human body and are an important part of Traditional Chinese Medicine (TCM) with a history of over two thousand years. TCM therapies combining acupoints, such as acupuncture and cupping, are widely used in Western countries as treatments for pain disorders, including migraines and lower back pain, due to their efficacy and safety. Similarly, medical devices such as acupuncture robots need to be integrated with acupoints to improve treatment effectiveness. However, as a relatively new field, methods for automatic and rapid acupoint location are still in their early stages and require urgent research.

[0003] Currently, methods for automatic acupoint localization mainly include the application of visual technology and the application of traditional intelligent algorithms. The process of applying visual technology can be summarized as: image acquisition, camera calibration, feature extraction, stereo matching, and 3D reconstruction. However, due to the complexity of camera calibration and correction, and the large analysis system reducing the real-time performance of localization, it suffers from significant limitations in application scenarios. Traditional intelligent algorithms, on the other hand, have a slow convergence process and low stability. In summary, current automatic acupoint localization methods suffer from limitations in application scenarios, low stability, poor real-time performance, and high equipment requirements, failing to simultaneously meet the requirements of high accuracy, robustness, and convenience. Image recognition and automatic acupoint localization are inextricably linked. Theoretically, the stronger the model's image recognition capability, the higher the accuracy of automatic acupoint localization. Compared to traditional image recognition, the biggest difference of deep learning lies in its utilization of big data, automatically learning features from large datasets containing tens of thousands of parameters. Furthermore, the more data used to train deep learning, the stronger the robustness and generalization ability of the deep learning algorithm. Due to the rapid development of deep learning, networks such as SPPNet, RCNN, and YOLO have been proposed. Compared with traditional image recognition, these networks have made a leap forward in their ability to extract image features, and their recognition efficiency is higher and their robustness is stronger. Therefore, many computer vision tasks such as image classification, object detection, semantic segmentation, and key point localization have been effectively solved.

[0004] Currently, the application of deep learning in automatic acupoint localization is still in its early stages. There is no precedent for using deep learning to solve the problem of automatic acupoint localization on the back. Therefore, this algorithm aims to study the application of deep learning in automatic acupoint localization on the back, with the goal of achieving acupoint localization on the back by recognizing only one image. It is dedicated to improving the accuracy of automatic acupoint localization, enhancing its robustness, and increasing its convenience and speed. Summary of the Invention

[0005] The purpose of this invention is to provide a back acupoint localization method based on prior information and an improved KeypointRcnn. This method can locate back acupoints by recognizing only one image, and has the characteristics of good real-time performance, high accuracy and convenient application.

[0006] The technical solution to achieve the purpose of this invention is as follows:

[0007] A method for locating back acupoints based on prior information and an improved Keypoint RCN includes the following steps:

[0008] Establish a back sample set: Collect back images, mark the largest inner rectangle of the back and 12 commonly used back acupoints, including Dazhui, Taodao, Shenzhu, Shendao, Lingtai, Zhiyang, Jinsuo, Zhongshu, Jizhong, Xuanshu, Mingmen and Yaoyangguan.

[0009] Improve and train the back acupoint localization model, including adopting the Adamax optimization algorithm, improving the loss function, and adding the CBAM attention mechanism;

[0010] Preliminary acupoint location: Acupoint location model is used to identify back images for preliminary acupoint location;

[0011] A rear midline positioning algorithm is set up, and the preliminary positioning results are corrected based on the rear midline algorithm and a correction equation is set up.

[0012] By summarizing the prior information on acupoints and expanding the localization results based on the prior information, a total of 23 acupoints on the back were finally located.

[0013] Compared with the prior art, the significant advantages of this invention are:

[0014] Easy to use: Low equipment requirements; acupoints on the back can be located simply by recognizing an image.

[0015] It has good real-time performance, with a single image positioning time of 1.9 seconds, which meets the real-time requirements.

[0016] It is highly robust, extracts deep information from images based on deep learning networks, and is not easily affected by noise. Attached Figure Description

[0017] Appendix Figure 1 Algorithm Flowchart

[0018] Appendix Figure 2 Keypoint RCNN network structure diagram

[0019] Appendix Figure 3 CBAM module structure and usage diagram

[0020] Appendix Figure 4 Flowchart of the method for locating the rear centerline

[0021] Appendix Figure 5 Acupoint Marking Chart

[0022] Appendix Figure 6 Acupoint location flowchart

[0023] Appendix Figure 7 Extended map of location results Detailed Implementation

[0024] The present invention will now be described in detail with reference to the accompanying drawings:

[0025] like Figure 1 As shown, the back acupoint localization method based on prior information and deep learning of the present invention includes the following steps: (10) Establishing a back sample set: collecting back images, marking the largest inner rectangle of the back and 12 commonly used back acupoints, including Dazhui, Taodao, Shenzhu, Shendao, Lingtai, Zhiyang, Jinsuo, Zhongshu, Jizhong, Xuanshu, Mingmen and Yaoyangguan.

[0026] (20) Improve and train the back acupoint location model: including the following steps: (201) Replace the SGD optimization algorithm used in the original model with the Adamax optimization algorithm.

[0027] (202) Using the improved loss function, with a1 = 3.7, a2 = 0.48, a3 = 3.7, a4 = 4.2, and a5 = 4.2, the formula is as follows:

[0028]

[0029] (203) Add the CBAM attention module, which includes the channel attention module and the spatial attention module.

[0030] (204) The dataset of back images is divided into training and testing sets in a ratio of 7:3. The abbreviations library is used to augment the back images and their labeled data, that is, to randomly change the brightness and contrast of the back images and rotate them 90 degrees a random number of times. The batch_size is set to 16 and the learning_rate is set to 0.001 to train the model for 50 epochs.

[0031] (30) Preliminary acupoint location: Acupoint location model was used to identify back images for preliminary acupoint location;

[0032] (40) Set up a rear centerline positioning algorithm, and correct the preliminary positioning results based on the rear centerline algorithm and set up a correction equation;

[0033] The back midline localization algorithm works as follows: First, the back image is binarized to increase recognition efficiency and highlight the back contour. Then, opening operations are used to eliminate image noise and remove small interference blocks. Finally, morphological gradient operations are used to extract the back boundary, and the formula is as follows:

[0034] gradient(src)=dilate(src)-erode(src)

[0035] In the formula, dilate(src) is the dilation operation, erode(src) is the erosion operation, and gradient(src) is the gradient of the image obtained by taking the difference between dilation and erosion.

[0036] The back boundary obtained from the gradient calculation above is contour fitted, the midpoint of the lower back contour line is marked, and the back centroid is marked using the centroid algorithm. Finally, the midpoint and the centroid are connected, and this line is the back midline that this paper aims to obtain. The centroid coordinates are... The calculation formula is as follows:

[0037] m ji =∑ x,y (array(x,y)*x i *x j )

[0038] In the formula, array(x,y) represents the gray value or binary value of pixel (x,y) in the image.

[0039]

[0040] In the formula, The coordinates of the centroid in the horizontal direction, i.e., the average x-coordinate of the image region. The coordinates of the centroid in the horizontal direction, i.e., the average y-coordinate of the image region, m 00 Zero-order moment: Represents the total number of pixels or area of ​​an image region.

[0041] (50) Summarize the prior information of acupoints, propose an evaluation index for acupoint location, expand the location results based on the prior information, and finally locate a total of 23 back acupoints.

[0042] Prior information about acupoints includes:

[0043] Acupoint correlation refers to the certain relationship between acupoints. For example, based on experience, the distance between Dazhui (GV14) and Shenzhu (GV12) is three inches; Shenzhu, Shendao, Zhiyang, Xuanshu, Mingmen, and Yaoyangguan are located on the posterior midline; Feishu, Xinshu, Geshu, Sanjiaoshu, Shenshu, and Dachangshu are located on the first lateral line of the Bladder Meridian.

[0044] Pohu, Shentang, Geguan, Mangmen, and Zhishi are located on the second lateral line of the Bladder Meridian; Shenzhu, Feishu, and Pohu are at the same level; Shendao, Xinshu, and Shentang are at the same level; Zhiyang, Geshu, and Geguan are at the same level; Xuanshu, Sanjiaoshu, and Mangmen are at the same level; Mingmen, Shenshu, and Zhishi are at the same level; Yaoyangguan and Dachangshu are at the same level (at the same level:

[0045] That is, the connecting line of these acupoints is perpendicular to the posterior median line);

[0046] The posterior median line is parallel to the first and second lateral lines of the Bladder Meridian. The distance between the posterior median line and the first lateral line of the Bladder Meridian is 1.5 cun, and the distance between the posterior median line and the second lateral line of the Bladder Meridian is 3 cun. Therefore, the posterior median line of the back can be located first, and then the points with large offsets can be corrected in combination with the posterior median line, and other acupoints on the back can be located by expanding according to the above experience.

[0047] The evaluation indexes for acupoint location include:

[0048] Calculate the Euclidean distance between the predicted value pt and the true value gt of N acupoint points according to the average error of the Euclidean distance, and take its average value as the quantization index:

[0049]

[0050] Measure the stability of the acupoint location algorithm according to the standard deviation of the Euclidean distance.

[0051]

[0052] where, e i represents the prediction error of the i-th test sample, represents the average value of the errors of all test samples.

[0053] Set the acupoint error threshold to 0.35 cun (the cun mentioned in this algorithm is the cun measured by the bone-measurement method, rather than a fixed value). When the distance between the predicted acupoint point and the true acupoint point is less than this threshold, the location of this acupoint point is accurate.

[0054]

[0055] Since it is necessary to convert the coordinate distance of the acupoints in the picture into the acupoint distance of the actual human body, this paper proposes a conversion method: According to traditional Chinese medicine theory, the distance between Dazhui acupoint and Shenzhu acupoint is three cun. Let r represent the coordinate error threshold and D represent the coordinate error allowable range area. Its calculation method is as follows:

[0056]

[0057] D = (x ± r, y ± r)

[0058] In the formula: x 大椎 and y 大椎The x and y coordinates of the Dazhui acupoint mark are respectively represented by x and y. 身柱 and y 身柱 y and y represent the horizontal and vertical coordinates of the Shenzhu acupoint marker, respectively, and x and y represent the acupoint location coordinates predicted by the model.

[0059] Based on the posterior midline and prior information, determine the positions of the first and second lateral lines of the Bladder Meridian. Draw a straight line perpendicular to the acupoint correction point and the posterior midline, intersecting the first and second lateral lines of the Bladder Meridian. The intersection point is the extended acupoint.

[0060] like Figure 2 The diagram shows the Keypoint R-CNN network architecture, which consists of a feature extractor structure, a region candidate network, and a Head of Interest (ROI) network. The feature extractor structure uses ResNet50 as the backbone network for feature extraction, which reduces hyperparameters while increasing complexity and improving accuracy. The region candidate network convolves the back image, uses softmax classification to identify candidate regions, and uses bounding boxes to regress and identify the true contours. Post-processing is then performed using anchor boxes, followed by non-maximum suppression fusion. The ROI network eliminates pixel shifts caused by quantization through ROIAlign and uses 3×3 convolutions, transposed convolutions, and upsampling for keypoint localization.

[0061] like Figure 3 The diagram shows the structure and usage of the CBAM module, which is divided into a channel attention module and a spatial attention module. First, the feature map is input into the channel attention module, which outputs the corresponding attention map. Then, the input feature map is multiplied by the attention map, and the output passes through the spatial attention module. The same operation is performed, and finally, the feature map is output.

[0062] like Figure 4 The diagram shows the flowchart of the method for locating the posterior midline. This process includes binarization, opening operation, morphological gradient operation, contour fitting, and centroid localization algorithm.

[0063] The back midline localization algorithm works as follows: First, the back image is binarized to increase recognition efficiency and highlight the back contour. Then, opening operations are used to eliminate image noise and remove small interference blocks. Finally, morphological gradient operations are used to extract the back boundary, and the formula is as follows:

[0064] gradient(src)=dilate(src)-erode(src)

[0065] The back boundary obtained from the gradient calculation above is contour fitted, the midpoint of the lower back contour line is marked, and the back centroid is marked using the centroid algorithm. Finally, the midpoint and the centroid are connected, and this line is the back midline that this paper aims to obtain. The centroid coordinates are... The calculation formula is as follows:

[0066] m ji =∑ x,y (array(x, y)*x) i *x j )

[0067]

[0068] The process for setting up the corrected equation is as follows: First, determine the equation L of the rear midline by locating it. Then, calculate the foot of the perpendicular O from each predicted point to the rear midline. GVi Then calculate the distance K from the predicted point to the foot of the perpendicular. GVi According to K at each prediction point GVi Value of K GVi Based on the sum of the values, a correction compensation coefficient G is proposed:

[0069] L: Ax0 + By0 + C = 0

[0070]

[0071]

[0072] In the formula, x0 and y0 are the coordinates of points on the line L, x GVi and yGVi Here, A, B, and C are the coordinates of the predicted acupoint, A, B, and C are the parameters of the linear equation, G is the correction compensation coefficient, and K is the coordinates of the predicted acupoint. GVi It is the distance from each prediction point to the foot of the perpendicular from the rear midline. It is the sum of the distances from the 3rd to the 14th prediction points.

[0073] Acupoint coordinate correction value set to

[0074] S x =G*(O GVi -x GVi ), S y =G*(O GVi -y Gvi )

[0075] Among them, S x and S y This is the acupoint coordinate correction amount, which indicates the correction of the acupoint coordinates based on the position of the perpendicular from the predicted point to the posterior midline. GVi The coordinates of the foot of the perpendicular, x GVi and y GVi These are the coordinates of the acupoint prediction points.

[0076] The corrected coordinates are:

[0077] (x c ,y c )=(xGVi +G*S x ,y GVi +G*S y )

[0078] Where (x) c ,y c (x) represents the corrected new coordinates. GVi and y GVi S represents the original predicted point coordinates. x and S y G is the correction amount, and G is the correction compensation coefficient.

[0079] like Figure 5 The diagram shown is a map of acupoints, marking the largest inner rectangle on the back and 12 acupoints on the back, including Dazhui, Taodao, Shenzhu, Shendao, Lingtai, Zhiyang, Jinsuo, Zhongshu, Jizhong, Xuanshu, Mingmen, and Yaoyangguan.

[0080] like Figure 6 The diagram shown is a flowchart for locating acupoints.

[0081] The results show that the predicted points and the marked points have a high degree of overlap. However, some points deviate. After acupoint correction, the overlap with the marked points is improved on the original basis. The Euclidean distance between the corrected points and the marked points is closer, and the accuracy of the model is higher.

[0082] like Figure 7 As shown, 11 acupoints were located through this step 0, so a total of 23 acupoints were located in this paper.

Claims

1. A method for locating acupoints on the back based on prior information and deep learning, characterized in that, Includes the following steps: (10) Establish a back sample set: Collect back images, mark the largest inner rectangle of the back and 12 commonly used back acupoints, including Dazhui, Taodao, Shenzhu, Shendao, Lingtai, Zhiyang, Jinsuo, Zhongshu, Jizhong, Xuanshu, Mingmen and Yaoyangguan. (20) Improve and train the back acupoint location model: including the following steps: (201) Replace the SGD optimization algorithm used in the original model with the Adamax optimization algorithm; (202) Using the improved loss function, with a1 = 3.7, a2 = 0.48, a3 = 3.7, a4 = 4.2, and a5 = 4.2, the formula is as follows: (203) Add the CBAM attention module, which includes the channel attention module and the spatial attention module; (204) Divide the back image dataset into training and testing sets in a 7:3 ratio. Use the abbreviations library to augment the back images and their labeled data, i.e., perform random changes in brightness and contrast and random 90-degree rotations on the back images. Set batch_size to 16 and learning_rate to 0.001 to train the model for 50 epochs. (30) Preliminary acupoint location: Acupoint location model was used to identify back images for preliminary acupoint location; (40) Set up a rear centerline positioning algorithm, and correct the preliminary positioning results based on the rear centerline algorithm and set up a correction equation; (50) Summarize the prior information of acupoints, propose an evaluation index for acupoint location, expand the location results based on the prior information, and finally locate a total of 23 back acupoints.

2. The method for locating acupoints on the back as described in claim 1, characterized in that, The back centerline algorithm in (40) includes the following steps: (41) Binarize the back image to increase recognition efficiency and highlight the back contour; (42) Opening operation eliminates image noise and removes small interference blocks; (43) Use morphological gradient operations to extract the back boundary; (44) Perform contour fitting on the back boundary obtained by gradient calculation and mark the midpoint of the lower back contour line; (45) Use the centroid algorithm to mark the back centroid, and the line connecting the midpoint and the centroid is the back midline; (46) Set up a correction equation to correct the preliminary positioning results.

3. The method for locating acupoints on the back as described in claim 2, characterized in that, The setting of the correction equation in (46) includes: (461) The equation of the back centerline is obtained by locating the back centerline, and then the foot of the perpendicular O from each predicted point to the back centerline is calculated. GVi Then calculate the distance K from the predicted point to the foot of the perpendicular. GVi According to K at each prediction point GVi Value of K GVi Based on the sum of the values, a correction compensation coefficient G is proposed: L: Ax0 + By0 + C = 0, In the formula, x0 and y0 are the coordinates of points on the line L, x GVi and y GVi Here, A, B, and C are the coordinates of the predicted acupoint, A, B, and C are the parameters of the linear equation, G is the correction compensation coefficient, and K is the coordinates of the predicted acupoint. GVi It is the distance from each prediction point to the foot of the perpendicular from the rear midline. It is the sum of the distances from the 3rd to the 14th prediction point; (462) The acupoint coordinate correction value is set to S x =G*(O GVi -x GVi ),S y =G*(O GVi -y GVi ), Among them, S x and S y This is the acupoint coordinate correction amount, which indicates the correction of the acupoint coordinates based on the position of the perpendicular from the predicted point to the posterior midline. GVi The coordinates of the foot of the perpendicular, x GVi and y GVi These are the coordinates of the acupoint prediction points; (463) The corrected new coordinates are: (x c ,y c )=(x GVi +G*S x ,y GVi +G*S y ), Where (x) c ,y c X represents the corrected new coordinates. Gvi and y GVi S represents the original predicted point coordinates. x and S y G represents the correction amount for the acupoint coordinates, and G is the correction compensation coefficient.

4. The method for locating acupoints on the back as described in claim 1, characterized in that, The a priori information of acupoints summarized in (50) is as follows: (51) Correlation between acupoints, that is, there is a certain relationship between the distances between acupoints. According to experience, the distance between Dazhui and Shenzhu acupoints is three inches; Shenzhu, Shendao, Zhiyang, Xuanshu, Mingmen and Yaoyangguan are located on the posterior midline; Feishu, Xinshu, Geshu, Sanjiaoshu, Shenshu and Dachangshu are located on the first lateral line of the bladder meridian; Pohu, Shentang, Geguan, Mangmen and Zhishi are located on the second lateral line of the bladder meridian. (52) Shenzhu, Feishu and Pohu are level; Shendao, Xinshu and Shentang are level; Zhiyang, Geshu and Geguan are level; Xuanshu, Sanjiaoshu and Mangmen are level; Mingmen, Shenshu and Zhishi are level; Yaoyangguan and Dachangshu are level; Level means that the line connecting these acupoints is perpendicular to the posterior midline; (53) The posterior midline is parallel to the first lateral line of the bladder meridian and the second lateral line of the bladder meridian. The distance between the posterior midline and the first lateral line of the bladder meridian is 1.5 cun, and the distance between the posterior midline and the second lateral line of the bladder meridian is 3 cun. Therefore, the posterior midline of the back can be located first, and then the points with large deviations can be corrected by combining the posterior midline. Based on the above experience, other acupoints on the back can be located. Here, "cun" is the cun measured in the bone measurement method, not a fixed value.

5. The method for locating acupoints on the back as described in claim 1, characterized in that, The acupoint location evaluation indicators mentioned in (50) include: (61) Calculate the Euclidean distance between the predicted value pt and the actual value gt of N acupoints based on the average error of the Euclidean distance, and take the average value as the quantitative indicator: (62) The stability of the acupoint location algorithm is measured by the standard deviation of Euclidean distance: (63) Based on literature and experience, the acupoint error threshold is set to 0.35 cun. When the distance between the predicted acupoint and the actual acupoint is less than this threshold, the acupoint is accurately located. Here, "cun" is the cun measured in the bone measurement method, not a fixed value. Where K is the number of test acupoints with an average error less than the acupoint error threshold; (64) Since it is necessary to convert the coordinate distance of acupoints in the image to the actual acupoint distance on the human body, this paper proposes a conversion method: According to traditional Chinese medicine theory, the distance between Dazhui acupoint and Shenzhu acupoint is three cun. Let r represent the coordinate error threshold and D represent the allowable range of coordinate error. The calculation method is as follows: D = (x ± r, y ± r), In the formula: x 大椎 and y 大椎 The x and y coordinates of the Dazhui acupoint mark are respectively represented by x and y. 身柱 and y 身柱 y and y represent the horizontal and vertical coordinates of the Shenzhu acupoint marker, respectively, and x and y represent the acupoint location coordinates predicted by the model.

6. The method for locating acupoints on the back as described in claim 1, characterized in that, The step of extending the positioning result based on prior information in (50) includes the following steps: (71) Determine the positions of the first lateral line and the second lateral line of the bladder meridian based on the posterior midline and prior information; (72) Draw a straight line perpendicular to the acupoint correction point and the posterior midline, intersecting the first side of the bladder meridian. The first line, the second lateral line, and their intersection point are the extended acupoints, ultimately locating a total of 23 acupoints on the back.

Images