Real-time acupuncture point positioning method and system based on three-dimensional point cloud index

Abstract

The invention discloses a real-time acupuncture point positioning method and system based on three-dimensional point cloud indexing.The method is applied to the technical field of image data processing.The method comprises the steps that a human body image of a user is obtained, the human body image is recognized and extracted, and posture parameters, shape parameters and expression parameters are obtained; iteratively calculating the posture parameters, the shape parameters and the expression parameters based on the constructed initial model, and outputting an optimal parameter result; estimating the joint point position of the user to obtain an estimation parameter; constructing a human body three-dimensional surface model according to the optimal parameter result and the estimation parameter; inputting the human body three-dimensional surface model based on the collected point cloud data as a template point cloud; acupoint positioning is carried out through the template point cloud and a position indexing method, and the position of an acupoint is generated; and displaying the acupoint position on a terminal device.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a real-time acupoint positioning method and system based on three-dimensional point cloud indexing. Background Art

[0002] Traditional Chinese medicine is very important in the field of medicine in my country. The location of acupoints has a long history in TCM diagnosis and treatment and plays an important role. At present, acupoint location mainly relies on TCM physicians based on bone measurement, body measurement and natural marker location methods, which have problems such as low efficiency, large workload, and reliance on physicians' experience and feelings. Therefore, it is particularly important to improve the accuracy of acupoint location, quantify the location of acupoints, and automate the process of acupoint selection, which is conducive to providing auxiliary diagnosis and treatment for TCM, and is also conducive to the inheritance and popularization of traditional TCM acupuncture treatment technology.

[0003] At present, the automatic acupoint positioning mainly includes the following instruments and methods, for example: a moxibustion acupoint positioning system and implementation method based on cloud computing and big data (Guo Xuanrui, Guo Xuanming. A moxibustion acupoint positioning system and implementation method based on cloud computing and big data [P]. Shaanxi: CN107731276A, 2018-02-23.), which requires complicated information collection equipment, which is not conducive to promotion and popularization; a facial acupoint positioning method and positioning device based on feature point positioning algorithm (Zhu Qing, Chang Menglong. Facial acupoint positioning method and positioning device based on feature point positioning algorithm [P]. Beijing: CN105930810A, 2016-09-07.), inputting a face image to obtain the facial feature reference points of the face image, according to the facial feature reference points of the face image, or inputting them into the training results of the neural network model, to obtain three different types of acupoints, this method uses facial feature points as input, the position of the acupoint may be affected by factors such as image pixels, and only for Facial acupoints are not holistic; a system and method for intelligent positioning of acupoints on the back of the human body (Cao Yingyu, Zhao Zhenxi, Chang Qi, Huang Junfen, Xue Long, Tang Liang, Wang Yu. A system and method for intelligent positioning of acupoints on the back of the human body [P]. Beijing: CN115409840A, 2022-11-29.), collect a picture of the back of the human body and introduce a semantic segmentation algorithm of the convolution kernel selection module, perform semantic segmentation on the back of the human body according to the picture of the back of the human body, obtain a semantic segmentation result and determine the back of the human body Acupoint location, this method only solves the acupoint positioning problem on the back; an adjustable acupoint positioning and fixing device (Zhu Qingguang, Guo Guangxin, Fang Min, Zhou Xu, Kong Lingjun. An adjustable acupoint positioning and fixing device [P]. Shanghai: CN115487061A, 2022-12-20.), a fixed ring and a fixed pad are fixed to various parts of the human body to help determine the position of the acupoints, thereby realizing acupoint positioning, but this method still requires manual positioning by professional doctors, and does not realize automatic positioning of the acupoints.

[0004] Therefore, in order to overcome these defects, the present application proposes a real-time acupoint positioning method and system based on three-dimensional point cloud indexing. Summary of the invention

[0005] The purpose of this application is to provide a real-time acupoint positioning method and system based on three-dimensional point cloud indexing, aiming to solve the above-mentioned problems.

[0006] To achieve the above objectives, this application provides the following technical solutions:

[0007] The present application provides a real-time acupoint positioning method based on three-dimensional point cloud indexing, comprising:

[0008] Acquire a human body image of a user, identify and extract the human body image, and obtain posture parameters, shape parameters, and expression parameters;

[0009] Based on the constructed initial model, iteratively calculate the posture parameters, shape parameters and expression parameters, and output the optimal parameter results;

[0010] estimating the joint point positions of the user to obtain estimated parameters;

[0011] Constructing a three-dimensional surface model of a human body according to the optimal parameter result and the estimated parameters;

[0012] Inputting the human body three-dimensional surface model based on the collected point cloud data as a template point cloud;

[0013] Acupoint positioning is performed by using the template point cloud and position indexing method to generate the position of the acupoint;

[0014] The acupuncture point locations are displayed on the terminal device.

[0015] Furthermore, the step of iteratively calculating the posture parameters, shape parameters and expression parameters based on the constructed initial model and outputting the optimal parameter results specifically includes the following steps:

[0016] Using pytorch and L-BFGS to perform line search of enhanced Wolfe conditions, and iteratively optimize the posture parameters, the shape parameters, and the expression parameters;

[0017] Calculate the Euclidean distance between the key points projected onto the two-dimensional image and the key points detected by the mediapipe pose algorithm;

[0018] Calculate the sum of the initial model's losses, including but not limited to: reprojection loss, body pose prior, face pose, hand pose, body shape, and face expression;

[0019] When the loss calculated by the initial model is less than the preset threshold, the initial fitting is successful, the iteration ends, and the optimal posture parameters, shape parameters, and expression parameters are output.

[0020] Furthermore, the step of estimating the joint point positions of the user to obtain estimated parameters specifically includes the following steps:

[0021] The depth-based skeleton estimation stream is converted into a pose parameter stream through a neural network to process the additional information provided by the depth information of the three-dimensional structure of the human body;

[0022] Calculate the rotation vector of the joint to match the Azure Kinect key points, using the forward calculation optimization algorithm and the Adam optimizer to adjust the learning rate;

[0023] After training, the correction of the Kinect skeleton angle is predicted, that is, the three-dimensional posture of the human body is estimated, and the real-time estimated parameters are obtained.

[0024] Furthermore, the step of inputting the collected point cloud data into the human body three-dimensional surface model as a template point cloud specifically includes the following steps:

[0025] From the three-dimensional surface model of the human body, storing the three-dimensional coordinates in the collected point cloud data into the three-dimensional surface model of the human body according to the data storage method;

[0026] The point cloud data is obtained by scanning the point cloud data of the traditional Chinese medicine acupuncture bronze man model by taking the acupoints in the bronze man model as the standard for acupoint positioning;

[0027] Fit the human body 3D surface model of the bronze man of traditional Chinese medicine acupuncture point cloud, and use the bronze man's 3D surface model of the human body as a template point cloud.

[0028] Furthermore, the step of locating acupoints by using the template point cloud and the position index method to generate the positions of acupoints specifically includes the following steps:

[0029] Segment all points in the template point cloud in three-dimensional space according to the spatial data structure of kd-tree, and divide the search space into layers;

[0030] Sort all the points after segmentation so that each point has a unique position index;

[0031] Acupoints are marked based on the template point cloud to record the three-dimensional coordinates of acupoints throughout the body;

[0032] Interpolating the three-dimensional coordinates of the acupuncture points in the template point cloud, and recording the corresponding relationship with the index of the original point;

[0033] Obtain the positions of the points of the acupuncture points and position indexes marked by the user in the template point cloud. For acupuncture points whose distance to the nearest point in the template point cloud is greater than a preset threshold, record the index of a group of nearest points and the position relationship between the acupuncture points and the group of points.

[0034] The human body three-dimensional surface model generated by the user is obtained, and the position of the acupuncture point corresponding to the user is obtained according to the index and relative position relationship of the acupuncture points marked in the template point cloud.

[0035] Furthermore, the step of segmenting all points in the template point cloud in three-dimensional space according to the spatial data structure of kd-tree and hierarchically dividing the search space specifically includes the following steps:

[0036] Based on the point p in the template point cloud i Three-dimensional coordinate position, select the variance σ of the coordinate values ​​in the x, y, and z directions x ,σ t ,σ z The maximum dimension, which is the dimension where the root node is located;

[0037]

[0038] Take the middle point on the dimension x where the root node is located as the root node, and split vertically from the root node; the root node is:

[0039] When n is an even number,

[0040] When n is an odd number,

[0041] In space, points containing small values ​​constitute the left subtree of the root node, and points containing large values ​​constitute the right subtree of the root node;

[0042] Select the middle point in the left subtree as the next split point, which is the left child of the root node; similarly, select the middle point in the right subtree as the next split point, which is the right child of the root node;

[0043] Repeat the selection until all points in the space are segmented.

[0044] Furthermore, the step of interpolating the three-dimensional coordinates of the acupuncture points in the template point cloud and recording the corresponding relationship with the index of the original point specifically includes the following steps:

[0045] Obtain the three-dimensional coordinates of all points in the template point cloud, and record the three-dimensional coordinates P of the acupuncture points marked in all points i The coordinates p of the point closest to the (x, y, z) point i ;

[0046] Set the threshold and calculate point P i and point p i The distance error between

[0047] When the distance error between two points is less than the threshold, get p i Index value and corresponding acupuncture point u in the human body 3D surface model i ;

[0048] When the distance error between two points is greater than the threshold, get p i The coordinates q of the nearest set of points in the 3D surface model of the human body i , and a set of index values;

[0049] The acupuncture point is interpolated into the template point cloud as the interpolation point v of the acupuncture point i , and record the coordinates of the nearest set of points q i The relative position relationship of

[0050] Mark all acupoints and get all marked acupoints u i and v i The position index and relative position relationship.

[0051] This application proposes a real-time acupoint positioning system based on three-dimensional point cloud indexing, including:

[0052] Acquisition module: obtains the user's body image, identifies and extracts the body image, and obtains posture parameters, shape parameters and expression parameters;

[0053] Model generation module: based on the constructed initial model, iteratively calculate the posture parameters, shape parameters and expression parameters, and output the optimal parameter results; estimate the joint point positions of the user to obtain estimated parameters; and construct a three-dimensional surface model of the human body according to the optimal parameter results and the estimated parameters;

[0054] 3D point cloud indexing module: inputting the human body 3D surface model based on the collected point cloud data as a template point cloud; locating acupoints through the template point cloud and position indexing method to generate the position of the acupoints;

[0055] Display module: displays the acupoint location on the terminal device.

[0056] The present application provides a device, which includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a real-time acupoint positioning method based on a three-dimensional point cloud index; the processor is used to execute the program instructions stored in the memory to implement a real-time acupoint positioning method based on a three-dimensional point cloud index.

[0057] The present application provides a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute a real-time acupoint positioning method based on a three-dimensional point cloud index.

[0058] The present application provides a real-time acupoint positioning method and system based on three-dimensional point cloud indexing, which has the following beneficial effects:

[0059] (1) The human body three-dimensional surface model proposed in this application is a parametric human body model. The posture parameters, shape parameters and expression parameters obtained according to the input key points are in line with the morphological characteristics of human body posture, body shape and expression, and are representative. At the same time, the human body three-dimensional surface model has the same point cloud arrangement format. When performing spatial position segmentation, the same part is assigned the same position index, and there will be no acupoint drift caused by image pixels when processing acupoint image information, thereby realizing acupoint positioning with controllable accuracy.

[0060] (2) There is no need to calculate the location of acupuncture points. The index values ​​of acupuncture points are recorded in the scanned point cloud of the bronze man of traditional Chinese medicine acupuncture, and the three-dimensional surface model of the human body fitted by the bronze man of traditional Chinese medicine acupuncture is obtained. The location of acupuncture points is obtained according to the same index value and affine transformation matrix, and the three-dimensional coordinates of the user's acupuncture points are calibrated, saving time.

[0061] (3) This application generates acupuncture points in a template point cloud that fits the user's body surface in any posture, which has a wide range of uses and can be used in acupuncture and moxibustion procedures as well as in the teaching of acupuncture points in traditional Chinese medicine;

[0062] (4) Real-time reconstruction of the SMPL-X model and application of a fast search algorithm can achieve real-time location of acupuncture points, which is more efficient;

[0063] (5) The present application utilizes the depth information of human body images to reduce the visual ambiguity caused by generating a three-dimensional human body surface model from a two-dimensional image, thereby achieving higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic flow chart of a real-time acupoint positioning method based on three-dimensional point cloud indexing according to Example 1 of the present application;

[0065] Figure 2 This is a structural schematic diagram of a real-time acupoint positioning system based on three-dimensional point cloud indexing according to Example 2 of the present application;

[0066] Figure 3 This is a schematic diagram of the device structure of Example 3 of the present application;

[0067] Figure 4 This is a schematic diagram of the storage medium structure of Example 4 of the present application. DETAILED DESCRIPTION

[0068] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0070] Example 1

[0071] See also Figure 1 , is a flow chart of a real-time acupoint positioning method based on three-dimensional point cloud indexing according to Example 1 of the present application; the steps include:

[0072] S1: Obtain a human body image of a user, identify and extract the human body image, and obtain posture parameters, shape parameters, and expression parameters.

[0073] In this embodiment, a U-Net network is used to recognize and segment the user's body image.

[0074] Normalize the key points detected by mediapipe pose to the interval (0, 1). Connect the key points of the face detected by mediapipe pose to the points with the highest similarity among the nodes of the human body 3D surface model, and construct a mapping relationship from mediapipe pose to the human body 3D surface model.

[0075] Use MediaPipe mesh predictor to extract expression and pose parameters in real time.

[0076] Use the gender classifier to obtain the gender probability, use the gender model with high probability, and use it as the basis to initialize the Tpose posture model. Take the full-body image and the key points obtained by mediapipe pose recognition as input. Use Resnet18 for training and finally output the gender probability. If the gender probabilities of the final output are all lower than the preset threshold, use the neutral model directly.

[0077] S2: Based on the constructed initial model, iteratively calculate the posture parameters, shape parameters and expression parameters, and output the optimal parameter results.

[0078] In this embodiment, pytorch and L-BFGS are used to perform line search for strengthening Wolfe conditions, and the posture parameters, the shape parameters, and the expression parameters are optimized iteratively.

[0079] Knowing the exact or approximate value of the camera focal length, estimate the camera translation and body orientation, then fix the camera parameters and optimize the shape parameters and pose parameters.

[0080] Calculate the Euclidean distance between the key points projected onto two dimensions by the initial model and the key points detected by the mediapipe pose algorithm.

[0081] Calculate the sum of the losses of the initial model, including but not limited to: reprojection loss, body pose prior, face pose, hand pose, body shape, and facial expression.

[0082] When the loss calculated by the initial model is less than the preset threshold, the initial fitting is successful, the iteration ends, and the optimal posture parameters, shape parameters, and expression parameters are output.

[0083] S3: Estimate the joint point positions of the user to obtain estimated parameters.

[0084] In this embodiment, the body shape vector is known, the key points of the human body are captured by the Azure Kinect camera, and the 3D human body markers are observed by the posture estimator at a certain moment.

[0085] Use pre-trained real-time estimators for body, face, and hand poses. Specifically, a recurrent neural network, GRU network, is used to make predictions using information from the previous frame.

[0086] The depth-based skeleton estimation stream is converted into a pose parameter stream through a neural network to process the additional information provided by the depth information of the three-dimensional structure of the human body.

[0087] Calculate the rotation vector of the joint to match the Azure Kinect key point. Specifically, a forward calculation optimization algorithm commonly used for training neural networks, the Adam optimizer, can adaptively adjust the learning rate.

[0088] After training, the correction of the Kinect skeleton angle is predicted, that is, the three-dimensional posture of the human body is estimated, and the real-time estimated parameters are obtained.

[0089] S4: constructing a three-dimensional surface model of the human body according to the optimal parameter result and the estimated parameters.

[0090] In this embodiment, the optimal parameter results and the real-time estimated parameters are fused to generate a set of coherent SMPL-X parameters, thereby achieving real-time generation of a three-dimensional surface model of the human body.

[0091] S5: Inputting the human body three-dimensional surface model based on the collected point cloud data as a template point cloud.

[0092] In this embodiment, the three-dimensional coordinates in the collected point cloud data are stored in the three-dimensional surface model of the human body according to the data storage method.

[0093] The point cloud data is obtained by scanning the bronze man of traditional Chinese medicine acupuncture, using the acupoints in the bronze man model as the standard for acupoint positioning.

[0094] Fit the human body 3D surface model of the bronze man of traditional Chinese medicine acupuncture point cloud, and use the bronze man's 3D surface model of the human body as a template point cloud.

[0095] S6: Acupoint positioning is performed using the template point cloud and position indexing method to generate the position of the acupoint.

[0096] In this embodiment, the acupoints are located by using the template point cloud and the position index method, and the step of generating the position of the acupoints specifically includes steps S61 to S65. The implementation method of each step is described in detail below.

[0097] S61: Segment all points in the template point cloud in three-dimensional space according to the spatial data structure of kd-tree, and divide the search space into layers.

[0098] Based on the point p in the template point cloud i Three-dimensional coordinate position, select the variance σ of the coordinate values ​​in the x, y, and z directions x ,σ y ,σ z The maximum dimension is the dimension where the root node is located. The formula is:

[0099]

[0100] Take the middle point on the dimension x where the root node is located as the root node, and split vertically from the root node; the root node is:

[0101] When n is an even number,

[0102] When n is an odd number,

[0103] In space, points containing small values ​​constitute the left subtree of the root node, and points containing large values ​​constitute the right subtree of the root node;

[0104] Select the middle point in the left subtree as the next split point, which is the left child of the root node; similarly, select the middle point in the right subtree as the next split point, which is the right child of the root node;

[0105] Repeat the selection until all points in the space are segmented.

[0106] S62: Sort all points after segmentation so that each point has a unique position index.

[0107] Sort each point in the point cloud in the order from the root node, the left node, and the right node.

[0108] S63: Mark the acupoints based on the template point cloud and record the three-dimensional coordinates of the acupoints throughout the body.

[0109] S64: interpolating and marking the three-dimensional coordinates of the acupuncture points in the template point cloud, and recording the corresponding relationship with the index of the original point.

[0110] Obtain the three-dimensional coordinates of all points in the template point cloud, and record the three-dimensional coordinates P of the acupuncture points marked in all points i The coordinates p of the point closest to the (x, y, z) point i .

[0111] Set the threshold to 5mm and calculate point P i and point p i When the distance error between two points is less than the threshold, get p i Index value and corresponding acupuncture point u in the human body 3D surface model i ; When the distance error between two points is greater than the threshold, get p i The coordinates q of the nearest set of points in the 3D surface model of the human body i , and a set of index values.

[0112] The acupuncture point is interpolated into the template point cloud as the interpolation point v of the acupuncture point i And record the coordinates q of the nearest set of points i Based on the above method, all acupoints are marked, and all marked acupoints u are obtained. i and v i The position index and relative position relationship.

[0113] S65: Obtain the positions of the points of the acupuncture points and position indexes marked by the user in the template point cloud. For acupuncture points whose distance to the nearest point in the template point cloud is greater than a preset threshold, record the index of a group of nearest points and the positional relationship between the acupuncture points and the group of points.

[0114] The posture parameters, shape parameters and expression parameters output from the human body 3D surface model are used as the template point cloud of the user.

[0115] All points in the template point cloud are traversed using the pre-order traversal method, so that the points in the user's template point cloud and the points of the human body three-dimensional surface model fitted by the traditional Chinese medicine acupuncture bronze man have the same spatial hierarchy division, and the same parts have the same position index.

[0116] For acupuncture points u that are less than the preset threshold i , get the point with the same index as the user in the template point cloud as the user's acupuncture point u i location.

[0117] For acupoints v greater than the preset threshold i , a set of index values ​​recorded by the human body three-dimensional surface model fitted by the traditional Chinese medicine acupuncture bronze man is used to obtain a set of points with the same index values ​​in the template point cloud of the user; according to the coordinates q of the neighboring points recorded in the human body three-dimensional surface model fitted by the traditional Chinese medicine acupuncture bronze man i , and get the affine transformation matrix.

[0118] Applying the affine transformation matrix to the interpolation points v of the acupuncture points in the three-dimensional surface model of the human body fitted by the traditional Chinese medicine acupuncture bronze man i , get the user's acupuncture point v i The location of the acupuncture points.

[0119] S66: Obtain the three-dimensional human body surface model generated by the user, and obtain the position of the acupuncture point corresponding to the user according to the index and relative position relationship of the acupuncture points marked in the template point cloud.

[0120] S7: Displaying the acupuncture point locations on a terminal device.

[0121] In this embodiment, the acupuncture point locations of the user are displayed on the augmented reality terminal device.

[0122] In summary, Example 1 of the present application uses the index method to locate acupoints in a three-dimensional human body surface model, stores the parameterized three-dimensional human body surface model according to the same data storage structure, and records the position index values ​​of the acupoints and the corresponding relationship; when the target three-dimensional human body surface model is read using the same data reading method, the three-dimensional coordinates of the user's acupoints are obtained according to the corresponding index values ​​and the affine transformation relationship. The traditional acupoint positioning method based on the feature point positioning algorithm requires target detection, target recognition or iterative calculation steps, which is time-consuming; while the present application marks the acupoints when detecting and identifying the three-dimensional human body surface model according to the marked acupoint position index values, which is real-time and provides a convenient and efficient method for acupoint positioning.

[0123] Example 2

[0124] See also Figure 2 , is a structural diagram of a real-time acupoint positioning system based on three-dimensional point cloud indexing according to Example 2 of the present application; the specific contents include:

[0125] Acquisition module: obtains the user's body image, identifies and extracts the body image, and obtains posture parameters, shape parameters and expression parameters;

[0126] Model generation module: based on the constructed initial model, iteratively calculate the posture parameters, shape parameters and expression parameters, and output the optimal parameter results; estimate the joint point positions of the user to obtain estimated parameters; and construct a three-dimensional surface model of the human body according to the optimal parameter results and the estimated parameters;

[0127] 3D point cloud indexing module: inputting the human body 3D surface model based on the collected point cloud data as a template point cloud; locating acupoints through the template point cloud and position indexing method to generate the position of the acupoints;

[0128] Display module: displays the acupoint location on the terminal device.

[0129] In this embodiment, the system consists of four modules, namely, an acquisition module, a model generation module, a three-dimensional point cloud index module, and a display module. The acquisition module acquires the human body image taken by the user based on the terminal device; the model generation module generates a three-dimensional surface model of the human body in real time; the three-dimensional point cloud index module generates the location of the acupuncture points by using the position index marked in the template point cloud; and the display module displays the user's acupuncture point location on the augmented reality terminal device.

[0130] In summary, Example 2 of the present application uses the three-dimensional surface model of the human body of the model generation module as a template point cloud for acupoint calibration, and can associate the input coordinates with the point cloud model by inputting the coordinates, so as to realize acupoint positioning of the model in any posture; wherein, the method of directly inputting the coordinate adjustment makes the acupoint positions marked in the template point cloud more accurate, and has a certain degree of adjustability when correcting the acupoint positions; compared with directly generating a three-dimensional surface model of the human body from a two-dimensional image, the use of depth information in the present application provides additional information about the three-dimensional structure of the body, which can better eliminate the single-angle visual ambiguity generated when generating a three-dimensional surface model of the human body from an RGB image, thereby reducing errors and improving the accuracy of posture estimation.

[0131] Example 3

[0132] See also Figure 3 , is a schematic diagram of the device structure of Embodiment 3 of the present application. The device 50 includes a processor 51 and a memory 52 coupled to the processor 51 .

[0133] The memory 52 stores program instructions for implementing the above-mentioned real-time acupoint positioning method based on three-dimensional point cloud indexing.

[0134] The processor 51 is used to execute program instructions stored in the memory 52 to implement real-time acupoint positioning based on three-dimensional point cloud indexing.

[0135] The processor 51 may also be referred to as a CPU (Central Processing Unit).

[0136] The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0137] Example 4

[0138] See also Figure 4 , which is a schematic diagram of the structure of the storage medium of Example 4 of the present application. The storage medium of the embodiment of the present application stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer, a server, a mobile phone, a tablet, and other devices.

[0139] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0140] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

[0141] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the appended claims and their equivalents.

[0142] Of course, the present invention may have many other implementations. Based on this implementation, other implementations obtained by ordinary technicians in this field without any creative work are all within the scope of protection of the present invention.

Claims

1. A real-time acupoint positioning method based on three-dimensional point cloud indexing, characterized in that: include: Acquire a human body image of a user, identify and extract the human body image, and obtain posture parameters, shape parameters, and expression parameters; Based on the constructed initial model, iteratively calculate the posture parameters, shape parameters and expression parameters, and output the optimal parameter results; estimating the joint point positions of the user to obtain estimated parameters; Constructing a three-dimensional surface model of a human body according to the optimal parameter result and the estimated parameters; Inputting the human body three-dimensional surface model based on the collected point cloud data as a template point cloud; Acupoint positioning is performed by using the template point cloud and position indexing method to generate the position of the acupoint; The acupuncture point locations are displayed on the terminal device.

2. A real-time acupoint positioning method based on three-dimensional point cloud indexing according to claim 1, characterized in that: The step of iteratively calculating the posture parameters, shape parameters and expression parameters based on the constructed initial model and outputting the optimal parameter results specifically includes the following steps: Using pytorch and L-BFGS to perform line search of enhanced Wolfe conditions, and iteratively optimize the posture parameters, the shape parameters, and the expression parameters; Calculate the Euclidean distance between the key points projected onto the two-dimensional image and the key points detected by the mediapipe pose algorithm; Calculate the sum of the initial model's losses, including but not limited to: reprojection loss, body pose prior, face pose, hand pose, body shape, and face expression; When the loss calculated by the initial model is less than the preset threshold, the initial fitting is successful, the iteration ends, and the optimal posture parameters, shape parameters, and expression parameters are output.

3. The real-time acupoint positioning method based on three-dimensional point cloud indexing according to claim 1 is characterized in that: The step of estimating the joint point positions of the user to obtain estimated parameters specifically includes the following steps: The depth-based skeleton estimation stream is converted into a pose parameter stream through a neural network to process the additional information provided by the depth information of the three-dimensional structure of the human body; Calculate the rotation vector of the joint to match the Azure Kinect key points, using the forward calculation optimization algorithm and the Adam optimizer to adjust the learning rate; After training, the correction of the Kinect skeleton angle is predicted, that is, the three-dimensional posture of the human body is estimated, and the real-time estimated parameters are obtained.

4. The real-time acupoint positioning method based on three-dimensional point cloud indexing according to claim 1 is characterized in that: The step of inputting the collected point cloud data into the human body three-dimensional surface model as a template point cloud specifically includes the following steps: From the three-dimensional surface model of the human body, storing the three-dimensional coordinates in the collected point cloud data into the three-dimensional surface model of the human body according to the data storage method; The point cloud data is obtained by scanning the point cloud data of the traditional Chinese medicine acupuncture bronze man model by taking the acupoints in the bronze man model as the standard for acupoint positioning; Fit the human body 3D surface model of the bronze man of traditional Chinese medicine acupuncture point cloud, and use the bronze man's 3D surface model of the human body as a template point cloud.

5. The real-time acupoint positioning method based on three-dimensional point cloud indexing according to claim 1 is characterized in that: The step of locating acupoints by using the template point cloud and the position index method to generate the positions of acupoints specifically includes the following steps: Segment all points in the template point cloud in three-dimensional space according to the spatial data structure of kd-tree, and divide the search space into layers; Sort all the points after segmentation so that each point has a unique position index; Acupoints are marked based on the template point cloud to record the three-dimensional coordinates of acupoints throughout the body; Interpolating the three-dimensional coordinates of the acupuncture points in the template point cloud, and recording the corresponding relationship with the index of the original point; Obtain the positions of the points of the acupuncture points and position indexes marked by the user in the template point cloud. For acupuncture points whose distance to the nearest point in the template point cloud is greater than a preset threshold, record the index of a group of nearest points and the position relationship between the acupuncture points and the group of points. The human body three-dimensional surface model generated by the user is obtained, and the position of the acupuncture point corresponding to the user is obtained according to the index and relative position relationship of the acupuncture points marked in the template point cloud.

6. A real-time acupoint positioning method based on three-dimensional point cloud indexing according to claim 5, characterized in that: The step of segmenting all points in the template point cloud in three-dimensional space according to the spatial data structure of kd-tree and hierarchically dividing the search space specifically includes the following steps: Based on the point p in the template point cloud i Three-dimensional coordinate position, select the variance σ of the coordinate values ​​in the x, y, and z directions x ,σ t ,σ z The maximum dimension, which is the dimension where the root node is located; Take the middle point on the dimension x where the root node is located as the root node, and split vertically from the root node; the root node is: When n is an even number, When n is an odd number, In space, points containing small values ​​constitute the left subtree of the root node, and points containing large values ​​constitute the right subtree of the root node; Select the middle point in the left subtree as the next split point, which is the left child of the root node; similarly, select the middle point in the right subtree as the next split point, which is the right child of the root node; Repeat the selection process until all points in the space are segmented.

7. The real-time acupoint positioning method based on three-dimensional point cloud indexing according to claim 5, characterized in that: The step of interpolating the three-dimensional coordinates of the acupuncture points in the template point cloud and recording the corresponding relationship with the index of the original point specifically includes the following steps: Obtain the three-dimensional coordinates of all points in the template point cloud, and record the three-dimensional coordinates P of the acupuncture points marked in all points i The coordinates p of the point closest to the (x,y,z) point i ; Set the threshold and calculate point P i and point p i The distance error between When the distance error between two points is less than the threshold, get p i Index value and corresponding acupuncture point u in the human body 3D surface model i ; When the distance error between two points is greater than the threshold, get p i The coordinates q of the nearest set of points in the 3D surface model of the human body i , and a set of index values; The acupuncture point is interpolated into the template point cloud as the interpolation point v of the acupuncture point i , and record the coordinates of the nearest set of points q i The relative position relationship of Mark all acupoints and get all marked acupoints u i and v i The position index and relative position relationship.

8. A system for real-time acupoint positioning method based on three-dimensional point cloud indexing according to claim 1, characterized in that: include: Acquisition module: obtains the user's body image, identifies and extracts the body image, and obtains posture parameters, shape parameters and expression parameters; Model generation module: based on the constructed initial model, iteratively calculates the posture parameters, shape parameters and expression parameters, and outputs the optimal parameter results; estimating the joint point positions of the user to obtain estimated parameters; Constructing a three-dimensional surface model of a human body according to the optimal parameter result and the estimated parameters; 3D point cloud indexing module: inputting the human body 3D surface model based on the collected point cloud data as a template point cloud; Acupoint positioning is performed by using the template point cloud and position indexing method to generate the position of the acupoint; Display module: displays the acupoint location on the terminal device.

9. A device, characterized in that: The device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a real-time acupoint positioning method based on three-dimensional point cloud indexing as described in any one of claims 1-7; and the processor is used to execute the program instructions stored in the memory to implement a real-time acupoint positioning method based on three-dimensional point cloud indexing.

10. A storage medium, characterized in that: The device stores program instructions executable by a processor, wherein the program instructions are used to execute a real-time acupoint positioning method based on three-dimensional point cloud indexing as described in any one of claims 1 to 7.

Images