Acupuncture point identification method, device, equipment and medium

Abstract

The embodiment of the invention provides an acupoint recognition method and device, equipment and a medium. The method comprises the steps of obtaining a to-be-recognized human body image; determining a point cloud projection image corresponding to the to-be-recognized human body image according to the to-be-recognized human body image; according to the point cloud projection image, the to-be-recognized human body image and a pre-trained recognition model, grid vertex information corresponding to the to-be-recognized human body image is determined, and the grid vertex information at least comprises coordinates of human body skin vertexes and position coordinates of human body joints; the human body acupoint position information corresponding to the grid vertex information is determined according to the grid vertex information and the pre-established acupoint template information, all the acupoint information of the human body in the image can be obtained by matching the acupoint template information, and the accuracy of acupoint recognition is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an acupoint recognition method, device, equipment and medium. Background Art

[0002] Acupuncture points in traditional Chinese medicine, scientifically known as acupoints, refer to special points on the meridian lines of the human body. As an important part of the theory of traditional Chinese medicine, they cooperate with meridians to form an important part of the theoretical system of traditional Chinese medicine. Acupuncture points are usually located on the meridians of the human body. Meridians are channels for describing the life activities of the human body in traditional Chinese medicine. The blood flowing through the meridians can affect various organs and tissues of the human body. The stimulation of acupuncture points can adjust the flow of blood and qi in the meridians and adjust the balance of yin and yang in the body, thereby achieving the effect of treating diseases, relieving symptoms and promoting health.

[0003] Accurate and rapid search for acupuncture points and meridian lines is the premise of Chinese medicine massage. After completing the acupuncture point identification, different treatment methods are applied to different acupuncture points to achieve the effect of treating the symptoms. How to effectively analyze the location of acupuncture points and meridian lines for different human bodies is the premise for the realization of intelligent physiotherapy robots, and it also affects the final treatment effect. Intelligent search for acupuncture points is completed by analyzing human body data and searching for acupuncture points through computer vision. The premise of intelligent analysis is to build a large number of human body data models, and use a large amount of labeled human acupuncture point data as input to train the AI ​​system to understand the distribution of human acupuncture points and find visual visualization features.

[0004] The current method is to select key points and use labeling to complete the labeling of human acupuncture points. They will use labels that are different from the color of human skin to stick on the acupuncture points, and then ensure that the human body is still and take another photo without labels. After this process, image data collection and effective labeling of these data in real applications can be realized.

[0005] If a large number of human acupuncture points need to be marked, it will take a long time to mark them. How to improve the marking efficiency is an urgent problem that needs to be solved. Summary of the invention

[0006] In view of the above problems, embodiments of the present invention are proposed to provide an acupoint identification method, device, equipment and medium that overcome the above problems or at least partially solve the above problems.

[0007] In a first aspect, an embodiment of the present invention provides an acupoint identification method, the method comprising:

[0008] Acquire a human body image to be identified;

[0009] According to the human body image to be identified, determining a point cloud projection image corresponding to the human body image to be identified;

[0010] Determine mesh vertex information corresponding to the human body image to be identified according to the point cloud projection image, the human body image to be identified and the pre-trained recognition model, wherein the mesh vertex information at least includes coordinates of human skin vertices and position coordinates of human joints;

[0011] According to the grid vertex information and the pre-established acupuncture point template information, the human body acupuncture point position information corresponding to the grid vertex information is determined;

[0012] The pre-trained recognition model is obtained by training the deep learning model using the first sample human body image;

[0013] The pre-established acupuncture point template information includes the corresponding relationship between the human body template library and the human body acupuncture point library, and the human body template includes the mesh vertex information of the human body image.

[0014] Optionally, determining a point cloud projection image corresponding to the human body image to be identified according to the human body image to be identified includes:

[0015] Determine a depth map of the human body image to be identified according to the human body image to be identified;

[0016] According to the depth map of the human body image to be identified, a point cloud projection image corresponding to the depth map is determined.

[0017] Optionally, the pre-trained recognition model is obtained by:

[0018] Acquire a first sample human body image;

[0019] According to the first sample human body image, determining a projected single-surface point cloud image corresponding to the first sample human body image;

[0020] Using a neural network model to perform feature compression on the first sample human body image and the single-sided point cloud image to obtain first high-dimensional information corresponding to the first sample human body image;

[0021] Encode the features of the human body template library to obtain the second highest dimensional information;

[0022] The first high-dimensional information and the second high-dimensional information are fused to obtain the third high-dimensional information;

[0023] Decompose the third high-dimensional information into feature blocks;

[0024] The self-attention mechanism is used to fuse and interact the feature blocks to obtain the dependency relationship between the feature blocks.

[0025] According to the dependency relationship between each feature block, the coordinates of the vertices of the human skin and the position coordinates of the human joints are determined;

[0026] Calculate the loss function based on the coordinates of the vertices of the human skin and the position coordinates of the human joints;

[0027] When the loss function is less than a preset value, the neural network model corresponding to the loss function is determined as the recognition model.

[0028] Optionally, the pre-established acupoint template information is obtained in the following manner:

[0029] Acquire a second sample human body image;

[0030] Using a human body segmentation algorithm to segment the second sample human body image, and obtaining point cloud information of a key area in the second sample human body image;

[0031] Performing main feature analysis on the point cloud information of the key area to determine the main component feature information corresponding to the point cloud information of the key area;

[0032] Determine a human body template model according to the principal component feature information of the shape change of the human body, the principal component feature information of the human body joint posture and the skinning algorithm;

[0033] The human body template model and the acupoint information of the human body are bound to obtain the acupoint template information.

[0034] In a second aspect, an embodiment of the present invention provides an acupoint identification device, the device comprising:

[0035] An acquisition module, used for acquiring a human body image to be identified;

[0036] A determination module, used to determine a point cloud projection image corresponding to the human body image to be identified according to the human body image to be identified;

[0037] A recognition module, used to determine mesh vertex information corresponding to the human image to be recognized based on the point cloud projection image, the human image to be recognized and a pre-trained recognition model, wherein the mesh vertex information at least includes coordinates of human skin vertices and position coordinates of human joints;

[0038] A matching module, used to determine the human body acupuncture point position information corresponding to the grid vertex information according to the grid vertex information and the pre-established acupuncture point template information;

[0039] The pre-trained recognition model is obtained by training the deep learning model using the first sample human body image;

[0040] The pre-established acupuncture point template information includes the corresponding relationship between the human body template library and the human body acupuncture point library, and the human body template includes the mesh vertex information of the human body image.

[0041] Optionally, a module is determined to:

[0042] Determine a depth map of the human body image to be identified according to the human body image to be identified;

[0043] According to the depth map of the human body image to be identified, a point cloud projection image corresponding to the depth map is determined.

[0044] Optionally, the device further includes a first establishing module, and the first establishing module is used to:

[0045] Acquire a first sample human body image;

[0046] According to the first sample human body image, determining a projected single-surface point cloud image corresponding to the first sample human body image;

[0047] Using a neural network model to perform feature compression on the first sample human body image and the single-sided point cloud image to obtain first high-dimensional information corresponding to the first sample human body image;

[0048] Encode the features of the human body template library to obtain the second highest dimensional information;

[0049] The first high-dimensional information and the second high-dimensional information are fused to obtain the third high-dimensional information;

[0050] Decompose the third high-dimensional information into feature blocks;

[0051] The self-attention mechanism is used to fuse and interact the feature blocks to obtain the dependency relationship between the feature blocks.

[0052] According to the dependency relationship between each feature block, the coordinates of the vertices of the human skin and the position coordinates of the human joints are determined;

[0053] Calculate the loss function based on the coordinates of the vertices of the human skin and the position coordinates of the human joints;

[0054] When the loss function is less than a preset value, the neural network model corresponding to the loss function is determined as the recognition model.

[0055] Optionally, the device further includes a second establishing module, and the second establishing module is used to:

[0056] Acquire a second sample human body image;

[0057] Using a human body segmentation algorithm to segment the second sample human body image, and obtaining point cloud information of a key area in the second sample human body image;

[0058] Performing main feature analysis on the point cloud information of the key area to determine the main component feature information corresponding to the point cloud information of the key area;

[0059] Determine a human body template model according to the principal component feature information of the shape change of the human body, the principal component feature information of the human body joint posture and the skinning algorithm;

[0060] The human body template model and the acupoint information of the human body are bound to obtain the acupoint template information.

[0061] In a third aspect, an embodiment of the present invention provides a terminal device, including: at least one processor and a memory;

[0062] The memory stores a computer program; and at least one processor executes the computer program stored in the memory to implement the acupoint identification method provided in the first aspect.

[0063] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed, the acupoint identification method provided in the first aspect is implemented.

[0064] The embodiments of the present invention include the following advantages:

[0065] The acupoint recognition method, device, equipment and medium provided in the embodiments of the present invention obtain a human body image to be recognized; determine a point cloud projection image corresponding to the human body image to be recognized based on the human body image to be recognized; determine mesh vertex information corresponding to the human body image to be recognized based on the point cloud projection image, the human body image to be recognized and a pre-trained recognition model, wherein the mesh vertex information includes at least the coordinates of the human skin vertices and the position coordinates of the human body joints; determine the human body acupoint position information corresponding to the mesh vertex information based on the mesh vertex information and pre-established acupoint template information. The embodiments of the present application can obtain all acupoint information of the human body in the image by matching the acupoint template information, thereby improving the accuracy of acupoint recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flowchart of steps of an embodiment of an acupoint identification method of the present invention;

[0067] Figure 2 It is a schematic diagram of the steps of template matching acupoint analysis of the present invention;

[0068] Figure 3 is a schematic diagram of the shape change process of the present invention;

[0069] Figure 4 It is a schematic diagram of the long-distance dependency relationship of the present invention;

[0070] Figure 5 It is another long-distance dependency diagram of the present invention;

[0071] Figure 6It is a schematic diagram of the result of acupoint identification of the present invention;

[0072] Figure 7 It is another schematic diagram of acupoint identification result of the present invention;

[0073] Figure 8 is a structural block diagram of an embodiment of an acupoint identification device of the present invention;

[0074] Fig. 9 It is a structural schematic diagram of a terminal device of the present invention. DETAILED DESCRIPTION

[0075] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] An embodiment of the present invention provides an acupoint identification method for identifying acupoints. The execution subject of this embodiment is an acupoint identification device, which is arranged on a terminal device, wherein the terminal device at least includes a computer, a tablet terminal, a robot, etc.

[0077] Reference Figure 1 , shows a flowchart of a method for identifying acupoints according to an embodiment of the present invention, the method may specifically include the following steps:

[0078] S101, obtaining a human body image to be identified;

[0079] Specifically, the terminal device collects a 3D image of a human body to be identified, that is, a three-dimensional image of a human body, through a camera.

[0080] S102, determining a point cloud projection image corresponding to the human body image to be identified according to the human body image to be identified;

[0081] Specifically, the terminal device processes the human body image to be identified to obtain a depth map of the human body image to be identified; and determines a point cloud projection image corresponding to the depth map based on the depth map of the human body image to be identified.

[0082] S103, determining mesh vertex information corresponding to the human body image to be identified according to the point cloud projection image, the human body image to be identified and the pre-trained recognition model, wherein the mesh vertex information at least includes coordinates of human skin vertices and position coordinates of human joints;

[0083] Specifically, the terminal device inputs the point cloud projection image and the human body image to be identified into a pre-trained recognition model to obtain the coordinates of the human skin vertices and the position coordinates of the human body joints of the human body image to be identified.

[0084] S104, determining the human body acupuncture point position information corresponding to the grid vertex information according to the grid vertex information and the pre-established acupuncture point template information;

[0085] The pre-trained recognition model is obtained by training the deep learning model using the first sample human body image;

[0086] The pre-established acupuncture point template information includes the corresponding relationship between the human body template library and the human body acupuncture point library, and the human body template includes the mesh vertex information of the human body image.

[0087] Specifically, the terminal device matches the obtained coordinates of the human skin vertices and the position coordinates of the human joints with the pre-established acupuncture point template information respectively to obtain the human acupuncture point position information in the image to be identified.

[0088] In the embodiment of the present application, a recognition model G is pre-trained. During use, the image information I and the corresponding depth map can be obtained by taking a 3D camera. The two pieces of information are input into the recognition model to obtain 6890 mesh vertices of the human body in the image. Then, all the acupuncture point information of the human body in the image can be obtained by matching the human acupuncture point template library proposed in the embodiment of the present application, such as Figure 5 These three-dimensional acupoint information has a wide range of applications, such as remote TCM teaching, trajectory guidance for physiotherapy robots, and TCM digitization.

[0089] The acupoint recognition method provided by the embodiment of the present invention obtains a human body image to be recognized; determines a point cloud projection image corresponding to the human body image to be recognized based on the human body image to be recognized; determines mesh vertex information corresponding to the human body image to be recognized based on the point cloud projection image, the human body image to be recognized and a pre-trained recognition model, wherein the mesh vertex information includes at least the coordinates of the human skin vertices and the position coordinates of the human body joints; determines the human body acupoint position information corresponding to the mesh vertex information based on the mesh vertex information and pre-established acupoint template information. The embodiment of the present application can obtain all acupoint information of the human body in the image by matching the acupoint template information, thereby improving the accuracy of acupoint recognition.

[0090] Optionally, determining a point cloud projection image corresponding to the human body image to be identified according to the human body image to be identified includes:

[0091] Determine a depth map of the human body image to be identified according to the human body image to be identified;

[0092] According to the depth map of the human body image to be identified, a point cloud projection image corresponding to the depth map is determined.

[0093] Specifically, during use, the 3D camera can be used to obtain image information I and the corresponding depth map, and the projection process of the depth map to the point cloud is completed using the following formula to obtain the corresponding V part .

[0094]

[0095] Where (x', y') represents the coordinates of each point in the image coordinate system, D represents the depth value, that is, the distance from the target point to the camera plane, and f x ,f y Represents the intrinsic parameter of the camera, that is, the scale factor in the xy direction. The depth map can be projected using the above formula to obtain a point cloud map (x, y, z).

[0096] Optionally, the pre-trained recognition model is obtained by:

[0097] Acquire a first sample human body image;

[0098] According to the first sample human body image, determining a projected single-surface point cloud image corresponding to the first sample human body image;

[0099] Using a neural network model to perform feature compression on the first sample human body image and the single-sided point cloud image to obtain first high-dimensional information corresponding to the first sample human body image;

[0100] Encode the features of the human body template library to obtain the second highest dimensional information;

[0101] The first high-dimensional information and the second high-dimensional information are fused to obtain the third high-dimensional information;

[0102] Decompose the third high-dimensional information into feature blocks;

[0103] The self-attention mechanism is used to fuse and interact the feature blocks to obtain the dependency relationship between the feature blocks.

[0104] According to the dependency relationship between each feature block, the coordinates of the vertices of the human skin and the position coordinates of the human joints are determined;

[0105] Calculate the loss function based on the coordinates of the vertices of the human skin and the position coordinates of the human joints;

[0106] When the loss function is less than a preset value, the neural network model corresponding to the loss function is determined as the recognition model.

[0107] Specifically, it is known that the human body template library cannot be applied because there are three parameters β and θ in the template library, and these two parameters need to be provided. In a real application scenario, these two parameter values need to be provided automatically. A model needs to be trained to provide these values to achieve the matching relationship between real human body data and the human body template library. There are generally two solutions for this process of obtaining β and θ. One is to solve for the optimal parameters through an iterative method with optimal error such as ICP, and the other is to directly train a model to predict the relevant parameters. Since it is difficult to collect high-quality full-body point cloud information in a real application scenario (only half-body point cloud can be obtained without occlusion), and considering that the iterative process requires a large amount of time consumption, the embodiment of this application adopts the second solution for the final design of the network model.

[0108] For the training related to digital humans, standard open-source datasets or fake synthetic human datasets are generally used, such as the human3.6M dataset, which contains human body point cloud data scanned in real scenarios and human body point cloud data. In these datasets, a lot of content such as single-sided point cloud data, overall point cloud data, and human body boundary information can be obtained. Based on this, a training dataset convenient for subsequent applications is made, and the dataset composition is where I represents the captured image information or the synthesized image information, and M represents the human body region Mask in the image, represents the 3D mesh vertex position coordinates of the real human body, represents the 3D joint coordinates of the real human body. It is hoped to train a transformer model to recover all the point cloud images from the original image I and the single-sided human body point cloud image and the joint coordinate positions of the human body Therefore, the input of the model is the image I and the projected single-sided point cloud image Vpart, and the output is and To be able to complete this task, a model as shown in Figure 2 is designed. The model takes SMPL as prior knowledge and fuses the input image information to complete the coordinate prediction of the final 6890 key points. These 6890 coordinate points correspond to 720 human body acupoints in the embodiment of this application.

[0109] In the embodiment of this application, the recognition model uses the transformer architecture. For the input <I represents the captured image information or the synthesized image information, V part projected single-sided point cloud image> ∈ R HWx4, first pass through a standard CNN to complete the feature compression to generate high-dimensional information, and also encode the features of the template library, and guide the prior knowledge of the template library to integrate into the current sample features. Then the acquired features are decomposed into different token blocks, and the decomposition process uses the form of average distribution. Then the following self-attention mechanism is used to complete the feature fusion and interaction between different tokens, so as to realize long-distance dependency relationships such as Figure 4 As shown, when occlusion occurs, prediction can also be made through the correlation between different points.

[0110]

[0111] The above formula is the self-attention module defined in the transformer. The module receives three different tokens, Q: the information to be queried, K: the vector to be queried, V: the vector obtained by the query, and d is the dimension of the vector. The right side of the formula consists of two parts. One is the softmax function, which is used to obtain the similarity weights of Q and K, and then use the weights to reset the V vector. In this process, each token will calculate the similarity with other tokens. This calculation method establishes a long-distance relationship between different tokens. Finally, the self-attention features are decoded to generate the final desired output: V3D, J3D (the coordinates of the vertices of the human skin and the position coordinates of the human joints predicted by the model). The acupuncture point location information can be obtained through these two features. The loss function of the entire model training process is represented by the following two functions:

[0112]

[0113]

[0114] Where: V 3D , J 3D They respectively represent the coordinates of the human skin vertices and the position coordinates of the human joints predicted by the model. represents the coordinates of the real human skin vertices and the position coordinates of the human joints. M and K represent the total number of skin vertices and the number of human joints respectively. L V , L J represents the loss function of the skin vertices and the joint node prediction loss function. It is hoped that the model can complete the mapping of these points under the guidance of the loss function, such as Figure 4 and Figure 5 shown.

[0115] Optionally, the pre-established acupoint template information is obtained in the following manner:

[0116] Acquire a second sample human body image;

[0117] Using a human body segmentation algorithm to segment the second sample human body image, and obtaining point cloud information of a key area in the second sample human body image;

[0118] Performing main feature analysis on the point cloud information of the key area to determine the main component feature information corresponding to the point cloud information of the key area;

[0119] Determine a human body template model according to the principal component feature information of the human body shape change, the principal component feature information of the human body joint posture and a skinning algorithm;

[0120] The human body template model and the acupoint information of the human body are bound to obtain the acupoint template information.

[0121] (2) In the embodiment of the present application, a human body template is first established, and then a human acupoint template library is established, as follows:

[0122] 1) Establishment of human body template

[0123] Acupuncture point prediction methods can be roughly divided into two categories. The first category is the common direct analysis of images to predict acupuncture point locations. This solution requires a large amount of data to train the mapping model, and the data set must be for acupuncture point recognition tasks. Data collection for this solution is very difficult. Another solution is to achieve acupuncture point alignment through template matching. Since digital human research is far ahead of acupuncture point analysis research, there are a lot of open source human body matching data with a data volume of up to one million. From this perspective, the human body matching solution has extremely strong environmental robustness. Because the data volume is extremely large and the synthetic environment is extremely complex, the use of this solution can greatly alleviate the problem of difficulty in collecting human body data in traditional Chinese medicine.

[0124] The next step is to use these millions of 3D human data to generate a template (SMPL-X) for subsequent matching. The SMPL-X standard human model is a point cloud sampling of millions of 3D human data sets, which is downsampled to 6890 points. Then, all these samples are analyzed by PCA to obtain the principal component features. After the above operations, a unified descriptor can be established. The influence of different descriptor parameters on the model is as follows: Figure 3 shown.

[0125] The skin model M receives two parameters: They respectively represent the shape changes and joint postures of the human body. and It is not a numerical value but a high-dimensional vector. When it is all set to 0, it is a standard human body model. Figure 3 As shown in (a), is the weight of the shape vector, when When changes occur, the human body model will increase the shape changes to produce effects such as height, weight, and local obesity. Figure 3 (b); It describes the posture of human joints. When the changes, the generated human body model will have changes in joint angles, such as Figure 3 As shown in (d). Through the descriptor, a human body model can be established using the following formula:

[0126]

[0127] W on the right side of the equation is the expression of the skinning function. Skinning is a technology in 3D digital human animation. It determines the position coordinates of each vertex of the human skin based on the posture of the 27 joints of the human body and the fat and thin shape characteristics of the human body. p Represents the skin vertices obtained from the standard 3D human body model. This vertex sequence does not actually introduce changes in human posture, but only describes the introduction of body shape vectors and joint posture The template changes, so the vertex sequence is through the body vector and joint posture The corrected human body template model is as follows Figure 3 (c) as shown. Represents the 3D position information of 27 joints of the human body, represents the human body's action posture, and ω represents the skin weight, that is, the influence of each joint posture on the change of skin vertices. The calculation of the above skin is shown in the following formula:

[0128]

[0129] In the above formula Already passed T p (represents the skin vertices obtained from the standard 3D human body model) After obtaining the skin vertex sequence of the human body template, the position coordinates of each vertex of this template need to be transferred after the human body changes its movement. The position coordinate transfer of each vertex is determined by each joint posture. The position J of each joint and the skin weight ω of each joint affect each other, and finally the coordinate position of each skin vertex after transfer is obtained by weighted summation.

[0130] Among them, Tp, The formula expression of is the prior art and will not be repeated here.

[0131] Furthermore, the human acupoint template library is established as follows:

[0132] After completing the establishment of the above-mentioned digital human body template, it is necessary to effectively associate all the acupuncture points of the human body. In the experiment, this project found that when the parameters β and θ of the human body shape model are changed, only the height and posture change, and each endpoint of the mesh is attached to a fixed position of the human body in a binding manner. Different parameters will only cause these binding points to have a certain displacement, and there will be no disorder in the order. Therefore, the acupuncture point information of the human body can be bound to the template information. That is, the 6890 key points P of the template are known, and the human body has a total of 720 key acupuncture points Ph. A mapping relationship F can be found so that F(P) = Ph to complete this mapping relationship.

[0133] The main application scenario in the embodiment of the present application is the moxibustion robot. Moxibustion emphasizes missing acupoints but not meridians, that is, meridians are more important for the effect of moxibustion, so a certain positioning theoretical error is allowed in the design. Under this setting, the present invention seeks a master of traditional Chinese medicine to directly mark the human body template. The master will first find two endpoints according to the distribution relationship of the acupoints on the human body, and then mark the acupoint information in the middle at equal intervals. For example, for the two bladder meridians on the back of a person, only two key acupoints need to be marked: Dazhui acupoint and Yaoyangguan. Other middle acupoints can be generated according to the proportion relationship of the human body. This operation effectively reduces the difficulty and amount of marking. This marking method is used for acupoints in areas such as hands, arms, legs, feet, and head. In this way, the mapping relationship from the acupoint template to the human body template library can be completed by marking the key points on the human body template library.

[0134] Template matching is used to complete the task of acupoint recognition. The embodiment of the present application uses a large-scale data set related to digital humans to establish a human body template, which has extremely strong environmental robustness.

[0135] Using 3D information as a guide to complete the entire template matching process, since 2D cameras lack the function of depth estimation, coordinate distortion often occurs. Using a 3D camera to guide the matching process can avoid this problem. A mapping method between the human template library and the human acupoint library was established, so that all 720 human acupoint information can be stably recovered from the 4890 points in the template library. The transformer network is used as the core architecture, which can effectively capture the long-distance dependencies of different acupoints, making the model more robust to distance.

[0136] First, use a 3D camera to take a photo, then use the human body segmentation algorithm to extract the human body area and the human body 3D point cloud information, and then integrate a 3D digital human body template. This human body template is trained using a public data set with millions of data. This template has multiple shape parameters to control the overall deformation of the human body, such as height, weight, local area weight, and the position of human limbs. You only need to complete the labeling of acupuncture points on this 3D digital human template. Then, this patent uses the collected three-dimensional human body data to match and deform the set template to make it overlap, and the complete labeling process of human acupuncture points can be completed. The training and testing scheme of the entire model is as follows: Figure 2 As shown, the acupoint results are shown as Figure 6 and Figure 7 .

[0137] The embodiment of the present application solves the four problems described above from the two perspectives of acquisition scheme design and algorithm design. For the first problem, the present invention uses the latest transformer algorithm framework for design. Compared with other models, the transformer can capture long-distance dependencies more strongly because of its self-attention mechanism. The self-attention mechanism will integrate the correlation between different acupoints and all other acupoints, thereby realizing the association between different acupoints. In addition, the present invention uses coordinate regression to complete network training. Coordinate regression requires the use of MLP for high-dimensional feature projection at the end, thereby realizing stronger acupoint coordinate relationship association.

[0138] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0139] The acupoint recognition method provided by the embodiment of the present invention obtains a human body image to be recognized; determines a point cloud projection image corresponding to the human body image to be recognized based on the human body image to be recognized; determines mesh vertex information corresponding to the human body image to be recognized based on the point cloud projection image, the human body image to be recognized and a pre-trained recognition model, wherein the mesh vertex information includes at least the coordinates of the human skin vertices and the position coordinates of the human body joints; determines the human body acupoint position information corresponding to the mesh vertex information based on the mesh vertex information and pre-established acupoint template information. The embodiment of the present application can obtain all acupoint information of the human body in the image by matching the acupoint template information, thereby improving the accuracy of acupoint recognition.

[0140] Another embodiment of the present invention provides an acupoint identification device for executing the acupoint identification method provided in the above embodiment.

[0141] Reference Figure 8 , shows a structural block diagram of an embodiment of an acupoint identification device of the present invention, the device may specifically include the following modules: an acquisition module 801, a determination module 802, an identification module 803 and a matching module 804, wherein:

[0142] The acquisition module 801 is used to acquire a human body image to be identified;

[0143] The determination module 802 is used to determine the point cloud projection image corresponding to the human body image to be identified according to the human body image to be identified;

[0144] The recognition module 803 is used to determine the mesh vertex information corresponding to the human body image to be recognized according to the point cloud projection image, the human body image to be recognized and the pre-trained recognition model, wherein the mesh vertex information at least includes the coordinates of the human skin vertices and the position coordinates of the human body joints;

[0145] The matching module 804 is used to determine the human body acupuncture point position information corresponding to the mesh vertex information according to the mesh vertex information and the pre-established acupuncture point template information;

[0146] The pre-trained recognition model is obtained by training the deep learning model using the first sample human body image;

[0147] The pre-established acupuncture point template information includes the corresponding relationship between the human body template library and the human body acupuncture point library, and the human body template includes the mesh vertex information of the human body image.

[0148] The acupoint recognition device provided by the embodiment of the present invention obtains a human body image to be recognized; determines a point cloud projection image corresponding to the human body image to be recognized based on the human body image to be recognized; determines mesh vertex information corresponding to the human body image to be recognized based on the point cloud projection image, the human body image to be recognized and a pre-trained recognition model, wherein the mesh vertex information includes at least the coordinates of the human skin vertices and the position coordinates of the human body joints; determines the human body acupoint position information corresponding to the mesh vertex information based on the mesh vertex information and pre-established acupoint template information. The embodiment of the present application can obtain all acupoint information of the human body in the image by matching the acupoint template information, thereby improving the accuracy of acupoint recognition.

[0149] Another embodiment of the present invention further supplements the acupoint identification device provided in the above embodiment.

[0150] Optionally, a module is determined to:

[0151] Determine a depth map of the human body image to be identified according to the human body image to be identified;

[0152] According to the depth map of the human body image to be identified, a point cloud projection image corresponding to the depth map is determined.

[0153] Optionally, the device further includes a first establishing module, and the first establishing module is used to:

[0154] Acquire a first sample human body image;

[0155] According to the first sample human body image, determining a projected single-surface point cloud image corresponding to the first sample human body image;

[0156] Using a neural network model to perform feature compression on the first sample human body image and the single-sided point cloud image to obtain first high-dimensional information corresponding to the first sample human body image;

[0157] Encode the features of the human body template library to obtain the second highest dimensional information;

[0158] The first high-dimensional information and the second high-dimensional information are fused to obtain the third high-dimensional information;

[0159] Decompose the third high-dimensional information into feature blocks;

[0160] The self-attention mechanism is used to fuse and interact the feature blocks to obtain the dependency relationship between the feature blocks.

[0161] According to the dependency relationship between each feature block, the coordinates of the vertices of the human skin and the position coordinates of the human joints are determined;

[0162] Calculate the loss function based on the coordinates of the vertices of the human skin and the position coordinates of the human joints;

[0163] When the loss function is less than a preset value, the neural network model corresponding to the loss function is determined as the recognition model.

[0164] Optionally, the device further includes a second establishing module, and the second establishing module is used to:

[0165] Acquire a second sample human body image;

[0166] Using a human body segmentation algorithm to segment the second sample human body image, and obtaining point cloud information of a key area in the second sample human body image;

[0167] Performing main feature analysis on the point cloud information of the key area to determine the main component feature information corresponding to the point cloud information of the key area;

[0168] Determine a human body template model according to the principal component feature information of the human body shape change, the principal component feature information of the human body joint posture and a skinning algorithm;

[0169] The human body template model and the acupoint information of the human body are bound to obtain the acupoint template information.

[0170] It should be noted that each implementable method in this embodiment may be implemented separately, or may be implemented in combination in any combination without conflict, and this application is not limited thereto.

[0171] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0172] The acupoint recognition device provided by the embodiment of the present invention obtains a human body image to be recognized; determines a point cloud projection image corresponding to the human body image to be recognized based on the human body image to be recognized; determines mesh vertex information corresponding to the human body image to be recognized based on the point cloud projection image, the human body image to be recognized and a pre-trained recognition model, wherein the mesh vertex information includes at least the coordinates of the human skin vertices and the position coordinates of the human body joints; determines the human body acupoint position information corresponding to the mesh vertex information based on the mesh vertex information and pre-established acupoint template information. The embodiment of the present application can obtain all acupoint information of the human body in the image by matching the acupoint template information, thereby improving the accuracy of acupoint recognition.

[0173] Still another embodiment of the present invention provides a terminal device for executing the acupoint identification method provided in the above embodiment.

[0174] Fig. 9 is a schematic diagram of the structure of a terminal device of the present invention, such as Fig. 9 As shown, the terminal device includes: at least one processor 901 and a memory 902;

[0175] The memory stores a computer program; and at least one processor executes the computer program stored in the memory to implement the acupoint identification method provided in the above embodiment.

[0176] The terminal device provided in the present embodiment obtains a human body image to be identified; determines a point cloud projection image corresponding to the human body image to be identified based on the human body image to be identified; determines mesh vertex information corresponding to the human body image to be identified based on the point cloud projection image, the human body image to be identified and a pre-trained recognition model, wherein the mesh vertex information includes at least the coordinates of the human skin vertices and the position coordinates of the human joints; determines the human acupoint position information corresponding to the mesh vertex information based on the mesh vertex information and pre-established acupoint template information. The embodiment of the present application can obtain all acupoint information of the human body in the image by matching the acupoint template information, thereby improving the accuracy of acupoint recognition.

[0177] Yet another embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the acupoint identification method provided in any of the above embodiments is implemented.

[0178] According to the computer-readable storage medium of the present embodiment, a human body image to be identified is acquired; based on the human body image to be identified, a point cloud projection image corresponding to the human body image to be identified is determined; based on the point cloud projection image, the human body image to be identified and a pre-trained recognition model, mesh vertex information corresponding to the human body image to be identified is determined, wherein the mesh vertex information includes at least the coordinates of the human skin vertices and the position coordinates of the human body joints; based on the mesh vertex information and pre-established acupoint template information, the human acupoint position information corresponding to the mesh vertex information is determined. The embodiment of the present application can obtain all acupoint information of the human body in the image by matching the acupoint template information, thereby improving the accuracy of acupoint recognition.

[0179] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0180] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0181] The embodiments of the present invention are described with reference to flowcharts and / or block diagrams of methods, electronic devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing electronic device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing electronic device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0182] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing electronic device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0183] These computer program instructions can also be loaded onto a computer or other programmable data processing electronic device so that a series of operating steps are executed on the computer or other programmable electronic device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable electronic device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0184] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0185] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or electronic device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or electronic device. In the absence of further restrictions, the elements defined by the sentence "comprises a..." do not exclude the presence of other identical elements in the process, method, article or electronic device that includes the elements.

[0186] The above is a detailed introduction to an acupoint identification method and an acupoint identification device provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for identifying acupoints, characterized in that: The method comprises: Acquire a human body image to be identified; According to the human body image to be identified, determining a point cloud projection image corresponding to the human body image to be identified; Determine mesh vertex information corresponding to the human body image to be identified according to the point cloud projection image, the human body image to be identified and a pre-trained recognition model, wherein the mesh vertex information at least includes coordinates of human skin vertices and position coordinates of human joints; Determining human body acupuncture point position information corresponding to the grid vertex information according to the grid vertex information and pre-established acupuncture point template information; Wherein, the pre-trained recognition model is obtained by training a deep learning model using a first sample human body image; The pre-established acupuncture point template information includes the correspondence between the human body template library and the human body acupuncture point library, and the human body template includes the mesh vertex information of the human body image.

2. The method according to claim 1, characterized in that: The step of determining, according to the human body image to be identified, a point cloud projection image corresponding to the human body image to be identified, comprises: Determining a depth map of the human body image to be identified according to the human body image to be identified; According to the depth map of the human body image to be identified, a point cloud projection image corresponding to the depth map is determined.

3. The method according to claim 1, characterized in that The pre-trained recognition model is obtained in the following manner: Acquire the first sample human body image; Determine, according to the first sample human body image, a projected single-surface point cloud image corresponding to the first sample human body image; Using a neural network model to perform feature compression on the first sample human body image and the single-plane point cloud image to obtain first high-dimensional information corresponding to the first sample human body image; Encode the features of the human body template library to obtain the second highest dimensional information; fusing the first high-dimensional information and the second high-dimensional information to obtain third high-dimensional information; Performing feature decomposition on the third high-dimensional information to obtain feature blocks; A self-attention mechanism is used to perform feature fusion and interaction on the feature blocks to obtain the dependency relationship between the feature blocks; According to the dependency relationship between the feature blocks, the coordinates of the vertices of the human skin and the position coordinates of the human joints are determined; Calculating a loss function according to the coordinates of the vertices of the human skin and the position coordinates of the human joints; When the loss function is less than a preset value, the neural network model corresponding to the loss function is determined as the recognition model.

4. The method according to claim 1, characterized in that The pre-established acupoint template information is obtained in the following manner: Acquire a second sample human body image; Using a human body segmentation algorithm to segment the second sample human body image, and obtaining point cloud information of a key area in the second sample human body image; Performing principal feature analysis on the point cloud information of the key area to determine principal component feature information corresponding to the point cloud information of the key area; Determine a human body template model according to the principal component feature information of the shape change of the human body, the principal component feature information of the human body joint posture and the skinning algorithm; The human body template model and the acupoint information of the human body are bound to obtain the acupoint template information.

5. An acupoint identification device, characterized in that: The device comprises: An acquisition module, used for acquiring a human body image to be identified; A determination module, used to determine a point cloud projection image corresponding to the human body image to be identified according to the human body image to be identified; A recognition module, used to determine the mesh vertex information corresponding to the human body image to be recognized according to the point cloud projection image, the human body image to be recognized and a pre-trained recognition model, wherein the mesh vertex information at least includes the coordinates of the human skin vertices and the position coordinates of the human body joints; A matching module, used to determine the human body acupuncture point position information corresponding to the mesh vertex information according to the mesh vertex information and pre-established acupuncture point template information; Wherein, the pre-trained recognition model is obtained by training a deep learning model using a first sample human body image; The pre-established acupuncture point template information includes the correspondence between the human body template library and the human body acupuncture point library, and the human body template includes the mesh vertex information of the human body image.

6. The device according to claim 5, characterized in that The determining module is used to: Determining a depth map of the human body image to be identified according to the human body image to be identified; According to the depth map of the human body image to be identified, a point cloud projection image corresponding to the depth map is determined.

7. The device according to claim 5, characterized in that The device further comprises a first establishing module, wherein the first establishing module is configured to: Acquire the first sample human body image; Determine, according to the first sample human body image, a projected single-surface point cloud image corresponding to the first sample human body image; Using a neural network model to perform feature compression on the first sample human body image and the single-plane point cloud image to obtain first high-dimensional information corresponding to the first sample human body image; Encode the features of the human body template library to obtain the second highest dimensional information; fusing the first high-dimensional information and the second high-dimensional information to obtain third high-dimensional information; Performing feature decomposition on the third high-dimensional information to obtain feature blocks; A self-attention mechanism is used to perform feature fusion and interaction on the feature blocks to obtain the dependency relationship between the feature blocks; According to the dependency relationship between the feature blocks, the coordinates of the vertices of the human skin and the position coordinates of the human joints are determined; Calculating a loss function according to the coordinates of the vertices of the human skin and the position coordinates of the human joints; When the loss function is less than a preset value, the neural network model corresponding to the loss function is determined as the recognition model.

8. The device according to claim 5, characterized in that The device further includes a second establishing module, wherein the second establishing module is configured to: Acquire a second sample human body image; Using a human body segmentation algorithm to segment the second sample human body image, and obtaining point cloud information of a key area in the second sample human body image; Performing principal feature analysis on the point cloud information of the key area to determine principal component feature information corresponding to the point cloud information of the key area; Determine a human body template model according to the principal component feature information of the shape change of the human body, the principal component feature information of the human body joint posture and the skinning algorithm; The human body template model and the acupoint information of the human body are bound to obtain the acupoint template information.

9. A terminal device, characterized in that: include: at least one processor and memory; The memory stores a computer program; The at least one processor executes the computer program stored in the memory to implement the acupoint identification method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the acupoint identification method according to any one of claims 1 to 4 is implemented.

Images