Human body meridian and acupoint positioning method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a human body meridian and acupoint positioning method and device, electronic equipment and a medium. The method comprises the steps that single-view-angle global and / or local target human body data of a target object is collected; inputting the target human body data into a pre-trained human body acupoint positioning network model for human body acupoint labeling; performing human body meridian and acupoint positioning on the target object based on the human body acupoint labeling result and the human body meridian direction; according to the human body acupoint positioning network model, a standard three-dimensional human body model is generated by constructing a first parameterized human body model of a standard posture and carrying out mesh refinement, acupoint points are marked on body surface vertexes of the model, and a standard three-dimensional human body acupoint model is constructed; and performing model training based on the standard three-dimensional human body acupoint model and the global and / or local parameterized human body data under the plurality of single views to obtain a human body acupoint positioning network model. Therefore, the positions of the acupoints and the meridians and collaterals can be found more accurately, quickly and visually, so that a meridian and acupoint system can be learned more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of human acupoint positioning and three-dimensional data processing, and in particular to a human meridian acupoint positioning method, device, electronic equipment and medium. Background Art

[0002] Acupoints, also known as acupuncture points, are the places where the qi of the human body's internal organs and meridians is infused into the body surface. The existing acupoint positioning method uses a handheld scanning device to scan a standard human body to obtain scanning data to establish a three-dimensional human body model, establish acupoints on the three-dimensional human body model to construct a three-dimensional meridian acupoint model, obtain scanning data at the surface landmarks of the patient's three-dimensional body surface structure model, match the constructed three-dimensional meridian acupoint model to the patient's three-dimensional body surface structure model, and display the patient's three-dimensional meridian acupoint system structure in real time.

[0003] However, the existing acupoint positioning methods, whether it is the three-dimensional data of the standard human body or the patient's body, need to use a handheld high-precision 3D scanner to obtain, which is not only expensive but also inconvenient to use; in addition, different people have different heights, weights, and subcutaneous tissue distributions, and Chinese acupuncture emphasizes the feel of getting qi, so it is difficult to use a fixed acupuncture depth to represent the acupoints; when matching the three-dimensional meridian acupoint model modeled on the standard human body to the three-dimensional surface structure model of the patient, it is necessary to pre-mark multiple surface landmarks on the three-dimensional human body model to complete the model matching. Since the human body is a non-rigid hinged structure with multiple structural properties such as height, weight, and body shape, it is difficult to obtain an accurate match between the two models by only stretching or shrinking the two models through dozens of surface landmarks. Summary of the invention

[0004] The present invention provides a method, device, electronic device and medium for locating human meridian and acupoints, which are used to solve the defects in the prior art that a handheld high-precision 3D scanner is needed to obtain three-dimensional data of the human body, and only dozens of body surface landmarks are used to stretch or shrink the three-dimensional meridian and acupoint model and the three-dimensional body surface structure model of the patient, and it is difficult to obtain an accurate match between the two models, so as to accurately, quickly and intuitively find the positions of meridians and acupoints.

[0005] The present invention provides a method for locating acupoints of human meridians, comprising: Collecting single-view global and / or local target human body data of the target object; Inputting the target human body data into a pre-trained human acupoint positioning network model, and performing human acupoint labeling on the target human body data through the human acupoint positioning network model to obtain human acupoint labeling results; Performing human meridian acupoint positioning on the target object based on the human acupoint annotation result and the human meridian trend; Among them, the human acupoint positioning network model is trained based on the following steps: Constructing a first parametric human model in a standard posture, and performing mesh refinement on the first parametric human model to generate a standard three-dimensional human model, where the number of surface points of the standard three-dimensional human model is greater than the number of surface points of the parametric human model; Annotating acupoint points on the body surface vertices of the standard three-dimensional human model to construct a standard three-dimensional human acupoint model; Obtaining global and / or local parametric human data under multiple single perspectives, and generating a three-dimensional human acupoint model corresponding to each global and / or local parametric human data under a single perspective based on the standard three-dimensional human acupoint model; Training a neural network model based on the global and / or local parametric human data under the multiple single perspectives and the three-dimensional human acupoint model corresponding to each global and / or local parametric human data under a single perspective to obtain a human acupoint positioning network model.

[0006] In a possible implementation manner, the method further includes: Connecting adjacent two acupoint points based on the human acupoint annotation result and the human meridian trend to obtain multiple line segments between acupoint points; Positioning multiple positioning points on each acupoint line segment, and searching for the nearest neighbor point of each positioning point on the target human data through the nearest neighbor search method; Setting corresponding colors for the nearest neighbor points of each positioning point based on different meridians to obtain the human meridian acupoint positioning corresponding to the target object.

[0007] In a possible implementation manner, the method further includes: Constructing a second parametric human model with different postures and body shapes; Performing mesh refinement on the second parametric human model with the same standard as the standard three-dimensional human acupoint model to generate a three-dimensional human model corresponding to the second parametric human model; Performing acupoint annotation on the three-dimensional human model through the acupoint annotation data in the standard three-dimensional human acupoint model to obtain a three-dimensional human acupoint model corresponding to the three-dimensional human model; Randomly setting the position of a virtual camera, and obtaining global and / or local human data of the three-dimensional human model under multiple single perspectives through the virtual camera, as well as the three-dimensional human acupoint model corresponding to each global and / or local human data under a single perspective.

[0008] In a possible implementation manner, the method further includes: Inputting the global and / or local parameterized human body data under the multiple single-view angles and the three-dimensional human acupoint model corresponding to the global and / or local parameterized human body data under each single-view angle into the neural network model, using the global and / or local parameterized human body data under the multiple single-view angles as model input data, and using the three-dimensional human acupoint model corresponding to the global and / or local parameterized human body data under each single-view angle as the true value to train the neural network model; During the training of the neural network model, adjusting the network parameters by using an error back propagation algorithm; When the neural network model is trained to a model convergence state, it is determined that the training of the neural network model is completed, and a human acupoint positioning network model is obtained.

[0009] In a possible implementation, the method further includes: Acupoints are marked on the surface vertices of the standard three-dimensional human body model based on the specified standard human acupoint surface positioning method and meridian acupoint names to construct a standard three-dimensional human acupoint model.

[0010] In a possible implementation, the method further includes: The human body meridian acupoint positioning result corresponding to the target object is displayed on an external display.

[0011] In a possible implementation, the method further includes: The single-view global and / or local target human body data of the target object is collected through the camera on the mixed reality device.

[0012] The present invention also provides a human body meridian acupoint positioning device, comprising the following modules: An acquisition module, used for acquiring single-view global and / or local target human body data of a target object; A labeling module, used for inputting the target human body data into a pre-trained human acupoint positioning network model, and labeling the target human body data with human acupoints through the human acupoint positioning network model to obtain human acupoint labeling results; A positioning module, used for positioning the human meridian acupoints of the target object based on the human acupoint marking results and the direction of the human meridians; A model training module is used to construct a first parametric human body model of a standard posture, refine the mesh of the first parametric human body model to generate a standard three-dimensional human body model, where the number of surface points of the standard three-dimensional human body model is greater than that of the parametric human body model; mark acupoint points on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human body acupoint model; obtain global and / or local parametric human body data under multiple single viewpoints, and generate a three-dimensional human body acupoint model corresponding to the global and / or local parametric human body data under each single viewpoint based on the standard three-dimensional human body acupoint model; train a neural network model based on the global and / or local parametric human body data under the multiple single viewpoints and the three-dimensional human body acupoint model corresponding to the global and / or local parametric human body data under each single viewpoint to obtain a human body acupoint positioning network model.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the human meridian and acupoint positioning method described in any one of the above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the human meridian and acupoint positioning method described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the human meridian and acupoint positioning method described in any one of the above is implemented.

[0016] The human meridian acupoint positioning method, device, electronic device and medium provided by the present invention collect single-view global and / or local target human body data of a target object; input the target human body data into a pre-trained human acupoint positioning network model, and perform human acupoint annotation on the target human body data through the human acupoint positioning network model to obtain a human acupoint annotation result; perform human meridian acupoint positioning on the target object based on the human acupoint annotation result and the human meridian trend; wherein, the human acupoint positioning network model is trained based on the following steps: constructing a first parameterized human body model in a standard posture, and performing mesh refinement on the first parameterized human body model to generate a standard three-dimensional human body model, the number of surface points of the standard three-dimensional human body model being greater than the number of surface points of the parameterized human body model; annotating acupoint points on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human acupoint model; obtaining a plurality of single-view global and / or local parameterized human body data, and generating a three-dimensional human acupoint model corresponding to each single-view global and / or local parameterized human body data based on the standard three-dimensional human acupoint model; training a neural network model based on the plurality of single-view global and / or local parameterized human body data and the three-dimensional human acupoint model corresponding to each single-view global and / or local parameterized human body data to obtain a human acupoint positioning network model. Compared with the prior art in which it is necessary to obtain human three-dimensional data by means of a handheld high-precision 3D scanner and only stretch or shrink between a three-dimensional stereoscopic meridian acupoint model and a three-dimensional stereoscopic body surface structure model of a patient through dozens of body surface landmark points, it is difficult to obtain an accurate match between the two models. With this solution, the positions of meridians and acupoints can be found more accurately, quickly and intuitively, so as to more accurately learn the meridian acupoint system, and then more accurately and effectively implement traditional Chinese medicine treatments such as acupuncture and massage. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a flowchart of the human meridian acupoint positioning method provided by the present invention.

[0019] Figure 2 It is a flowchart of the training method of the human acupoint positioning network model provided by the present invention.

[0020] Figure 3 It is one of the schematic diagrams of the human meridian acupoint positioning result provided by the present invention.

[0021] Figure 4 It is the second schematic diagram of the positioning result of human meridian acupoints provided by the present invention.

[0022] Figure 5 It is the schematic diagram of the standard three-dimensional human acupoint model provided by the present invention.

[0023] Figure 6 It is the schematic diagram of the second parameterized human body model with different postures and body types provided by the present invention.

[0024] Figure 7 It is the schematic diagram of the structure of the human meridian acupoint positioning device provided by the present invention.

[0025] Figure 8 It is the schematic diagram of the structure of the electronic device provided by the present invention. Detailed implementation manners

[0026] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0027] For the convenience of understanding the embodiments of the present invention, the following will further explain and illustrate with specific embodiments in conjunction with the accompanying drawings. The embodiments do not constitute a limitation to the embodiments of the present invention.

[0028] Figure 1 It is the flow schematic diagram of the human meridian acupoint positioning method provided by the present invention. As Figure 1 shown, the method includes the following: S11. Collect the single-view global and / or local target human body data of the target object.

[0029] The embodiments of the present invention are preferentially applicable to the use scenarios of locating human meridian acupoints, for example, Chinese medicine acupuncture, conditioning, scientific research, popular science and other scenarios. The user collects the single-view global and / or local target human body data of the target object from the user's perspective through the RGBD camera that comes with the mixed reality device. Among them, the user can be a doctor, a family member of the patient, or a practitioner in a Chinese medicine conditioning place, and the target object is the target patient. The mixed reality device can be a head-mounted mixed reality device or mixed reality glasses, etc. The target human body data can be a single-view human body RGB and depth image of the target object obtained in real time from the user's perspective, and the target human body model under this perspective is obtained based on the human body image analysis. The collection of human body data through the RGBD camera that comes with the mixed reality device does not require additional three-dimensional data acquisition equipment, which greatly improves the convenience and real-time performance of use and reduces costs.

[0030] S12, inputting the target human body data into a pre-trained human acupoint positioning network model, and performing human acupoint labeling on the target human body data through the human acupoint positioning network model to obtain human acupoint labeling results.

[0031] In an embodiment of the present invention, a human acupoint localization network model is pre-trained, and the model has the ability to deform the obtained global and / or local single-view human body data onto a standard human body model. The target human body data is input into the pre-trained human acupoint localization network model, and the acupoint points marked on the standard human body model can be mapped in real time to the obtained single-view global and / or local human body data of the target object, and then the acupoint localization of the target object itself is constructed on the obtained global and / or local single-view human body data of the target object to obtain the human acupoint labeling result. The specific model training process is as follows: Figure 2 The corresponding embodiments are described in detail and will not be described in detail here.

[0032] S13, locating the human meridian acupoints of the target object based on the human acupoint labeling results and the directions of the human meridians.

[0033] Further, based on the human acupoint annotation results of the above model and the existing standard human fourteen meridian directions, two adjacent acupoint points are connected to obtain multiple line segments between acupoint points; multiple positioning points are taken on each acupoint line segment, and the nearest neighbor point of each positioning point on the single-view global and / or local human body data of the target object is found through the nearest neighbor search method; different colors are preset for each meridian, and corresponding colors are set for the nearest neighbor points of each positioning point found based on different meridians, so as to obtain the human meridian acupoint positioning corresponding to the target object, such as Figure 3 and Figure 4 shown.

[0034] Specifically, along the running directions of the fourteen meridians, connect the positions of adjacent acupoints in three-dimensional space to obtain line segments in three-dimensional space. Then, set sufficiently small equal intervals to take points on these line segments. Use the nearest neighbor search method to find the nearest neighbor points on the global and / or local single-view human body data of the obtained target object, and assign different colors to these points according to different meridians. In this way, a three-dimensional meridian acupoint model of the target object's human body surface from the user's perspective can be obtained in real time and displayed on the virtual reality device. As the user's perspective moves, the user can real-time locate the three-dimensional acupoint model of the target object's human body surface at the current perspective.

[0035] Further, an external display can be connected to display the positioning of the human meridian acupoints of the target object on the external display, so that others except the user can see the three-dimensional human meridian acupoint model of the patient's human body surface in real time.

[0036] The human meridian acupoint positioning method provided by the present invention includes: collecting the single-view global and / or local target human body data of the target object; inputting the target human body data into a pre-trained human acupoint positioning network model, and performing human acupoint annotation on the target human body data through the human acupoint positioning network model to obtain a human acupoint annotation result; performing human meridian acupoint positioning on the target object based on the human acupoint annotation result and the running directions of human meridians. Among them, the human acupoint positioning network model is trained based on the following steps: constructing a first parameterized human body model in a standard posture, and performing mesh refinement on the first parameterized human body model to generate a standard three-dimensional human body model, the number of surface points of the standard three-dimensional human body model being greater than that of the parameterized human body model; annotating acupoints on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human acupoint model; obtaining a plurality of single-view global and / or local parameterized human body data, and generating a three-dimensional human acupoint model corresponding to each single-view global and / or local parameterized human body data based on the standard three-dimensional human acupoint model; training a neural network model based on the plurality of single-view global and / or local parameterized human body data and the three-dimensional human acupoint models corresponding to each single-view global and / or local parameterized human body data to obtain a human acupoint positioning network model. Compared with the prior art, which requires the use of a handheld high-precision 3D scanner to obtain human three-dimensional data and only stretches or shrinks between the three-dimensional stereoscopic meridian acupoint model and the three-dimensional stereoscopic body surface structure model of the patient through dozens of body surface landmark points, it is difficult to obtain an accurate match between the two models. With this method, the positions of acupoints and meridians can be found more accurately, quickly, and intuitively, so as to more accurately obtain the meridian acupoint system, and then more accurately and effectively implement traditional Chinese medicine treatments such as acupuncture and massage.

[0037] Figure 2It is a schematic flowchart of the method for training a human acupoint positioning network model provided by the present invention, and the method includes the following: S21. Construct a first parameterized human body model in a standard posture, and refine the mesh of the first parameterized human body model to generate a standard three-dimensional human body model.

[0038] In the embodiment of the present invention, a first parameterized human body model in a standard posture is constructed in a parameterized human body model generation system. The parameterized human body model generation system can generate a human body model by setting human body shape and posture parameters, without the need for additional scanning or obtaining other standard human body models defined by rules.

[0039] Furthermore, in order to improve the accuracy of acupoint annotation of the standard human body model, the first parameterized human body model is refined multiple times to increase the number of surface points of the human body model and generate a standard three-dimensional human body model. Mesh refinement refers to the process in computer graphics of increasing the number of mesh points on the model surface to improve model details. This process can be achieved through various algorithms, such as Loop subdivision, Catmull-Clark subdivision, or Doo-Sabin subdivision. By refining the mesh multiple times, the surface of the human body model can be made smoother and more detailed. Increasing the number of surface points in the human body model can improve the resolution of the model, enabling the model to more accurately simulate the fine structures of the human body, such as muscles and skin folds. This is particularly important for acupoint annotation because the positions of acupoints usually require very precise positioning.

[0040] S22. Mark acupoint points on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human body acupoint model.

[0041] On the body surface vertices of the refined standard three-dimensional human body model above, according to the positioning rules of traditional Chinese medicine acupoints, each acupoint can be accurately marked. By marking acupoint points on the body surface vertices of the human body model, a complete standard three-dimensional human body acupoint model can be constructed. It should be noted that the standard three-dimensional human body acupoint model includes multiple acupoint positioning point data, that is, acupoint coordinate data, as Figure 4 shown. This model not only contains the geometric information of the human body but also the accurate positions of acupoints, providing an intuitive tool for the diagnosis, treatment, and teaching of traditional Chinese medicine.

[0042] S23. Obtain multiple global and / or local parameterized human body data under a single viewing angle, and generate a three-dimensional human body acupoint model corresponding to each global and / or local parameterized human body data under a single viewing angle based on the standard three-dimensional human body acupoint model.

[0043] Construct multiple second parametric human models with different postures and body types in the parametric human model generation system. Further, refine the meshes of the multiple second parametric human models according to the same mesh refinement criteria as the standard three-dimensional human acupoint model to obtain a three-dimensional human model corresponding to each second parametric human model, as Figure 6 shown.

[0044] Further, annotate the acupoints of the above-obtained three-dimensional human model with the acupoint annotation data in the standard three-dimensional human acupoint model to obtain a three-dimensional human acupoint model corresponding to the three-dimensional human model.

[0045] It should be noted that the standard three-dimensional human acupoint model includes multiple acupoint positioning point data, that is, acupoint coordinate data. For example, if 700 acupoint points are marked on the standard three-dimensional human model, the standard three-dimensional human acupoint model includes the marked serial numbers of these 700 acupoint points and the coordinate data of each acupoint point.

[0046] Further, randomly set the position of the virtual camera, and obtain global and / or local human data of the three-dimensional human model from multiple single perspectives through the virtual camera. Finally, multiple global and / or local human data from multiple single perspectives and the three-dimensional human acupoint models corresponding to the global and / or local human data from each single perspective can be obtained.

[0047] For example, 100,000 second parametric human models with different postures and body types can be constructed and mesh-refined in the parametric human model generation system to obtain a three-dimensional human model corresponding to each second parametric human model, and then acupoint annotation can be performed to obtain a three-dimensional human acupoint model corresponding to the three-dimensional human model. Then, randomly set the position of a virtual camera in each second parametric human model scene, and 100,000 global and / or local parametric human data from multiple single perspectives can be obtained through the virtual camera. Or, randomly set the positions of five virtual cameras in each second parametric human model scene, and 500,000 global and / or local parametric human data from multiple single perspectives can be obtained through these virtual cameras. Further, for example, if a virtual camera captures the back data of the three-dimensional human model, the acupoint data on the back (including the acupoint serial number and the coordinate data of each acupoint) can be obtained according to the three-dimensional human acupoint model corresponding to this three-dimensional human model.

[0048] S24. Train a neural network model based on the global and / or local parametric human data from multiple single perspectives and the three-dimensional human acupoint models corresponding to the global and / or local parametric human data from each single perspective to obtain a human acupoint positioning network model.

[0049] Input the obtained multiple global and / or local parametric human body data under a single perspective and the corresponding three-dimensional human acupoint models for each single perspective of global and / or local parametric human body data into the neural network model. Use the global and / or local parametric human body data under multiple single perspectives as the model input data, and use the corresponding three-dimensional human acupoint models for each single perspective of global and / or local parametric human body data as the ground truth to train the neural network model. The finally obtained model can identify and locate the acupoint positions on the global and / or local parametric human body data under a single perspective. During the training process of the neural network model, adjust the network parameters through the error backpropagation algorithm; when the neural network model is trained to the model convergence state, determine that the training of the neural network model is completed, and obtain the human acupoint localization network model.

[0050] Among them, the judgment of model training convergence is as follows: Change of loss function: Observing the change of loss values in the training set and the validation set is the most intuitive method to judge whether the model converges. If the training loss continues to decrease and the validation loss also decreases, it indicates that the model is still learning; if the training loss decreases while the validation loss stabilizes or begins to rise, it may indicate that the model begins to overfit; if both the training loss and the validation loss tend to be stable and the values of the two are not very different, it may indicate that the model has converged.

[0051] Training curve: By plotting the curves of training and validation losses changing with time (or the number of iterations), the convergence situation of the model can be judged more intuitively. If the curves tend to be stable, it usually means that the model has converged.

[0052] Overfitting and underfitting: In the case of overfitting, the model performs well on the training set but poorly on the validation set; in the case of underfitting, the loss of the model on the training set is very high, and the loss value is still very high at the end of training, and the training loss and the validation loss fluctuate greatly.

[0053] Performance on the validation set: If the performance of the model on the validation set (such as accuracy, F1 score, etc.) no longer improves, or the improvement is very slow, it can also be used as a signal of model convergence.

[0054] Learning rate adjustment: If the learning rate has been adjusted to be very small, but the performance of the model still does not improve significantly, this may mean that the model is close to or has reached convergence.

[0055] Gradient vanishing or explosion: If the gradient of the model is very small (close to 0), it may lead to gradient vanishing, and the model updates very slowly, seemingly "converging" but actually not really learning; if the gradient is very large, it may lead to gradient explosion, the model parameter updates are too large, and the loss value may increase explosively.

[0056] Based on the above points, it is possible to determine whether the model has converged by observing the trend of the loss value, the performance on the validation set, and the magnitude of the gradient. Generally, when both the training loss and the validation loss tend to stabilize, and the performance of the model on the validation set no longer improves significantly, it can be considered that the model has converged.

[0057] The model has the ability to deform the obtained global and / or local single-view human body data onto a standard human body model.

[0058] In the embodiment of the present invention, by utilizing the structural characteristics of the parametric human body model, real-time searching and display of acupoints and meridians of a target object with any posture and body shape at any viewing angle can be achieved only through one-time annotation of acupoints all over the body, solving the problem of inaccurate searching by only using rigid stretching and scaling transformation and the dependence on a large amount of annotated data in traditional neural networks; by using a mixed reality device to collect human body data in real time and display the meridians and acupoints on the human body, the problem that traditional acupoint searching methods cannot follow the movement is avoided. This method for locating human body meridians and acupoints can help people with learning needs for meridians and acupoints and related practitioners reduce the learning curve, and find the positions of acupoints and meridians more accurately, quickly, and intuitively, so as to learn the meridian and acupoint system more accurately and implement traditional Chinese medicine treatments such as acupuncture and massage more precisely and effectively.

[0059] The human body meridian and acupoint locating device provided by the present invention will be described below. The human body meridian and acupoint locating device described below can be mutually referred to with the body meridian and acupoint locating method described above.

[0060] Figure 7 is a schematic structural diagram of the human body meridian and acupoint locating device provided by the present invention, specifically including: An acquisition module 701, configured to acquire single-view global and / or local target human body data of a target object. For detailed description, refer to the relevant description corresponding to the above method embodiment, which will not be elaborated here.

[0061] A marking module 702, configured to input the target human body data into a pre-trained human acupoint locating network model, and perform human acupoint marking on the target human body data through the human acupoint locating network model to obtain a human acupoint marking result. For detailed description, refer to the relevant description corresponding to the above method embodiment, which will not be elaborated here.

[0062] A positioning module 703, configured to perform human body meridian and acupoint positioning on the target object based on the human acupoint marking result and the human meridian direction. For detailed description, refer to the relevant description corresponding to the above method embodiment, which will not be elaborated here.

[0063] The model training module 704 is used to construct a first parametric human body model in a standard pose, refine the mesh of the first parametric human body model to generate a standard three-dimensional human body model, where the number of surface points of the standard three-dimensional human body model is greater than that of the parametric human body model; mark acupoint points on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human body acupoint model; obtain global and / or local parametric human body data under multiple single viewpoints, and generate a three-dimensional human body acupoint model corresponding to the global and / or local parametric human body data under each single viewpoint based on the standard three-dimensional human body acupoint model; train a neural network model based on the global and / or local parametric human body data under the multiple single viewpoints and the three-dimensional human body acupoint model corresponding to the global and / or local parametric human body data under each single viewpoint to obtain a human body acupoint positioning network model. For the detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be elaborated here.

[0064] Figure 8 An example of the physical structure diagram of an electronic device is shown as Figure 8 shown. The electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the human meridian acupoint positioning method, which includes: collecting global and / or local target human body data of a target object from a single viewpoint; inputting the target human body data into a pre-trained human body acupoint positioning network model, and performing human body acupoint annotation on the target human body data through the human body acupoint positioning network model to obtain a human body acupoint annotation result; performing human meridian acupoint positioning on the target object based on the human body acupoint annotation result and the human meridian trend; where the human body acupoint positioning network model is trained based on the following steps: constructing a first parametric human body model in a standard pose, refining the mesh of the first parametric human body model to generate a standard three-dimensional human body model, where the number of surface points of the standard three-dimensional human body model is greater than that of the parametric human body model; marking acupoint points on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human body acupoint model; obtaining global and / or local parametric human body data under multiple single viewpoints, and generating a three-dimensional human body acupoint model corresponding to the global and / or local parametric human body data under each single viewpoint based on the standard three-dimensional human body acupoint model; training a neural network model based on the global and / or local parametric human body data under the multiple single viewpoints and the three-dimensional human body acupoint model corresponding to the global and / or local parametric human body data under each single viewpoint to obtain a human body acupoint positioning network model.

[0065] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0066] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the human meridian acupoint positioning method provided by the above-mentioned various methods. The method includes: collecting single-view global and / or local target human body data of a target object; inputting the target human body data into a pre-trained human acupoint positioning network model, and performing human acupoint annotation on the target human body data through the human acupoint positioning network model to obtain a human acupoint annotation result; performing human meridian acupoint positioning on the target object based on the human acupoint annotation result and the human meridian trend; wherein, the human acupoint positioning network model is trained based on the following steps: constructing a first parameterized human body model in a standard posture, and performing mesh refinement on the first parameterized human body model to generate a standard three-dimensional human body model, and the number of surface points of the standard three-dimensional human body model is greater than the number of surface points of the parameterized human body model; annotating acupoint points on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human acupoint model; obtaining a plurality of single-view global and / or local parameterized human body data, and generating a three-dimensional human acupoint model corresponding to each single-view global and / or local parameterized human body data based on the standard three-dimensional human acupoint model; training a neural network model based on the plurality of single-view global and / or local parameterized human body data and the three-dimensional human acupoint models corresponding to each single-view global and / or local parameterized human body data to obtain a human acupoint positioning network model.

[0067] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the human meridian acupoint positioning method provided by the above-mentioned various methods. The method includes: collecting single-view global and / or local target human body data of a target object; inputting the target human body data into a pre-trained human acupoint positioning network model, and performing human acupoint annotation on the target human body data through the human acupoint positioning network model to obtain a human acupoint annotation result; performing human meridian acupoint positioning on the target object based on the human acupoint annotation result and the human meridian direction; wherein, the human acupoint positioning network model is trained based on the following steps: constructing a first parameterized human body model in a standard posture, and performing mesh refinement on the first parameterized human body model to generate a standard three-dimensional human body model, the number of surface points of the standard three-dimensional human body model being greater than the number of surface points of the parameterized human body model; annotating acupoint points on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human acupoint model; obtaining a plurality of single-view global and / or local parameterized human body data, and generating a three-dimensional human acupoint model corresponding to each single-view global and / or local parameterized human body data based on the standard three-dimensional human acupoint model; training a neural network model based on the plurality of single-view global and / or local parameterized human body data and the three-dimensional human acupoint models corresponding to each single-view global and / or local parameterized human body data to obtain a human acupoint positioning network model.

[0068] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0069] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, also by hardware. Based on such an understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for locating acupoints of human meridians, characterized in that: include: Collecting single-view global and / or local target human body data of the target object; Inputting the target human body data into a pre-trained human acupoint positioning network model, and performing human acupoint labeling on the target human body data through the human acupoint positioning network model to obtain human acupoint labeling results; Based on the human acupoint labeling results and the direction of human meridians, the human meridian acupoints of the target object are located; The human body acupoint positioning network model is trained based on the following steps: Constructing a first parameterized human body model of a standard posture, and performing mesh refinement on the first parameterized human body model to generate a standard three-dimensional human body model, wherein the number of surface points of the standard three-dimensional human body model is greater than the number of surface points of the parameterized human body model; Marking acupoints on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human body acupoint model; Acquire global and / or local parameterized human body data under multiple single viewing angles, and generate a three-dimensional human body acupoint model corresponding to the global and / or local parameterized human body data under each single viewing angle based on the standard three-dimensional human body acupoint model; The neural network model is trained based on the global and / or local parameterized human body data under the multiple single-view angles and the three-dimensional human acupoint model corresponding to the global and / or local parameterized human body data under each single-view angle to obtain a human acupoint positioning network model.

2. The method according to claim 1, characterized in that The positioning of the human meridian acupoints of the target object based on the human acupoint marking results and the direction of the human meridians includes: Connecting two adjacent acupoints based on the human acupoint labeling results and the directions of human meridians to obtain multiple line segments between the acupoints; Locating a plurality of positioning points on each acupoint line segment, and searching for the nearest neighbor point of each positioning point on the target human body data by a nearest neighbor search method; Based on different meridians, corresponding colors are set for the nearest neighbor points of each positioning point to obtain the human meridian acupoint positioning corresponding to the target object.

3. The method according to claim 1, characterized in that The method of acquiring global and / or local parameterized human body data under multiple single viewing angles, and generating a three-dimensional human body acupoint model corresponding to the global and / or local parameterized human body data under each single viewing angle based on the standard three-dimensional human body acupoint model, comprises: Construct a second parametric human model of different postures and body shapes; Performing mesh refinement on the second parameterized human body model according to the same standard as the standard three-dimensional human acupoint model to generate a three-dimensional human body model corresponding to the second parameterized human body model; Acupoints are annotated on the three-dimensional human body model using the acupoint annotation data in the standard three-dimensional human body acupoint model to obtain a three-dimensional human body acupoint model corresponding to the three-dimensional human body model; The position of the virtual camera is randomly set, and the global and / or local human body data of the three-dimensional human body model under multiple single-view angles, as well as the three-dimensional human acupoint model corresponding to the global and / or local human body data under each single-view angle, are obtained through the virtual camera.

4. The method according to claim 3, characterized in that The training of the neural network model based on the global and / or local parameterized human body data under the multiple single-view angles and the three-dimensional human acupoint model corresponding to the global and / or local parameterized human body data under each single-view angle to obtain the human acupoint positioning network model includes: Inputting the global and / or local parameterized human body data under the multiple single-view angles and the three-dimensional human acupoint model corresponding to the global and / or local parameterized human body data under each single-view angle into the neural network model, using the global and / or local parameterized human body data under the multiple single-view angles as model input data, and using the three-dimensional human acupoint model corresponding to the global and / or local parameterized human body data under each single-view angle as the true value to train the neural network model; During the training of the neural network model, adjusting the network parameters by using an error back propagation algorithm; When the neural network model is trained to a model convergence state, it is determined that the training of the neural network model is completed, and a human acupoint positioning network model is obtained.

5. The method according to any one of claims 1 to 4, characterized in that: The step of marking acupoints on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human body acupoint model comprises: Acupoints are marked on the surface vertices of the standard three-dimensional human body model based on the specified standard human acupoint surface positioning method and meridian acupoint names to construct a standard three-dimensional human acupoint model.

6. The method according to claim 2, characterized in that The method further comprises: The human body meridian acupoint positioning result corresponding to the target object is displayed on an external display.

7. The method according to claim 1, characterized in that The collecting of single-view global and / or local target human body data of the target object includes: The single-view global and / or local target human body data of the target object is collected through the camera on the mixed reality device.

8. A human meridian acupoint positioning device, characterized in that: include: An acquisition module, used for acquiring single-view global and / or local target human body data of a target object; A labeling module, used for inputting the target human body data into a pre-trained human acupoint positioning network model, and labeling the target human body data with human acupoints through the human acupoint positioning network model to obtain human acupoint labeling results; A positioning module, used for positioning the human meridian acupoints of the target object based on the human acupoint marking results and the direction of the human meridians; A model training module is used to construct a first parameterized human body model of a standard posture, and to perform mesh refinement on the first parameterized human body model to generate a standard three-dimensional human body model, wherein the number of surface points of the standard three-dimensional human body model is greater than the number of surface points of the parameterized human body model; to mark acupoints on the body surface vertices of the standard three-dimensional human body model to construct a standard three-dimensional human body acupoint model; to obtain global and / or local parameterized human body data under multiple single perspectives, and to generate a three-dimensional human body acupoint model corresponding to the global and / or local parameterized human body data under each single perspective based on the standard three-dimensional human body acupoint model; to train a neural network model based on the global and / or local parameterized human body data under multiple single perspectives and the three-dimensional human body acupoint model corresponding to the global and / or local parameterized human body data under each single perspective to obtain a human body acupoint positioning network model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method for locating acupoints of the human meridians as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for locating acupoints of the human meridians as claimed in any one of claims 1 to 7 is implemented.

Images