Methods, devices, electronic equipment and storage media for acupoint location

Abstract

The application provides an acupoint determination method and device, electronic equipment and storage medium, and relates to the technical field of data processing. The method comprises the following steps: acquiring point cloud data of a to-be-tested person; inputting the point cloud data of the to-be-tested person into a three-dimensional human body construction model to obtain a three-dimensional to-be-tested human body model with semantic information corresponding to the to-be-tested person output by the three-dimensional human body construction model; inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; and determining acupoints of the to-be-tested person based on the three-dimensional standard human body model with semantic information, at least one standard acupoint labeled in the standard human body and the three-dimensional to-be-tested human body model with semantic information. According to the acupoint determination method provided by the application, the labeled standard acupoints can be accurately matched to the three-dimensional human body of a patient, and accurate searching of acupoints is realized.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, electronic device, and storage medium for determining acupoints. Background Technology

[0002] Traditional Chinese medicine treatments such as acupuncture and massage rely on accurate acupoints to be effective. Currently, the method for determining acupoints involves inputting scanned 3D human point cloud data into a trained human model construction network to obtain a 3D human model of the subject. Then, multiple surface landmarks are marked in the 3D human model. By appropriately stretching or shrinking, the acupoints in the 3D meridian acupoint model are matched with the landmarks in the patient's 3D body surface structure model.

[0003] However, the human body is a non-rigid hinge structure with various structural attributes such as height, weight, and build. It is difficult to achieve a precise match between two models by stretching or shrinking them using only a few dozen surface markers, thus reducing the accuracy of acupoint location. Summary of the Invention

[0004] This invention provides a method, device, electronic device, and storage medium for determining acupoints, which solves the problem of inaccurate acupoint determination caused by matching the meridian acupoint model with the patient's three-dimensional body surface structure model through appropriate stretching or shrinking in the prior art. It enables the accurate matching of marked standard acupoints to the patient's three-dimensional human body, thereby improving the accuracy of acupoint search.

[0005] This invention provides a method for determining acupoints, comprising:

[0006] Obtain point cloud data of the test subject;

[0007] The point cloud data of the subject is input into a three-dimensional human body construction model to obtain a three-dimensional human body model with semantic information corresponding to the subject output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on point cloud data of various human postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each position point in the human body model is located.

[0008] The standard point cloud data corresponding to the standard human body is input into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information output by the three-dimensional human body construction model.

[0009] Based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model of the subject with semantic information, the acupoints of the subject are determined.

[0010] According to a method for determining acupoints provided by the present invention, the method for determining the acupoints of the subject based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model to be tested with semantic information includes: determining the first spatial position corresponding to each of the standard acupoints in the standard human body; determining the first target key point closest to each of the first spatial positions on the three-dimensional standard human body model with semantic information; determining the first key point number corresponding to each of the first target key points in the three-dimensional standard human body model with semantic information; and determining the acupoints of the subject based on the first key point number and the three-dimensional human body model to be tested with semantic information.

[0011] According to a method for determining acupoints provided by the present invention, the step of determining the acupoints of the subject based on the first key point sequence number and the three-dimensional human body model with semantic information includes: determining the second spatial position of at least one target acupoint in the three-dimensional human body model with semantic information based on the first key point sequence number and the sequence number in the three-dimensional human body model with semantic information; determining the second target key point closest to each second spatial position in the point cloud data of the subject; and determining each second target key point as the acupoint of the subject.

[0012] According to a method for determining acupoints provided by the present invention, the three-dimensional human body construction model is trained in the following manner: acquiring complete point cloud data samples of complete parameterized human body models with different postures and / or different body types, and single-view point cloud data samples corresponding to the complete parameterized human body models; inputting the single-view point cloud data samples into an initial three-dimensional human body construction model to obtain a reconstructed three-dimensional human body model with semantic information and a first global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model; inputting the reconstructed three-dimensional human body model with semantic information into the initial three-dimensional human body construction model to obtain a second global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model; determining a first loss value between the first global feature and the second global feature based on global feature constraint loss; determining a second loss value between the reconstructed three-dimensional human body model with semantic information and the complete point cloud data samples based on reconstruction loss; determining the total loss value of the initial three-dimensional human body construction model based on the first loss value and the second loss value; and continuously training the initial three-dimensional human body construction model based on the total loss value until the total loss value converges to obtain the three-dimensional human body construction model.

[0013] According to a method for determining acupoints provided by the present invention, before obtaining complete point cloud data samples of complete parameterized human models with different postures and / or different body shapes, the method further includes: obtaining at least one human parameter; the human parameter includes posture parameters and body shape parameters; and determining the complete parameterized human model corresponding to each of the posture parameters and body shape parameters.

[0014] According to a method for determining acupoints provided by the present invention, obtaining the single-viewpoint cloud data sample includes: determining the viewing range of the single viewpoint; and determining the single-viewpoint cloud data sample corresponding to the complete point cloud data sample within the viewing range based on the viewing range and a hidden point removal algorithm.

[0015] According to a method for determining acupoints provided by the present invention, before inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information output by the three-dimensional human body construction model, the method further includes: obtaining the number of mesh divisions; if the number of mesh divisions is greater than a division threshold, determining the division degree as fine division, and performing fine division on the parameterized human body model with a standard pose based on the number of mesh divisions to obtain a standard model containing multiple key points; and based on the key points in the standard model, marking acupoints in the standard model to obtain the at least one standard acupoint.

[0016] The present invention also provides an acupoint determination device, comprising the following modules:

[0017] The acquisition module is used to acquire point cloud data of the subject.

[0018] The test human model construction module is used to input the point cloud data of the test subject into the three-dimensional human model construction model to obtain the three-dimensional test human model with semantic information output by the three-dimensional human model construction model; the three-dimensional human model construction model is trained based on point cloud data of various human postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each position point in the human model is located.

[0019] The standard human body model construction module is used to input the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model, and obtain the three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model.

[0020] The acupoint determination module is used to determine the acupoints of the subject based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model of the subject with semantic information.

[0021] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the acupoint determination method as described above.

[0022] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the acupoint determination method as described above.

[0023] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the acupoint determination method as described above.

[0024] The acupoint determination method, apparatus, electronic device, and storage medium provided by this invention acquire point cloud data of the subject; input the point cloud data of the subject into a 3D human body construction model to obtain a 3D human body model with semantic information corresponding to the subject output by the 3D human body construction model; input standard point cloud data corresponding to a standard human body into the 3D human body construction model to obtain a 3D standard human body model with semantic information corresponding to the standard human body output by the 3D human body construction model; and determine the acupoints of the subject based on the 3D standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the 3D human body model of the subject with semantic information. Thus, the 3D human body construction model can reconstruct 3D human body models with semantic information for various human postures and / or body types, improving the robustness and generalization of the model. Then, based on the 3D human body model of the subject with semantic information, the standard acupoints, and the 3D standard human body model with semantic information, the marked standard acupoints can be accurately matched to the patient's 3D human body, achieving accurate acupoint location. Furthermore, it considers various human postures and / or body types in actual situations, improving the practicality of the acupoint determination method. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is one of the flowcharts illustrating the acupoint determination method provided by the present invention.

[0027] Figure 2 This is a schematic diagram of the structure of a three-dimensional human body model with semantic information provided by the present invention.

[0028] Figure 3 This is a structural schematic diagram of the parametric human body models of various postures and body shapes provided by the present invention.

[0029] Figure 4A This is a schematic diagram of the structure of standard acupoints marked from the front view in the standard model provided by this invention.

[0030] Figure 4B This is a schematic diagram of the structure of standard acupoints marked from the rear view in the standard model provided by this invention.

[0031] Figure 5 This is the second flowchart illustrating the acupoint determination method provided by the present invention.

[0032] Figure 6 This invention provides an acupoint determination device.

[0033] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0035] The methods for determining acupoints disclosed in the prior art include: (1) performing three-dimensional reconstruction on a standard human body to construct a three-dimensional human body model; (2) creating an interactive sphere representing the acupoints, i.e., a three-dimensional meridian acupoint model, based on the acupuncture depth and range of each acupoint; (3) pre-marking multiple body surface landmarks based on the three-dimensional human body model, and matching the three-dimensional meridian acupoint model constructed in the three-dimensional human body model to the patient's three-dimensional body surface structure model by appropriately stretching or shrinking based on the multiple body surface landmarks; and (4) displaying the patient's three-dimensional meridian acupoint system structure in real time.

[0036] In addition, the method for constructing a three-dimensional human body model includes: (1) obtaining the single-view human body point cloud data of the test subject; (2) inputting the single-view human body point cloud data into the trained human body model construction network to obtain the three-dimensional human body model of the test subject.

[0037] In existing technologies, both standard human and patient human 3D data are obtained through high-precision 3D scanners, which are not only expensive but also inconvenient to use. The constructed 3D models of both standard and patient humans are ordinary 3D mesh models without semantic information. When matching the 3D meridian and acupoint model built on the standard human body to the patient's 3D body surface structure model, multiple surface markers (41 markers are given in this example) need to be pre-marked on the 3D human body model to complete the matching. The specific matching process is as follows: based on the differences between the patient's 3D body surface structure model and the virtual model, the meridian and acupoint mixed reality interactive system is used to appropriately stretch or shrink the model; the 3D meridian and acupoint model already constructed in the virtual model is then precisely matched with the patient's 3D body surface structure model to ensure that the meridian and acupoint model accurately covers the corresponding areas on the patient's 3D body surface model. The human body is a non-rigid hinge structure with various structural attributes such as height, weight, and build. It is difficult to achieve a precise match between two models by stretching or shrinking them using only a few dozen surface markers, thus reducing the accuracy of acupoint location.

[0038] Furthermore, existing technologies only describe the construction of 3D human body models from single-view point cloud data. The human body model construction network trained cannot robustly process both incomplete and complete single-view human body point clouds simultaneously, resulting in inaccurate acupoint determination.

[0039] To address the aforementioned problems, this invention provides a method for acupoint determination. By constructing a three-dimensional human body model, various human postures and / or body types with semantic information can be reconstructed, improving the robustness and generalization of the model. Then, based on the semantically information-rich three-dimensional human body model to be tested, standard acupoints, and a semantically information-rich three-dimensional standard human body model, the labeled standard acupoints can be accurately matched to the patient's three-dimensional human body, achieving accurate acupoint location. Furthermore, it considers various human postures and / or body types in real-world situations, enhancing the practicality of the acupoint determination method.

[0040] The following is combined with Figures 1-5 The present invention describes an acupoint determination method applicable to any human body. The execution subject of this method can be an electronic device or an acupoint determination method installed in the electronic device. The acupoint determination device can be implemented by software, hardware, or a combination of both.

[0041] Figure 1 This is one of the flowcharts illustrating the acupoint determination method provided by the present invention, such as... Figure 1 As shown, the method includes the following:

[0042] Step 101: Obtain the point cloud data of the subject.

[0043] Here, the subject can be in any posture or of any body shape, and the point cloud data can be complete point cloud data or single-view point cloud data. This invention does not limit this.

[0044] The point cloud data of the test subject can be point cloud data collected by a scanner; it can also be RGB images and depth images obtained by a Red Green BlueDepth (RGBD) camera and then converted into point cloud data; or it can be ordinary images obtained by an ordinary camera, and the corresponding parametric human body model created by the ordinary images, and the data collected from the parametric human body model.

[0045] Step 102: Input the point cloud data of the subject into the three-dimensional human body construction model to obtain the three-dimensional human body model with semantic information corresponding to the subject output by the three-dimensional human body construction model.

[0046] The three-dimensional human body construction model is trained based on point cloud data of various human body postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each location point in the human body model is located.

[0047] Here, different semantic information represents different human body regions. Each point cloud data corresponds to a predefined semantic information, such as the 1000th point representing the earlobe and the 500th point representing the navel.

[0048] It should be noted that the acquired point cloud data is disordered, while the three-dimensional human body model under test with semantic information is ordered, and each point has corresponding information.

[0049] For example, Figure 2 This is a schematic diagram of the structure of the three-dimensional human body model with semantic information provided by the present invention, such as... Figure 2 As shown, different colored areas represent different semantic information, and different semantic information represents different human body areas. For example, the first 100 points represent the left leg area, which is represented by purple. The 500th to 600th points represent the head area, which is represented by green.

[0050] Figure 3 This is a structural schematic diagram of the parametric human body models of various postures and body shapes provided by the present invention, such as... Figure 3 As shown, point cloud data of parametric human body models with various poses and body shapes are collected as training datasets to train the initial model and obtain a 3D human body construction model, thereby improving the robustness and generalization ability of the 3D human body construction model.

[0051] Step 103: Input the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information output by the three-dimensional human body construction model.

[0052] Here, a standard human body refers to a body where all parts are fully displayed without any obstructions, such as not having a hand covering the abdomen.

[0053] Step 104: Based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model to be tested with semantic information, determine the acupoints of the subject to be tested.

[0054] Here, the acupoints of the test subject can be determined by any suitable method. For example, the acupoints of the test subject can be obtained by mapping the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model of the test subject with semantic information. Alternatively, the acupoints of the test subject can be obtained by inputting the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model of the test subject with semantic information into a neural network model.

[0055] In one example, determining the acupoints of the test subject based on the semantically information-based three-dimensional standard human body model, at least one standard acupoint marked in the standard human body, and the semantically information-based three-dimensional test human body model includes: determining the first spatial position corresponding to each of the standard acupoints in the standard human body; determining the first target key point closest to each of the first spatial positions on the semantically information-based three-dimensional standard human body model; determining the first key point number corresponding to each of the first target key points in the semantically information-based three-dimensional standard human body model; and determining the acupoints of the test subject based on the first key point number and the semantically information-based three-dimensional test human body model.

[0056] Here, spatial location refers to the three-dimensional coordinate position (x, y, z), and the first key point number is the number of that position in the vertices of the three-dimensional standard human body model with semantic information.

[0057] It should be noted that a three-dimensional standard human body model with semantic information is reconstructed based on the point cloud data in the parametric standard human body model. The three-dimensional standard human body model corresponds almost exactly to the parametric standard human body model. The spatial positions of standard acupoints in the three-dimensional standard human body model are almost identical to their spatial positions in the parametric standard human body model. Therefore, the first target key point that is closest to the first spatial position is the acupoint on the three-dimensional standard human body model with semantic information. Then, the first key point number of each first target key point in the three-dimensional standard human body model with semantic information is determined. Based on the first key point number and the three-dimensional human body model to be tested with semantic information, the acupoints of the subject are determined.

[0058] In this embodiment of the invention, by reconstructing the correspondence between the three-dimensional human body model to be tested with semantic information and the three-dimensional standard human body model, the marked standard acupoint locations can be accurately matched to the three-dimensional human body of the subject, thereby improving the accuracy of acupoint location.

[0059] For example, determining the acupoints of the subject based on the first key point number and the three-dimensional human body model with semantic information includes: determining the second spatial location of at least one target acupoint in the three-dimensional human body model with semantic information based on the first key point number and the number in the model; determining the second target key point closest to each second spatial location in the point cloud data of the subject; and determining each second target key point as the acupoint of the subject.

[0060] It should be noted that the index of each point in the semantically information-rich 3D human body model reconstructed from the 3D human body reconstruction model, and the semantic information corresponding to each index, can be the same. For example, if the 100th point in the semantically information-rich 3D human body model represents the nose, then the 100th point in the semantically information-rich 3D standard human body model also represents the nose. Therefore, the index of the first key point can be mapped to the index in the semantically information-rich 3D human body model to obtain the acupoints in the semantically information-rich 3D human body model. Then, since the point cloud data of the subject lacks semantic information, it is impossible to find the acupoints of the subject based on the index. Instead, the acupoints of the subject can be determined based on spatial location and nearest neighbor search. In other words, the point in the subject's point cloud data that is closest to the second spatial location is the acupoint of the subject.

[0061] In this embodiment of the invention, by reconstructing the correspondence between the three-dimensional human body model to be tested with semantic information and the three-dimensional standard human body model, the marked standard acupoint locations can be accurately matched to the three-dimensional human body of the subject, thereby improving the accuracy of acupoint location.

[0062] In this embodiment of the invention, point cloud data of the test subject is acquired; the point cloud data of the test subject is input into a 3D human body construction model to obtain a 3D human body model with semantic information corresponding to the test subject output by the 3D human body construction model; standard point cloud data corresponding to a standard human body is input into the 3D human body construction model to obtain a 3D standard human body model with semantic information corresponding to the standard human body output by the 3D human body construction model; based on at least one standard acupoint marked in the 3D standard human body model with semantic information and the 3D human body model of the test subject with semantic information, the acupoints of the test subject are determined. Thus, the 3D human body construction model can reconstruct 3D human body models with semantic information for various human postures and / or body types, improving the robustness and generalization of the model. Then, based on the 3D human body model of the test subject with semantic information, the standard acupoints, and the 3D standard human body model with semantic information, the marked standard acupoints can be accurately matched to the patient's 3D human body, achieving accurate acupoint location. Furthermore, it considers various human postures and / or body types in actual situations, improving the practicality of the acupoint determination method.

[0063] The training process of the 3D human body construction model is described below.

[0064] The 3D human body construction model is trained as follows: Complete point cloud data samples of complete parameterized human body models with different poses and / or different body shapes are obtained, along with single-view point cloud data samples corresponding to the complete parameterized human body models. The single-view point cloud data samples are input into an initial 3D human body construction model to obtain a reconstructed 3D human body model with semantic information and a first global feature extracted by the feature extraction network in the initial 3D human body construction model. The reconstructed 3D human body model with semantic information is input into the initial 3D human body construction model to obtain a second global feature extracted by the feature extraction network in the initial 3D human body construction model. A first loss value is determined between the first global feature and the second global feature based on global feature constraint loss. A second loss value is determined between the reconstructed 3D human body model with semantic information and the complete point cloud data samples based on reconstruction loss. The total loss value of the initial 3D human body construction model is determined based on the first loss value and the second loss value. The initial 3D human body construction model is continuously trained based on the total loss value until the total loss value converges, thus obtaining the 3D human body construction model.

[0065] Here, methods for obtaining a complete parameterized human body model of a sample include, but are not limited to, the SCAPE model (digital parametric human body model) and the Skinned Multi-Person Linear Model (SMPL).

[0066] Here, the global feature constraint loss is used to represent the difference between the global features corresponding to the complete point cloud data sample of the reconstructed 3D human body model with semantic information and the global features corresponding to the single-view point cloud data sample. The smaller the global feature constraint loss, the closer the extracted and reconstructed complete 3D human body model features are to the input single-view human body model features.

[0067] It should be noted that the global feature constraint loss can be determined by any suitable loss function, including but not limited to Mean Squared Error Loss (MSELoss), Mean Absolute Error (MAE), Huber Loss, etc.

[0068] Here, the reconstruction loss is used to represent the difference between the complete point cloud data of the reconstructed 3D human body model with semantic information and the complete point cloud data sample of the complete parameterized human body model. The smaller the reconstruction loss, the closer the reconstructed 3D human body model with semantic information is to the complete parameterized human body model sample.

[0069] It should be noted that the reconstruction loss can be determined by any suitable loss function, including but not limited to Mean Squared Error Loss (MSELoss), Mean Absolute Error (MAE), Huber Loss, etc.

[0070] Here, the total loss value can be the sum of the first loss value and the second loss value, or it can be a weighted sum of the first loss value and the second loss value. The weights of the first loss value and the second loss value can be adjusted according to the actual situation. This invention does not limit this.

[0071] It should be noted that the training of the 3D human body construction model is considered complete when the training criteria consisting of the global feature constraint loss and reconstruction loss meet the preset requirements.

[0072] It should be noted that the training process of the 3D human body construction model is a negative feedback process. By continuously calculating the difference between the global features corresponding to the complete point cloud data of the reconstructed 3D human body model with semantic information and the global features corresponding to the single-view point cloud data sample, as well as the difference between the complete point cloud data of the reconstructed 3D human body model with semantic information and the complete point cloud data sample of the complete parameterized human body model, the total loss value is obtained. The total loss value is fed back to the model to adjust the parameters and continue to determine the next total loss value until the total loss value converges, thus obtaining the 3D human body construction model.

[0073] In this embodiment of the invention, a large number of parameterized human body models with various postures and body shapes are used as training data to train a neural network model. Using this trained neural network model, the point cloud data of the collected subject and the point cloud data of the standardized human body model can be reconstructed into a three-dimensional human body model with semantic information. At the same time, various postures and body shapes improve the robustness and accuracy of the model and avoid overfitting. Furthermore, the three-dimensional human body construction model trained by global feature constraint loss and reconstruction loss can construct a human body model based on complete point cloud data or based on single-view point cloud data, thereby realizing the determination of acupoints from complete human body point cloud and single-view point cloud.

[0074] For example, obtaining the single-viewpoint cloud data sample includes: determining the view range of the single view; and based on the view range and the hidden point removal algorithm, determining the single-viewpoint cloud data sample corresponding to the complete point cloud data sample within the view range.

[0075] Here, in addition to determining the field of view, the viewpoint can also be determined. Collecting complete point cloud data from different field of view and different viewpoints improves the diversity of point cloud data and enhances the robustness and accuracy of the trained model.

[0076] Here, hidden points refer to points that are not visible within the field of view. Multiple single-view point cloud data are generated through a hidden point removal algorithm.

[0077] In this embodiment of the invention, by collecting complete point cloud data from different perspectives and viewpoints, the diversity of point cloud data is improved, thereby enhancing the robustness and accuracy of the training model.

[0078] Furthermore, before acquiring complete point cloud data samples of complete parametric human models with different poses and / or different body shapes, the method further includes:

[0079] Acquire at least one human body parameter; the human body parameter includes posture parameters and body shape parameters;

[0080] Based on the posture parameters and body shape parameters, a complete parameterized human body model corresponding to each human body parameter is determined.

[0081] Here, human body parameters can be obtained from public datasets or collected by acquisition devices.

[0082] Here, the method for determining the complete parameterized human body model can be any suitable method: for example, inputting the posture parameters and body shape parameters into the software to obtain the complete parameterized human body model; or, using a reconstruction algorithm to construct the complete parameterized human body model.

[0083] In this embodiment of the invention, a large number of parametric human body models with various postures and body shapes are used as training data, thus avoiding the collection of large-scale three-dimensional human body data.

[0084] Furthermore, before inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information output by the three-dimensional human body construction model, the method further includes:

[0085] Get the number of grid divisions;

[0086] If the number of mesh divisions is greater than the division threshold, the division degree is determined to be fine division, and the parametric human body model with standard pose is finely divided based on the number of mesh divisions to obtain a standard human body model; the standard human body model has more position points than the parametric human body model with standard pose.

[0087] Based on the location points in the standard human body model, acupoints are marked in the standard human body model to obtain at least one standard acupoint.

[0088] Here, a higher degree of fineness in the division indicates more grids and more location points; a lower degree of fineness indicates fewer grids and fewer location points.

[0089] It should be noted that, in order to ensure the accuracy of acupoint labeling in the later stages, the higher the level of detail in the classification, the better.

[0090] Figure 4A This is a schematic diagram of the structure of standard acupoints marked from the frontal view in the standard model provided by this invention, such as... Figure 4A As shown, the red dots indicate the marked standard acupoints.

[0091] Figure 4B This is a schematic diagram of the structure of standard acupoints annotated from a rear-view perspective in the standard model provided by this invention, such as... Figure 4B As shown, the red dots indicate the marked standard acupoints.

[0092] In this embodiment of the invention, annotations are directly applied to the refined standard parametric human body model without additional scanning. Acupoints for patients of any posture and body type can be determined by annotating all acupoints in one step, solving the problem of dependence on massive amounts of annotation data. Furthermore, the parametric human body model is refined multiple times to increase the number of location points and improve annotation accuracy, thereby enhancing the accuracy of acupoint determination.

[0093] Figure 5 This is the second flowchart illustrating the acupoint determination method provided by the present invention, as shown below. Figure 5 As shown, it includes:

[0094] Step 501: Refine the mesh of the parametric human body model in the standard pose to generate a standard model with denser position points.

[0095] Step 502: Mark the acupoints in the standard model to obtain at least one standard acupoint.

[0096] Step 503: Based on the complete point cloud data of the complete parameterized human body model of different postures and / or different body shapes and the single-view point cloud data corresponding to the complete parameterized human body model of the sample, continuously train the initial three-dimensional human body construction model until the difference value converges, and obtain the three-dimensional human body construction model.

[0097] Specifically, a large number of parametric human body models with various poses and body shapes are generated. Virtual camera positions are randomly set, and multiple single-view point cloud datasets are generated using a hidden point removal algorithm. These generated single-view point cloud datasets are used as input to an initial 3D human body construction model. This yields a reconstructed 3D human body model with semantic information and the first global features extracted by the feature extraction network in the initial 3D human body construction model. The reconstructed 3D human body model with semantic information is then used as input to the initial 3D human body construction model to obtain the second global features extracted by the feature extraction network in the initial 3D human body construction model. Based on global feature constraint loss, a first loss value is determined between the first and second global features. Based on reconstruction loss, a second loss value is determined between the complete point cloud data of the reconstructed 3D human body model with semantic information and the complete point cloud data samples of the complete parametric human body model. Based on the first and second loss values, a total loss value is determined. The model is continuously trained using error backpropagation until the total loss value converges, resulting in a 3D human body construction model. This 3D human body construction model can reconstruct a 3D human body model with semantic information from input complete / single-view point cloud data.

[0098] Step 504: Input the standard point cloud data corresponding to the standard human body into the 3D human body construction model to obtain the 3D standard human body model with semantic information output by the 3D human body construction model.

[0099] Step 505: Map the standard acupoints marked on the standard model to the reconstructed 3D standard human body model with semantic information through nearest neighbor search.

[0100] Step 506: Input the point cloud data of the subject into the 3D human body construction model to obtain the 3D human body model with semantic information corresponding to the subject output by the 3D human body construction model.

[0101] Step 507: Based on the semantic information correspondence, directly search for the key points of the three-dimensional human body model with semantic information according to the location point number to obtain the acupoints to be tested on the model.

[0102] Step 508: In the point cloud data of the test subject, determine the second target key point that is closest to each second spatial position to obtain the acupoints of the test subject.

[0103] The acupoint determination method provided by this invention has the following advantages: Utilizing the structured characteristics and semantic information of parametric human body models, acupoint determination can be achieved for patients of any posture and body type with only one annotation of all acupoints, solving the problem of dependence on massive amounts of labeled data; during neural network training, a large number of parametric human body models of various postures and body types are used as training data, avoiding the collection of large-scale three-dimensional human body data; the trained neural network reconstructs the input point cloud with semantic information and ultimately locates acupoints, achieving higher accuracy than simply matching the three-dimensional meridian acupoint model with the patient's three-dimensional body surface structure model through appropriate stretching or shrinking; it can simultaneously achieve acupoint search from single-view human point clouds and complete human point clouds.

[0104] The acupoint determination device provided by the present invention is described below. The acupoint determination device described below and the acupoint determination method described above can be referred to in correspondence.

[0105] Figure 6 The acupoint determination device provided by this invention, such as... Figure 6 As shown, the acupoint determination device 600 includes the following modules:

[0106] The acquisition module 601 is used to acquire the point cloud data of the subject.

[0107] The test human model construction module 602 is used to input the point cloud data of the test subject into the three-dimensional human model construction model to obtain the three-dimensional test human model with semantic information corresponding to the test subject output by the three-dimensional human model construction model; the three-dimensional human model construction model is trained based on point cloud data of various human postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each position point in the human model is located.

[0108] The standard human body model construction module 603 is used to input the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information output by the three-dimensional human body construction model.

[0109] The acupoint determination module 604 is used to determine the acupoints of the subject based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model to be tested with semantic information.

[0110] In one embodiment, the acupoint determination module 604 is specifically used for: determining the first spatial position corresponding to each of the standard acupoints in the standard human body; determining the first target key point closest to each of the first spatial positions on the three-dimensional standard human body model with semantic information; determining the first key point number corresponding to each of the first target key points in the three-dimensional standard human body model with semantic information; and determining the acupoints of the subject based on the first key point number and the three-dimensional human body model to be tested with semantic information.

[0111] In one embodiment, the acupoint determination module 604 is further specifically used to: determine the second spatial position of at least one target acupoint in the three-dimensional human body model with semantic information based on the first key point number and the number in the three-dimensional human body model with semantic information; determine the second target key point closest to each second spatial position in the point cloud data of the subject; and determine each second target key point as the acupoint of the subject.

[0112] In one embodiment, the acupoint determination device 600 further includes a training module, specifically configured to: acquire complete point cloud data samples of complete parameterized human body model samples with different postures and / or different body types, and single-view point cloud data samples corresponding to the complete parameterized human body model samples; input the single-view point cloud data samples into an initial three-dimensional human body construction model to obtain a reconstructed three-dimensional human body model with semantic information and a first global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model; input the reconstructed three-dimensional human body model with semantic information into the initial three-dimensional human body construction model to obtain a second global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model; determine a first loss value between the first global feature and the second global feature based on global feature constraint loss; determine a second loss value between the reconstructed three-dimensional human body model with semantic information and the complete point cloud data samples based on reconstruction loss; determine the total loss value of the initial three-dimensional human body construction model based on the first loss value and the second loss value; and continuously train the initial three-dimensional human body construction model based on the total loss value until the total loss value converges to obtain the three-dimensional human body construction model.

[0113] In one embodiment, before acquiring complete point cloud data samples of complete parameterized human models with different poses and / or different body shapes, the training module is further specifically used to: acquire at least one human parameter; the human parameter includes pose parameters and body shape parameters; and determine the complete parameterized human model corresponding to each of the pose parameters and body shape parameters.

[0114] In one embodiment, the training module is further configured to: determine the viewing range of a single viewpoint; and, based on the viewing range and the hidden point removal algorithm, determine the single-viewpoint cloud data sample corresponding to the complete point cloud data sample within the viewing range.

[0115] In one embodiment, before inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information output by the three-dimensional human body construction model, the acupoint determination device 600 further includes a division module, specifically used for: obtaining the number of mesh divisions; if the number of mesh divisions is greater than the division threshold, determining the division degree as fine division, and performing fine division on the parameterized human body model with a standard pose based on the number of mesh divisions to obtain a standard human body model; the standard human body model has more position points than the parameterized human body model with a standard pose; based on the position points in the standard human body model, marking acupoints in the standard human body model to obtain the at least one standard acupoint.

[0116] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other through the communications bus 740. The processor 710 can call logical instructions in the memory 730 to execute an acupoint determination method, which includes: acquiring point cloud data of a subject; inputting the point cloud data of the subject into a three-dimensional human body construction model to obtain a three-dimensional human body model with semantic information corresponding to the subject output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on point cloud data of various human postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each position point in the human body model is located; inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; and determining the acupoints of the subject based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model with semantic information.

[0117] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the acupoint determination method provided by the above methods. The method includes: acquiring point cloud data of a subject; inputting the point cloud data of the subject into a three-dimensional human body construction model to obtain a three-dimensional human body model with semantic information corresponding to the subject output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on point cloud data of various human postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each position point in the human body model is located; inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; and determining the acupoints of the subject based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model with semantic information.

[0119] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the acupoint determination method provided by the above methods. This method includes: acquiring point cloud data of a subject; inputting the point cloud data of the subject into a three-dimensional human body construction model to obtain a three-dimensional human body model with semantic information corresponding to the subject output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on point cloud data of various human postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each location point in the human body model is located; inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; and determining the acupoints of the subject based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model with semantic information.

[0120] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An acupoint determination method, characterized by comprising: The method comprises the following steps: acquiring point cloud data of a to-be-tested person; inputting the point cloud data of the to-be-tested person into a three-dimensional human body construction model to obtain a three-dimensional to-be-tested human body model with semantic information corresponding to the to-be-tested person output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on point cloud data of various human body postures and / or various human body shapes, and the semantic information is used to represent human body region information of each position point in the human body model; inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; determining first spatial positions corresponding to at least one standard acupoint in the standard human body; the standard acupoint is an acupoint marked in the standard human body; determining first target key points closest to each of the first spatial positions on the three-dimensional standard human body model with semantic information; determining first key point serial numbers corresponding to each of the first target key points in the three-dimensional standard human body model with semantic information; the first key point serial number is a serial number of the first spatial position in the vertex of the three-dimensional standard human body model with semantic information, and the first spatial position is a three-dimensional coordinate position; based on the first key point serial number and a serial number in the three-dimensional to-be-tested human body model with semantic information, determining second spatial positions of at least one to-be-tested target acupoint in the three-dimensional to-be-tested human body model with semantic information; determining second target key points closest to each of the second spatial positions in the point cloud data of the to-be-tested person; determining each of the second target key points as an acupoint of the to-be-tested person.

2. The acupoint determination method according to claim 1, characterized by The three-dimensional human body construction model is obtained based on the following method: acquiring complete point cloud data samples of complete parameterized human body model samples in different postures and / or different shapes and single-view point cloud data samples corresponding to the complete parameterized human body model samples; inputting the single-view point cloud data samples into an initial three-dimensional human body construction model to obtain a reconstructed three-dimensional human body model with semantic information and first global features extracted by a feature extraction part network in the initial three-dimensional human body construction model; inputting the reconstructed three-dimensional human body model with semantic information into the initial three-dimensional human body construction model to obtain second global features extracted by the feature extraction part network in the initial three-dimensional human body construction model; determining a first loss value between the first global features and the second global features based on a global feature constraint loss; determining a second loss value between the reconstructed three-dimensional human body model with semantic information and the complete point cloud data samples based on a reconstruction loss; determining a total loss value of the initial three-dimensional human body construction model based on the first loss value and the second loss value; continuously training the initial three-dimensional human body construction model based on the total loss value until the total loss value converges, to obtain the three-dimensional human body construction model.

3. The acupoint determination method according to claim 2, characterized by Before the step of acquiring the complete point cloud data samples of the complete parameterized human body model in different postures and / or different shapes, the method further comprises the following steps: Obtaining at least one human body parameter; the human body parameter includes a posture parameter and a body shape parameter; Based on each of the posture parameters and each of the body shape parameters, determine the complete parameterized human body model corresponding to each of the human body parameters.

4. The acupoint determination method according to claim 2, characterized by, Obtaining the single-view point cloud data sample includes: Determine the view range of the single view; Based on the view range and the hidden point removal algorithm, determine the single-view point cloud data sample corresponding to the complete point cloud data sample under the view range.

5. The acupoint determination method according to claim 1, characterized by, Before the method further includes: Obtaining the number of grid divisions; In the case where the number of grid divisions is greater than the division threshold, the division degree is determined to be fine division, and the parameterized human body model with standard posture is finely divided based on the number of grid divisions to obtain a standard human body model; the position points of the standard human body model are more than those in the parameterized human body model with standard posture; Based on the position points in the standard human body model, mark the acupoints in the standard human body model to obtain at least one standard acupoint.

6. An acupoint determination device, characterized by comprising: Including: An acquisition module is configured to obtain point cloud data of a subject; A subject human body model construction module is configured to input the point cloud data of the subject into a three-dimensional human body construction model to obtain a three-dimensional subject human body model with semantic information corresponding to the subject output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on point cloud data of various human body postures and / or various human body shapes, and the semantic information is used to represent human body region information of each position point in the human body model; A standard human body model construction module is configured to input standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; An acupoint determination module is configured to determine at least one first spatial position corresponding to a standard acupoint in the standard human body; The standard acupoint is an acupoint marked in the standard human body; On the three-dimensional standard human body model with semantic information, determine a first target key point closest to each of the first spatial positions respectively; Determine a first key point sequence number of each of the first target key points in the three-dimensional standard human body model with semantic information; The first key point sequence number is the sequence number of the first spatial position in the vertex of the three-dimensional standard human body model with semantic information, and the first spatial position is a three-dimensional coordinate position; Based on the first key point sequence number and the sequence number in the three-dimensional subject human body model with semantic information, determine a second spatial position of at least one subject target acupoint in the three-dimensional subject human body model with semantic information; In the point cloud data of the subject, determine a second target key point closest to each of the second spatial positions respectively; Determine each of the second target key points as the acupoint of the subject.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The computer program is executed by the processor to implement the acupoint determination method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the acupoint determination method according to any one of claims 1 to 5.

Images