Acupuncture point determination method and device, electronic equipment and storage medium

Abstract

The invention provides an acupuncture point determination method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining point cloud data of a to-be-tested person; inputting the point cloud data of the to-be-tested person into the three-dimensional human body construction model to obtain a three-dimensional to-be-tested human body model which is output by the three-dimensional human body construction model, corresponds to the to-be-tested person and has semantic information; inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model which is output by the three-dimensional human body construction model, corresponds to the standard human body and has semantic information; based on the three-dimensional standard human body model with the semantic information, the at least one standard acupuncture point marked in the standard human body and the three-dimensional human body model to be detected with the semantic information, the acupuncture points of the person to be detected are determined. According to the acupuncture point determination method provided by the invention, the marked standard acupuncture points can be accurately matched to the three-dimensional human body of the patient, and accurate searching of the acupuncture points is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a method and device for determining acupoints, an electronic device, and a storage medium. Background Art

[0002] Traditional Chinese medicine treatment means such as acupuncture and massage need to rely on accurate acupoints to be effectively implemented. At present, the acupoint determination method is to input the scanned three-dimensional human point cloud data into the trained human model construction network to obtain the three-dimensional human model of the person to be measured, and then mark multiple body surface landmark points in the three-dimensional human model. Through appropriate stretching or shrinking, the acupoints in the three-dimensional stereoscopic meridian and acupoint model are matched with the landmark points of the three-dimensional stereoscopic body surface structure model of the patient.

[0003] However, the human body is a non-rigid hinge structure with various structural attributes such as height, weight, and body type. It is difficult to obtain an accurate match between the two models only by stretching or shrinking between the two models through dozens of body surface landmark points, thereby reducing the accuracy of acupoint determination. Summary of the Invention

[0004] The present invention provides a method and device for determining acupoints, an electronic device, and a storage medium, which are used to solve the defect in the prior art that the matching between the meridian and acupoint model and the three-dimensional stereoscopic body surface structure model of the patient is achieved through appropriate stretching or shrinking, resulting in inaccurate acupoint determination, and to realize the accurate matching of the marked standard acupoints to the three-dimensional human body of the patient, thereby improving the accuracy of acupoint search.

[0005] The present invention provides a method for determining acupoints, including: Obtaining the point cloud data of the person to be measured; Inputting the point cloud data of the person to be measured into the three-dimensional human construction model to obtain the three-dimensional to-be-measured human model with semantic information corresponding to the person to be measured output by the three-dimensional human construction model; the three-dimensional human construction model is trained based on the point cloud data of various human postures and / or various human body types, and the semantic information is used to characterize the human region information where each position point in the human model is located; Inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human construction model to obtain the three-dimensional standard human model with semantic information corresponding to the standard human body output by the three-dimensional human construction model; Based on the three-dimensional standard human model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional to-be-measured human model with semantic information, determining the acupoints of the person to be measured.

[0006] A method for determining acupoints provided by the present invention, which determines the acupoints of the person to be measured based on the three-dimensional standard human model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human model to be measured with semantic information, includes: determining the first spatial positions corresponding to each of the standard acupoints in the standard human body; respectively determining the first target key points closest to each of the first spatial positions on the three-dimensional standard human model with semantic information; determining the first key point numbers corresponding to each of the first target key points in the three-dimensional standard human model with semantic information; and determining the acupoints of the person to be measured based on the first key point numbers and the three-dimensional human model to be measured with semantic information.

[0007] A method for determining acupoints provided by the present invention, which determines the acupoints of the person to be measured based on the first key point numbers and the three-dimensional human model to be measured with semantic information, includes: determining the second spatial positions of at least one acupoint to be measured in the three-dimensional human model to be measured with semantic information based on the first key point numbers and the numbers in the three-dimensional human model to be measured with semantic information; respectively determining the second target key points closest to each of the second spatial positions in the point cloud data of the person to be measured; and determining each of the second target key points as the acupoints of the person to be measured.

[0008] A three-dimensional human body construction model provided by the present invention is trained based on the following method, including: obtaining the complete point cloud data samples of complete parametric human body model samples with different postures and / or different body types and the single-viewpoint point cloud data samples corresponding to the complete parametric human body model samples; inputting the single-viewpoint point cloud data samples into an initial three-dimensional human body construction model to obtain a reconstructed three-dimensional human model with semantic information and the first global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model; inputting the reconstructed three-dimensional human model with semantic information into the initial three-dimensional human body construction model to obtain the second global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model; determining the first loss value between the first global feature and the second global feature based on the global feature constraint loss; determining the second loss value between the reconstructed three-dimensional human model with semantic information and the complete point cloud data samples based on the reconstruction loss; determining the total loss value of the initial three-dimensional human body construction model based on the first loss value and the second loss value; and continuously training the initial three-dimensional human body construction model based on the total loss value until the total loss value converges to obtain the three-dimensional human body construction model.

[0009] A method for determining acupoints provided by the present invention further includes, before obtaining the complete point cloud data sample of a complete parametric human body model with different postures and / or different body types: obtaining at least one human parameter; the human parameter includes a posture parameter and a body type parameter; based on each of the posture parameters and each of the body type parameters, determining a complete parametric human body model corresponding to each of the human parameters.

[0010] A method for determining acupoints provided by the present invention, obtaining the single-viewpoint point cloud data sample includes: determining the viewing range of a single view; based on the viewing range and a hidden point removal algorithm, determining the single-viewpoint point cloud data sample corresponding to the complete point cloud data sample within the viewing range.

[0011] A method for determining acupoints provided by the present invention further includes, before inputting the standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model: obtaining the number of mesh divisions; in the case where the number of mesh divisions is greater than a division threshold, determining that the division degree is a fine division, and performing a fine division on the parametric human body model with a standard posture based on the number of mesh divisions to obtain a standard model including a plurality of key points; based on the key points in the standard model, marking acupoints in the standard model to obtain the at least one standard acupoint.

[0012] The present invention also provides an acupoint determination device, including the following modules: An acquisition module, configured to acquire the point cloud data of a person to be measured; A to-be-measured human body model construction module, configured to input the point cloud data of the person to be measured into a three-dimensional human body construction model to obtain a three-dimensional to-be-measured human body model with semantic information corresponding to the person to be measured output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on the point cloud data of various human postures and / or various human body types, and the semantic information is used to represent the human body region information where each position point in the human body model is located; A standard human body model construction module, configured to input the standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; An acupoint determination module, configured to determine the acupoints of the person to be measured based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional to-be-measured human body model with semantic information.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the acupoint determination method as described in any one of the above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the acupoint determination method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the acupoint determination method as described in any one of the above is implemented.

[0016] The acupoint determination method, device, electronic device, and storage medium provided by the present invention obtain the point cloud data of the person to be measured; input the point cloud data of the person to be measured into a three-dimensional human body construction model to obtain a three-dimensional human body model with semantic information corresponding to the person to be measured output by the three-dimensional human body construction model; input the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; and determine the acupoints of the person to be measured based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional human body model with semantic information of the person to be measured. In this way, the three-dimensional human body model with semantic information of various human postures and / or body shapes can be reconstructed through the three-dimensional human body construction model, improving the robustness and generalization of the model. Then, further based on the three-dimensional human body model with semantic information of the person to be measured, the standard acupoints, and the three-dimensional standard human body model with semantic information, the marked standard acupoints can be accurately matched to the three-dimensional human body of the patient, realizing the accurate search for acupoints. And various human postures and / or body shapes in the actual situation are also considered, improving the practicability of the acupoint determination method. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 is one of the flowcharts of the acupoint determination method provided by the present invention.

[0019] Figure 2 is the structural diagram of the three-dimensional human body model with semantic information provided by the present invention.

[0020] Figure 3It is a schematic structural diagram of a parametric human body model with various postures and body types provided by the present invention.

[0021] Figure 4A It is a schematic structural diagram of standard acupoints marked from the front view in the standard model provided by the present invention.

[0022] Figure 4B It is a schematic structural diagram of standard acupoints marked from the rear view in the standard model provided by the present invention.

[0023] Figure 5 It is the second schematic flow diagram of the acupoint determination method provided by the present invention.

[0024] Figure 6 It is the acupoint determination device provided by the present invention.

[0025] Figure 7 It is a schematic structural diagram of an electronic device provided by the present invention. Detailed implementation manners

[0026] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0027] The acupoint determination methods disclosed in the prior art include: (1) performing three-dimensional reconstruction on a standard human body to construct a three-dimensional human body model; (2) creating an interactive sphere representing an acupoint, i.e., a three-dimensional meridian acupoint model, according to the acupuncture depth and range of each acupoint; (3) based on the three-dimensional human body model, pre-marking multiple body surface landmark points, and based on the multiple body surface landmark points, stretching or shrinking appropriately to match the three-dimensional meridian acupoint model constructed in the three-dimensional human body model to the three-dimensional body surface structure model of the patient; (4) displaying the three-dimensional meridian acupoint system structure of the patient in real time.

[0028] In addition, the method for constructing a three-dimensional human body model includes: (1) obtaining single-view human body point cloud data of a person to be measured; (2) inputting the single-view human body point cloud data into a trained human body model construction network to obtain the three-dimensional human body model of the person to be measured.

[0029] In the prior art, the three-dimensional data of both the standard human body and the patient's human body are obtained by high-precision 3D scanners, which are not only expensive but also inconvenient to use. The three-dimensional models of the standard human body and the patient's human body are both ordinary three-dimensional mesh models without semantic information. When matching the three-dimensional stereoscopic meridian acupoint model built on the standard human body to the three-dimensional stereoscopic body surface structure model of the patient, it is necessary to pre-mark multiple body surface landmark points (41 body surface landmark points are given in the embodiment) on the three-dimensional human model to complete the model matching. The specific matching process is as follows: According to the difference between the three-dimensional stereoscopic body surface structure model of the patient and the virtual model, use the meridian acupoint mixed reality interaction system to perform appropriate stretching or shrinking; when accurately matching the three-dimensional stereoscopic meridian acupoint model already built in the virtual model with the three-dimensional stereoscopic body surface structure model of the patient, ensure that the meridian acupoint model can accurately cover the corresponding area on the patient's 3D body surface model. The human body is a non-rigid hinge structure with various structural attributes such as height, weight, and body type. It is very difficult to obtain an accurate match between the two models only by stretching or shrinking between the two models through dozens of body surface landmark points, thus reducing the accuracy of acupoint determination.

[0030] Moreover, in the prior art, only the construction of a three-dimensional human model from single-viewpoint point cloud data is described, and the trained human model construction network cannot robustly process single-view incomplete human point clouds and complete human point clouds simultaneously, resulting in inaccurate acupoint determination.

[0031] Based on the above problems, the present invention provides an acupoint determination method. Through a three-dimensional human construction model, a three-dimensional human model with semantic information of various human postures and / or human body types can be reconstructed, improving the robustness and generalization of the model. Then, further according to the three-dimensional human model to be measured with semantic information, standard acupoints, and the three-dimensional standard human model with semantic information, the marked standard acupoints can be accurately matched to the three-dimensional human body of the patient, realizing the accurate search for acupoints. And various human postures and / or body types in the actual situation are also considered, improving the practicability of the acupoint determination method.

[0032] The following combines Figures 1 - 5 to describe the acupoint determination method of the present invention. This acupoint determination method is applicable to any human body. The execution subject of this method can be an electronic device or an acupoint determination method set in the electronic device. This acupoint determination device can be implemented through software, hardware, or a combination of both.

[0033] Figure 1 is one of the schematic flowcharts of the acupoint determination method provided by the present invention. As Figure 1 shown, this method includes the following: Step 101, obtain the point cloud data of the person to be measured.

[0034] Here, the person to be measured can be in any posture or of any body type, and the point cloud data can be complete point cloud data or single-view point cloud data. The present invention does not limit this.

[0035] The point cloud data of the person to be measured can be the point cloud data collected by a scanner; it can also be the RGB image and depth image obtained by using a Red Green Blue Depth (RGBD) camera and then converted into point cloud data; it can also be the ordinary image obtained by using an ordinary camera, creating a corresponding parametric human model from the ordinary image, and collecting from the parametric human model.

[0036] Step 102: Input the point cloud data of the person to be measured into the three-dimensional human body construction model to obtain the three-dimensional human body model with semantic information corresponding to the person to be measured output by the three-dimensional human body construction model.

[0037] Among them, the three-dimensional human body construction model is trained based on the point cloud data of various human postures and / or various human body types, and the semantic information is used to represent the human body area information where each position point in the human body model is located.

[0038] Here, different semantic information represents different human body areas, and each point cloud data corresponds to a predefined semantic information. For example, the 1000th point represents the earlobe, and the 500th point represents the navel, etc.

[0039] It should be noted that the obtained point cloud data is disorderly, and the three-dimensional human body model with semantic information is in order, and each point has corresponding information.

[0040] Exemplarily, Figure 2 is the structural schematic diagram of the three-dimensional human body model with semantic information provided by the present invention. As Figure 2 shown, different color areas represent different semantic information, and different semantic information represents different human body areas. For example, the first 100 points represent the left leg area, and the left leg area is represented by purple. For example, the 500th - 600th points represent the head area, and the head area is represented by green.

[0041] Figure 3 is the structural schematic diagram of the parametric human body models of various postures and body types provided by the present invention. As Figure 3 shown, the point cloud data of the parametric human body models of various postures and body types is collected as a training data set to train the initial model to obtain the three-dimensional human body construction model, improving the robustness and generalization ability of the three-dimensional human body construction model.

[0042] Step 103: Input the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model, and obtain the three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model.

[0043] Here, the standard human body means that all parts of the human body are fully displayed without any occluded parts. For example, the situation where the hand occludes the abdomen will not occur.

[0044] Step 104: Based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional human body model to be measured with semantic information, determine the acupoints of the person to be measured.

[0045] Here, any suitable method can be used to determine the acupoints of the person to be measured. For example, the acupoints of the person to be measured can be obtained through the mapping relationship among the three-dimensional standard human body model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional human body model to be measured with semantic information. Another example is to input the three-dimensional standard human body model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional human body model to be measured with semantic information into a neural network model to obtain the acupoints of the person to be measured, etc.

[0046] In an example, the determining of the acupoints of the person to be measured based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional human body model to be measured with semantic information includes: determining the first spatial positions corresponding to the respective standard acupoints in the standard human body; respectively determining the first target key points closest to the respective first spatial positions on the three-dimensional standard human body model with semantic information; determining the first key point numbers corresponding to the respective first target key points in the three-dimensional standard human body model with semantic information; and determining the acupoints of the person to be measured based on the first key point numbers and the three-dimensional human body model to be measured with semantic information.

[0047] Here, the spatial position refers to the three-dimensional coordinate position (x, y, z), and the first key point number is the number of the position among the vertices of the three-dimensional standard human body model with semantic information.

[0048] It should be noted that a three-dimensional standard human body model with semantic information is reconstructed based on the point cloud data in the parametric standard human body model. The three-dimensional standard human body model almost corresponds to the parametric standard human body model. The spatial positions of the standard acupoints on the three-dimensional standard human body model are almost the same as those on the parametric standard human body model. Therefore, the first target key point closest to the first spatial position is the acupoint on the three-dimensional standard human body model with semantic information. Then, the first key point numbers of each first target key point in the three-dimensional standard human body model with semantic information are determined. Based on the first key point numbers and the three-dimensional human body model to be measured with semantic information, the acupoints of the person to be measured are determined.

[0049] In the embodiment of the present invention, through the correspondence relationship between the reconstructed three-dimensional human body model to be measured with semantic information and the three-dimensional standard human body model, the accurately marked standard acupoint positions can be precisely matched to the three-dimensional human body of the person to be measured, improving the accuracy of acupoint search.

[0050] Exemplarily, determining the acupoints of the person to be measured based on the first key point numbers and the three-dimensional human body model to be measured with semantic information includes: determining the second spatial positions of at least one target acupoint to be measured in the three-dimensional human body model to be measured with semantic information based on the first key point numbers and the numbers in the three-dimensional human body model to be measured with semantic information; respectively determining the second target key points closest to each of the second spatial positions in the point cloud data of the person to be measured; and determining each of the second target key points as the acupoints of the person to be measured.

[0051] It should be noted that the numbers of each point in the three-dimensional human body model to be measured with semantic information or the three-dimensional standard human body model reconstructed by the three-dimensional human body construction model and the semantic information corresponding to each number can be the same. For example, if the 100th point in the three-dimensional human body model to be measured with semantic information represents the nose, then the 100th point in the three-dimensional standard human body model with semantic information also represents the nose. Therefore, the first key point numbers can be mapped to the numbers in the three-dimensional human body model to be measured with semantic information to obtain the acupoints in the three-dimensional human body model to be measured with semantic information. Then, since the point cloud data of the person to be measured has no semantic information and the acupoints of the person to be measured cannot be found according to the numbers, the acupoints in the person to be measured can be determined based on the spatial position and the nearest neighbor search. That is to say, the point closest to the second spatial position in the point cloud data of the person to be measured is the acupoint of the person to be measured.

[0052] In the embodiment of the present invention, through the correspondence relationship between the reconstructed three-dimensional human body model to be measured with semantic information and the three-dimensional standard human body model, the accurately marked standard acupoint positions can be precisely matched to the three-dimensional human body of the person to be measured, improving the accuracy of acupoint search.

[0053] In an embodiment of the present invention, by obtaining the point cloud data of a person to be measured; inputting the point cloud data of the person to be measured into a three-dimensional human body construction model, a three-dimensional human body model to be measured with semantic information corresponding to the person to be measured output by the three-dimensional human body construction model is obtained; inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model, a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model is obtained; based on at least one standard acupoint marked on the standard human body in the three-dimensional standard human body model with semantic information and the three-dimensional human body model to be measured with semantic information, the acupoints of the person to be measured are determined. In this way, the three-dimensional human body construction model can reconstruct three-dimensional human body models with semantic information in various human postures and / or human body shapes, improving the robustness and generalization of the model. Then, further based on the three-dimensional human body model to be measured with semantic information, the standard acupoints, and the three-dimensional standard human body model with semantic information, the marked standard acupoints can be accurately matched to the three-dimensional human body of the patient, realizing the accurate search for acupoints. And various human postures and / or body shapes in the actual situation are also considered, improving the practicality of the acupoint determination method.

[0054] The training process of the three-dimensional human body construction model is described below.

[0055] The three-dimensional human body construction model is trained based on the following method: obtaining the complete point cloud data samples of complete parametric human body model samples with different postures and / or different body shapes and the single-viewpoint cloud data samples corresponding to the complete parametric human body model samples; inputting the single-viewpoint cloud data samples into an initial three-dimensional human body construction model, obtaining a reconstructed three-dimensional human body model with semantic information and the first global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model; inputting the reconstructed three-dimensional human body model with semantic information into the initial three-dimensional human body construction model, obtaining the second global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model; based on the global feature constraint loss, determining the first loss value between the first global feature and the second global feature; based on the reconstruction loss, determining the second loss value between the reconstructed three-dimensional human body model with semantic information and the complete point cloud data samples; based on the first loss value and the second loss value, determining the total loss value of the initial three-dimensional human body construction model; based on the total loss value, continuously training the initial three-dimensional human body construction model until the total loss value converges, obtaining the three-dimensional human body construction model.

[0056] Here, the methods for obtaining the sample complete parametric human body model include but are not limited to the SCAPE model (human digital parameter model), the Skinned Multi-Person Linear Model (SMPL), etc.

[0057] Here, the global feature constraint loss is used to represent the difference between the global features corresponding to the complete point cloud data sample of the reconstructed 3D human model with semantic information and the global features corresponding to the single-view point cloud data sample. The smaller the global feature constraint loss, the closer the extracted global features of the reconstructed complete 3D human model are to those of the input single-view human model.

[0058] It should be noted that the global feature constraint loss can be determined by any suitable loss function, including but not limited to Mean Squared Error Loss (MSELoss), Mean Absolute Error (MAE), Huber Loss function, etc.

[0059] Here, the reconstruction loss is used to represent the difference between the complete point cloud data of the reconstructed 3D human model with semantic information and the complete point cloud data sample of the complete parametric human model sample. The smaller the reconstruction loss, the closer the reconstructed 3D human model with semantic information is to the complete parametric human model sample.

[0060] It should be noted that the reconstruction loss can be determined by any suitable loss function, including but not limited to Mean Squared Error Loss (MSELoss), Mean Absolute Error (MAE), Huber Loss function, etc.

[0061] Here, the total loss value can be the sum of the first loss value and the second loss value, or it can be the weighted sum of the first loss value and the second loss value. The weights of the first loss value and the second loss value can be adjusted according to the actual situation. The present invention does not limit this.

[0062] It should be noted that the training of the 3D human construction model is completed until the training completion basis jointly constituted by the global feature constraint loss and the reconstruction loss meets the preset requirements, and then it is determined that the training of the 3D human construction model is completed.

[0063] It should be noted that the training process of the 3D human construction model is a negative feedback process. By continuously calculating the difference between the global features corresponding to the complete point cloud data of the reconstructed 3D human model with semantic information and the global features corresponding to the single-view point cloud data sample, and the difference between the complete point cloud data of the reconstructed 3D human model with semantic information and the complete point cloud data sample of the complete parametric human model sample, the total loss value is obtained. The total loss value is fed back to the model to adjust the parameters and continue to determine the next total loss value until the total loss value converges, and then the 3D human construction model is obtained.

[0064] In the embodiments of the present invention, a large number of parametric human models with various postures and body shapes are used as training data to train a neural network model. Using the trained neural network model, the point cloud data of the person to be measured and the point cloud data of the standardized human model can be reconstructed into a three-dimensional human model with semantic information. At the same time, various postures and body shapes improve the robustness and accuracy of the model, avoiding overfitting. Moreover, the three-dimensional human construction model trained by the global feature constraint loss and the reconstruction loss can construct a human model according to the complete point cloud data or according to the single-viewpoint cloud data, so as to realize the determination of acupoints of the complete human point cloud and the determination of acupoints of the single-viewpoint cloud.

[0065] Exemplarily, obtaining the single-viewpoint cloud data sample includes: determining the viewing range of the single view; based on the viewing range and the hidden point removal algorithm, determining the single-viewpoint cloud data sample corresponding to the complete point cloud data sample under the viewing range.

[0066] Here, in addition to determining the viewing range, the viewing point can also be determined. The complete point cloud data is collected from different viewing ranges and different viewing points to improve the diversity of the point cloud data and the robustness and accuracy of the training model.

[0067] Here, the hidden points refer to the points that are invisible in the viewing range. Through the hidden point removal algorithm, multiple single-viewpoint cloud data are generated.

[0068] In the embodiments of the present invention, by collecting the complete point cloud data from different viewing ranges and different viewing points, the diversity of the point cloud data is improved, and the robustness and accuracy of the training model are improved.

[0069] Further, before obtaining the complete point cloud data sample of the complete parametric human model with different postures and / or different body shapes, the method further includes: Obtaining at least one human parameter; the human parameter includes a posture parameter and a body shape parameter; Based on each of the posture parameters and each of the body shape parameters, determining the complete parametric human model corresponding to each of the human parameters.

[0070] Here, the human parameters can be obtained from a public dataset or collected by a collection device.

[0071] Here, the method for determining the sample complete parametric human model can be any suitable method: for example, inputting the posture parameter and the body shape parameter into software to obtain the sample complete parametric human model; or, using a reconstruction algorithm to construct the sample complete parametric human model.

[0072] In the embodiments of the present invention, by using a large number of parametric human models with various postures and body shapes as training data, the collection of large-scale three-dimensional human data is avoided.

[0073] Further, before inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model, the method further includes: Obtaining the number of mesh divisions; In the case where the number of mesh divisions is greater than the division threshold, determining that the division degree is fine division, and finely dividing the parametric human body model with the standard pose based on the number of mesh divisions to obtain a standard human body model; the number of position points of the standard human body model is more than that of the parametric human body model with the standard pose; Marking acupoints on the standard human body model based on the position points in the standard human body model to obtain the at least one standard acupoint.

[0074] Here, the higher the degree of fine division, the more meshes and position points there are; the lower the degree of fine division, the fewer meshes and position points there are.

[0075] It should be noted that in order to ensure the accuracy of acupoint marking in the later stage, the higher the degree of fine division, the better.

[0076] Figure 4A is a schematic structural diagram of the standard acupoints marked in the front view of the standard model provided by the present invention, as Figure 4A shown, the red dots represent the marked standard acupoints.

[0077] Figure 4B is a schematic structural diagram of the standard acupoints marked in the rear view of the standard model provided by the present invention, as Figure 4B shown, the red dots represent the marked standard acupoints.

[0078] In the embodiment of the present invention, directly marking on the refined standard parametric human body model without additional scanning, and the acupoints of patients with any posture and body type can be determined only through one-time marking of the acupoints of the whole body, solving the problem of dependence on a large amount of marked data. Moreover, by performing multiple mesh refinements on the parametric human body model, the number of position points is increased, the marking accuracy is improved, and thus the accuracy of acupoint determination is enhanced.

[0079] Figure 5 is the second schematic flowchart of the acupoint determination method provided by the present invention, as Figure 5 shown, including: Step 501, performing mesh refinement on the parametric human body model with the standard pose to generate a standard model with denser position points.

[0080] Step 502, marking acupoints on the standard model to obtain at least one standard acupoint.

[0081] Step 503: Based on the sample complete point cloud data of the parametric human body model with different postures and / or different body types and the single-view point cloud data corresponding to the sample complete parametric human body model, continuously train the initial three-dimensional human body construction model until the difference value converges to obtain a three-dimensional human body construction model.

[0082] Specifically, generate a large number of parametric human body models with various postures and body types, randomly set the virtual camera positions, and generate multiple single-view point cloud data through the hidden point removal algorithm; use the generated single-view point cloud data as the input of the initial three-dimensional human body construction model to obtain the reconstructed three-dimensional human body model with semantic information and the first global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model. Use the reconstructed three-dimensional human body model with semantic information as the input of the initial three-dimensional human body construction model to obtain the second global feature extracted by the feature extraction part network in the initial three-dimensional human body construction model. Based on the global feature constraint loss, determine the first loss value between the first global feature and the second global feature. Based on the reconstruction loss, determine the second loss value between the complete point cloud data of the reconstructed three-dimensional human body model with semantic information and the complete point cloud data sample of the complete parametric human body model sample. According to the first loss value and the second loss value, determine the total loss value, and continuously train the model using error backpropagation until the total loss value converges to obtain a three-dimensional human body construction model. This three-dimensional human body construction model can reconstruct a three-dimensional human body model with semantic information from the input complete / single-view point cloud data.

[0083] Step 504: Input the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model.

[0084] Step 505: Map the standard acupoints marked on the standard model to the reconstructed three-dimensional standard human body model with semantic information through nearest neighbor search.

[0085] Step 506: Input the point cloud data of the person to be measured into the three-dimensional human body construction model to obtain a three-dimensional human body model with semantic information corresponding to the person to be measured output by the three-dimensional human body construction model.

[0086] Step 507: According to the semantic information correspondence, directly search for the key points of the three-dimensional human body model with semantic information of the person to be measured according to the position point numbers to obtain the acupoints to be measured on this model.

[0087] Step 508: In the point cloud data of the person to be measured, respectively determine the second target key points with the closest second spatial positions to obtain the acupoints of the person to be measured.

[0088] The acupoint determination method provided by the present invention has the following advantages: By utilizing the structural characteristics and semantic information of the parametric human model, the acupoint determination for patients with any posture and body type can be achieved only through one-time annotation of acupoints throughout the body, solving the problem of dependence on a large amount of annotated data; when training the neural network, a large number of parametric human models with various postures and body types are used as training data, avoiding the acquisition of large-scale three-dimensional human data; through the trained neural network, the input point cloud is reconstructed with semantic information and finally the acupoints are searched, which has a higher accuracy than simply matching the three-dimensional meridian and acupoint model with the three-dimensional body surface structure model of the patient by moderate stretching or shrinking; the acupoint search for single-view human point cloud and complete human point cloud can be achieved simultaneously.

[0089] The acupoint determination device provided by the present invention will be described below. The acupoint determination device described below can be correspondingly referred to the acupoint determination method described above.

[0090] Figure 6 is the acupoint determination device provided by the present invention, as Figure 6 shown. The acupoint determination device 600 includes the following modules: An acquisition module 601, configured to acquire the point cloud data of the person to be measured; A to-be-measured human model construction module 602, configured to input the point cloud data of the person to be measured into the three-dimensional human construction model, and obtain the three-dimensional to-be-measured human model with semantic information corresponding to the person to be measured output by the three-dimensional human construction model; the three-dimensional human construction model is trained based on the point cloud data of various human postures and / or various human body types, and the semantic information is used to represent the human region information where each position point in the human model is located; A standard human model construction module 603, configured to input the standard point cloud data corresponding to the standard human into the three-dimensional human construction model, and obtain the three-dimensional standard human model with semantic information corresponding to the standard human output by the three-dimensional human construction model; An acupoint determination module 604, configured to determine the acupoints of the person to be measured based on the three-dimensional standard human model with semantic information, at least one standard acupoint annotated on the standard human, and the three-dimensional to-be-measured human model with semantic information.

[0091] In one embodiment, the acupoint determination module 604 is specifically configured to: determine the first spatial positions corresponding to the respective standard acupoints in the standard human; respectively determine the first target key points closest to the respective first spatial positions on the three-dimensional standard human model with semantic information; determine the first key point numbers corresponding to the respective first target key points in the three-dimensional standard human model with semantic information; and determine the acupoints of the person to be measured based on the first key point numbers and the three-dimensional to-be-measured human model with semantic information.

[0092] In one embodiment, the acupoint determination module 604 is further specifically configured to: determine the second spatial position of at least one target acupoint to be measured in the three-dimensional human body model to be measured with semantic information based on the serial number of the first key point and the serial number in the three-dimensional human body model to be measured with semantic information; in the point cloud data of the person to be measured, respectively determine the second target key points closest to each of the second spatial positions; and determine each of the second target key points as the acupoints of the person to be measured.

[0093] In one embodiment, the acupoint determination device 600 further includes a training module, which is specifically configured to: obtain the complete point cloud data samples of the complete parametric human body model samples with different postures and / or different body types and the single-view point cloud data samples corresponding to the complete parametric human body model samples; input the single-view point cloud data samples into the initial three-dimensional human body construction model to obtain the reconstructed three-dimensional human body model with semantic information and the first global features extracted by the feature extraction part network in the initial three-dimensional human body construction model; input the reconstructed three-dimensional human body model with semantic information into the initial three-dimensional human body construction model to obtain the second global features extracted by the feature extraction part network in the initial three-dimensional human body construction model; determine the first loss value between the first global features and the second global features based on the global feature constraint loss; determine the second loss value between the reconstructed three-dimensional human body model with semantic information and the complete point cloud data samples based on the reconstruction loss; determine the total loss value of the initial three-dimensional human body construction model based on the first loss value and the second loss value; and continuously train the initial three-dimensional human body construction model based on the total loss value until the total loss value converges to obtain the three-dimensional human body construction model.

[0094] In one embodiment, before obtaining the complete point cloud data samples of the complete parametric human body model with different postures and / or different body types, the training module is further specifically configured to: obtain at least one human parameter; the human parameter includes a posture parameter and a body type parameter; and determine the complete parametric human body model corresponding to each of the human parameters based on each of the posture parameters and each of the body type parameters.

[0095] In one embodiment, the training module is further specifically configured to: determine the viewing range of a single view; and determine the single-view point cloud data samples corresponding to the complete point cloud data samples within the viewing range based on the viewing range and the hidden point removal algorithm.

[0096] In one embodiment, before inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model, the acupoint determination device 600 further includes a division module, specifically used for: obtaining the number of mesh divisions; in the case where the number of mesh divisions is greater than the division threshold, determining that the division degree is fine division, and finely dividing the parametric human body model with the standard posture based on the number of mesh divisions to obtain a standard human body model; the number of position points of the standard human body model is more than that of the parametric human body model with the standard posture; based on the position points in the standard human body model, marking acupoints in the standard human body model to obtain the at least one standard acupoint.

[0097] Figure 7 is a schematic structural diagram of an electronic device provided by the present invention, as Figure 7 shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communication interface 720, and the memory 730 complete mutual communication through the communication bus 740. The processor 710 can call the logical instructions in the memory 730 to execute the acupoint determination method, and the method includes: obtaining the point cloud data of the person to be measured; inputting the point cloud data of the person to be measured into the three-dimensional human body construction model to obtain the three-dimensional measured human body model with semantic information corresponding to the person to be measured output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on the point cloud data of various human postures and / or various human body types, and the semantic information is used to characterize the human body area information where each position point in the human body model is located; inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; determining the acupoints of the person to be measured based on the three-dimensional standard human body model with semantic information, the at least one standard acupoint marked on the standard human body, and the three-dimensional measured human body model with semantic information.

[0098] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0099] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the acupoint determination method provided by the above-mentioned various methods. The method includes: obtaining the point cloud data of the person to be measured; inputting the point cloud data of the person to be measured into a three-dimensional human body construction model to obtain a three-dimensional human body model with semantic information corresponding to the person to be measured output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on the point cloud data of various human postures and / or various human body shapes, and the semantic information is used to represent the human body area information where each position point in the human body model is located; inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional human body model with semantic information corresponding to the person to be measured, determine the acupoints of the person to be measured.

[0100] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is used to execute the acupoint determination method provided by the above-mentioned various methods. The method includes: obtaining the point cloud data of the person to be measured; inputting the point cloud data of the person to be measured into a three-dimensional human body construction model to obtain a three-dimensional human body model with semantic information corresponding to the person to be measured output by the three-dimensional human body construction model; the three-dimensional human body construction model is trained based on the point cloud data of various human postures and / or various human body shapes, and the semantic information is used to represent the human body area information where each position point in the human body model is located; inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked on the standard human body, and the three-dimensional human body model with semantic information of the person to be measured, determine the acupoints of the person to be measured.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0102] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining an acupoint, characterized in that: include: Obtaining point cloud data of the person to be tested; Inputting the point cloud data of the subject to be tested into a three-dimensional human body construction model, obtaining a three-dimensional human body model with semantic information corresponding to the subject to be tested output by the three-dimensional human body construction model; the three-dimensional human body construction model is obtained by training based on point cloud data of various human postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each position point in the human body model is located; Inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model, and obtaining a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; The acupoints of the subject to be tested are determined based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model to be tested with semantic information.

2. The acupoint determination method according to claim 1, characterized in that: The method of determining the acupuncture points of the subject to be tested based on the three-dimensional standard human body model with semantic information, at least one standard acupuncture point marked in the standard human body, and the three-dimensional human body model to be tested with semantic information includes: Determining a first spatial position corresponding to each of the standard acupuncture points in the standard human body; On the three-dimensional standard human body model with semantic information, respectively determining the first target key points closest to each of the first spatial positions; Determine the first key point sequence number corresponding to each of the first target key points on the three-dimensional standard human body model with semantic information; Acupuncture points of the subject to be tested are determined based on the first key point sequence number and the three-dimensional human body model to be tested having semantic information.

3. The acupoint determination method according to claim 2, characterized in that: The step of determining the acupuncture points of the subject to be tested based on the first key point sequence number and the three-dimensional human body model to be tested having semantic information comprises: Determine a second spatial position of at least one target acupuncture point to be tested in the three-dimensional human body model to be tested with semantic information based on the first key point sequence number and the sequence number in the three-dimensional human body model to be tested with semantic information; In the point cloud data of the subject, respectively determining the second target key points closest to each of the second spatial positions; Each of the second target key points is determined as an acupuncture point of the subject.

4. The acupoint determination method according to any one of claims 1 to 3, characterized in that: The three-dimensional human body model is trained based on the following method: Acquire complete point cloud data samples of complete parameterized human body model samples of different postures and / or different body shapes and single-view point cloud data samples corresponding to the complete parameterized human body model samples; Inputting the single-view point cloud data sample into an initial three-dimensional human body construction model to obtain a reconstructed three-dimensional human body model with semantic information and a first global feature extracted by a feature extraction part network in the initial three-dimensional human body construction model; Inputting the reconstructed three-dimensional human body model with semantic information into the initial three-dimensional human body construction model to obtain a second global feature extracted by a feature extraction part network in the initial three-dimensional human body construction model; Determining a first loss value between the first global feature and the second global feature based on a global feature constraint loss; Based on the reconstruction loss, determining a second loss value between the reconstructed three-dimensional human body model with semantic information and the complete point cloud data sample; Determining a total loss value of the initial three-dimensional human body construction model based on the first loss value and the second loss value; Based on the total loss value, the initial three-dimensional human body construction model is continuously trained until the total loss value converges to obtain the three-dimensional human body construction model.

5. The acupoint determination method according to claim 4, characterized in that: Before obtaining complete point cloud data samples of complete parameterized human body models of different postures and / or different body shapes, the method further includes: Acquire at least one human body parameter; the human body parameter includes posture parameters and body shape parameters; Based on the posture parameters and the body shape parameters, a complete parameterized human body model corresponding to each human body parameter is determined.

6. The acupoint determination method according to claim 4, characterized in that: Acquiring the single-view point cloud data sample, including: Determine the viewing angle range of a single viewing angle; Based on the viewing angle range and the hidden point removal algorithm, a single-view point cloud data sample corresponding to the complete point cloud data sample in the viewing angle range is determined.

7. The acupoint determination method according to any one of claims 1 to 3, characterized in that: Before inputting the standard point cloud data corresponding to the standard human body into the three-dimensional human body construction model to obtain the three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model, the method further includes: Get the number of grid divisions; In the case where the number of grid divisions is greater than the division threshold, determining the division degree as fine division, and finely dividing the parameterized human body model with standard posture based on the number of grid divisions to obtain a standard human body model; the position points of the standard human body model are more than the position points in the parameterized human body model with standard posture; Based on the position points in the standard human body model, acupoints are marked in the standard human body model to obtain the at least one standard acupoint.

8. An acupoint determination device, characterized in that: include: An acquisition module is used to acquire point cloud data of the subject to be tested; A human body model construction module is used to input the point cloud data of the subject to be tested into a three-dimensional human body construction model, and obtain a three-dimensional human body model with semantic information corresponding to the subject to be tested output by the three-dimensional human body construction model; the three-dimensional human body construction model is obtained by training based on point cloud data of various human body postures and / or various human body shapes, and the semantic information is used to characterize the human body region information where each position point in the human body model is located; A standard human body model construction module, used for inputting standard point cloud data corresponding to a standard human body into the three-dimensional human body construction model, and obtaining a three-dimensional standard human body model with semantic information corresponding to the standard human body output by the three-dimensional human body construction model; The acupoint determination module is used to determine the acupoints of the subject to be tested based on the three-dimensional standard human body model with semantic information, at least one standard acupoint marked in the standard human body, and the three-dimensional human body model to be tested with semantic information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the acupoint determination method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the acupoint determination method according to any one of claims 1 to 7 is implemented.

Images