Human body meridian point detection method and device, electronic equipment and storage medium

Abstract

The application provides a human meridian point detection method and device, electronic equipment and storage medium, and relates to the field of computer vision.The method comprises the following steps: obtaining a human image sequence corresponding to a user; inputting the human image sequence into a meridian point detection model to obtain a meridian point detection result output by the meridian point detection model; and the meridian point detection model is obtained based on a sample balanced focal loss function, a plurality of frames of heat maps corresponding to a human image sequence sample of a preset time interval, and effective pixel points corresponding to the plurality of frames of heat maps.The time sequence information of the front and rear frames is introduced in the application, and the image quality reflected by the effective pixel points of the plurality of frames of heat maps is used to dynamically adjust the weights corresponding to the plurality of frames of heat maps, so that even if the acupoint in the image is blocked, the acupoint coordinates can still be accurately inferred through the time sequence information.Meanwhile, the sample balanced focal loss function is used to solve the imbalance problem of positive and negative samples, and the acupoint detection accuracy in the occlusion scene is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting acupoints on the human body. Background Technology

[0002] In recent years, with the development of artificial intelligence technology, TCM physiotherapy robots have been gradually applied in clinical practice to achieve the standardization and intelligentization of meridian therapy. These robots typically rely on computer vision technology to locate acupoints on the human body. Most existing acupoint detection methods are based on processing single-frame static images, extracting image features and regressing to generate heatmaps to locate acupoints. However, in actual physiotherapy, when the robotic arm moves above the human body, it inevitably obstructs the camera's view, causing the visual features of the target acupoints to be lost in a single-frame image, i.e., the "occlusion problem." In single-frame detection mode, once an acupoint is occluded, the algorithm cannot obtain effective visual features, leading to serious missed or false detections.

[0003] In existing technologies, to address occlusion and dynamic blurring issues during motion, a Pose Temporal Merger (PTM) algorithm is generally employed. This algorithm corrects detection results by fusing heatmap information from the current frame, the previous frame, and the next frame. However, the following problems still exist: (1) Existing PTM-type algorithms are mainly optimized for fast motion scenarios. However, traditional Chinese medicine physiotherapy is a slow motion scenario, where the human body is relatively still and the robotic arm moves relatively slowly. When the temporal interval does not match the motion characteristics, the same acupoint in several consecutive frames with a very short time interval may be occluded by the robotic arm.

[0004] (2) Existing methods typically employ simple addition, group convolution, or weighting strategies based on fixed time intervals when fusing multiple heatmaps, which are not robust enough when dealing with complex and dynamically changing occlusion situations.

[0005] (3) Most key point detection algorithms use the mean square error loss function, which cannot effectively handle the class imbalance problem and severely restricts the detection accuracy of the model for occluded acupoints.

[0006] Therefore, improving the detection accuracy and stability of acupoints in slow-motion physiotherapy scenarios is an urgent problem to be solved. Summary of the Invention

[0007] This invention provides a method, device, electronic device, and storage medium for detecting acupoints in the human body, in order to solve the shortcomings of low detection accuracy and stability in the existing technology when detecting acupoints in slow-motion physiotherapy scenarios.

[0008] This invention provides a method for detecting acupoints in the human body, comprising the following steps.

[0009] Obtain the human body image sequence corresponding to the user.

[0010] The human image sequence is input into the acupoint detection model to obtain the acupoint detection results output by the acupoint detection model. The acupoint detection model is trained based on the sample balance focus loss function, the multi-frame heatmaps corresponding to human image sequence samples with a preset time interval, and the effective pixel points corresponding to the multi-frame heatmaps.

[0011] According to the method for detecting acupoints in the human body provided by the present invention, the acupoint detection model is trained based on the following steps: The human image sequence samples are input into the initial acupoint detection model to obtain the current frame prediction heatmap, the previous frame prediction heatmap, and the next frame prediction heatmap; the frame interval between the previous frame prediction heatmap and the next frame prediction heatmap is the preset time interval; The current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame prediction heatmap are determined respectively, as well as their respective weights; A comprehensive deviation heatmap is determined based on the current frame deviation heatmap, the previous frame deviation heatmap, the subsequent frame deviation heatmap, and their respective weights. Based on the current frame predicted heatmap and the current frame ground truth heatmap, determine the sample balance focus loss function; The initial acupoint detection model is trained based on the comprehensive deviation heatmap and the sample balance focus loss function to obtain the acupoint detection model.

[0012] According to the method for detecting acupoints in the human body provided by the present invention, the step of determining the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame predicted heatmap includes: Based on the current frame predicted heatmap and the current frame true heatmap, determine the current frame deviation heatmap; Based on the current frame predicted heatmap and the previous frame ground truth heatmap corresponding to the previous frame human image sample, determine the previous frame deviation heatmap; Based on the predicted heatmap of the current frame and the ground truth heatmap of the subsequent frame corresponding to the human image sample of the subsequent frame, the deviation heatmap of the subsequent frame is determined; the frame interval between the human image sample of the subsequent frame and the human image sample of the previous frame is the preset time interval.

[0013] According to the human acupoint detection method provided by the present invention, the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame predicted heatmap are determined respectively, including: Based on a first preset threshold, the effective pixel points of the current frame corresponding to the current frame prediction heatmap, the effective pixel points of the previous frame corresponding to the previous frame prediction heatmap, and the effective pixel points of the subsequent frame corresponding to the subsequent frame prediction heatmap are determined respectively. Based on the effective pixels of the current frame, the effective pixels of the previous frame, and the effective pixels of the next frame, the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the next frame deviation heatmap are determined respectively.

[0014] According to the human acupoint detection method provided by the present invention, the step of determining the effective pixel points of the current frame corresponding to the current frame predicted heatmap, the effective pixel points of the previous frame corresponding to the previous frame predicted heatmap, and the effective pixel points of the subsequent frame corresponding to the subsequent frame predicted heatmap based on a first preset threshold includes: All pixels in the current frame prediction heatmap with a confidence level greater than or equal to the first preset threshold are determined as valid pixels in the current frame. All pixels in the predicted heatmap of the previous frame with a confidence level greater than or equal to the first preset threshold are determined as valid pixels of the previous frame. All pixels in the predicted heatmap of the next frame with a confidence level greater than or equal to the first preset threshold are determined as valid pixels in the next frame.

[0015] According to the human acupoint detection method provided by the present invention, the step of determining the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap based on the effective pixels of the current frame, the effective pixels of the previous frame, and the effective pixels of the subsequent frame includes: The total number of valid pixels is determined based on the number of pixels corresponding to the current frame's valid pixels, the previous frame's valid pixels, and the next frame's valid pixels. The ratio of the number of current pixels corresponding to the current frame's valid pixels to the total number of valid pixels is determined as the weight corresponding to the current frame's deviation heatmap. The ratio of the number of previous frame pixels corresponding to the previous frame effective pixels to the total number of effective pixels is determined as the weight corresponding to the previous frame deviation heatmap. The ratio of the number of pixels in the subsequent frame corresponding to the effective pixels in the subsequent frame to the total number of effective pixels is determined as the weight corresponding to the subsequent frame deviation heatmap.

[0016] According to the human acupoint detection method provided by the present invention, the step of determining the sample balance focus loss function based on the current frame predicted heatmap and the current frame ground truth heatmap includes: When the prediction probability of each effective pixel in the current frame in the current frame prediction heatmap is equal to the second preset threshold, the first sub-loss function is determined based on the ground truth probability of the effective pixel in the current frame in the current frame ground truth heatmap and the first adjustment factor. If the prediction probability corresponding to each effective pixel in the current frame prediction heatmap is not equal to the second preset threshold, a second sub-loss function is determined based on the prediction probability, the true probability, the first adjustment factor, and the second adjustment factor corresponding to the effective pixel in the current frame; the first adjustment factor and the second adjustment factor are used to balance the attention of the initial acupoint detection model to positive and negative samples; The current frame deviation heatmap is determined based on the total number of key points, all first sub-loss functions, and all second sub-loss functions; the total number of key points is the number of valid pixels in the current frame with a prediction probability equal to a second preset threshold.

[0017] The present invention also provides a human acupoint detection device, comprising the following modules.

[0018] The acquisition module is used to acquire the human body image sequence corresponding to the user.

[0019] The detection module is used to input the human image sequence into the acupoint detection model and obtain the acupoint detection results output by the acupoint detection model. The acupoint detection model is trained based on the sample balance focus loss function, the multi-frame heatmap corresponding to the human image sequence samples with a preset time interval, and the effective pixel points corresponding to the multi-frame heatmap.

[0020] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human acupoint detection method as described above.

[0021] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the human acupoint detection method as described above.

[0022] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the human acupoint detection method as described above.

[0023] The present invention provides a method, apparatus, electronic device, and storage medium for detecting acupoints in the human body. It trains an initial acupoint detection model using a sample-balanced focus loss function, multi-frame heatmaps corresponding to human image sequence samples at preset time intervals, and the effective pixels corresponding to these heatmaps. The human image sequence is then input into the acupoint detection model to obtain the output acupoint detection results. In this invention, temporal information from consecutive frames is introduced through human image sequence samples at preset time intervals. The weights corresponding to the multi-frame heatmaps are dynamically adjusted based on the image quality reflected by the effective pixels. Even if acupoints in the image are occluded, the acupoint coordinates can still be accurately inferred from the temporal information, significantly improving the accuracy of acupoint detection in occluded scenarios. Simultaneously, the sample-balanced focus loss function effectively reduces the weight of simple negative samples in gradient updates, allowing the model to focus on learning the acupoint features of difficult-to-classify positive samples, solving the problem of positive-negative sample imbalance, and further improving the accuracy of acupoint detection. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating the method for detecting acupoints in the human body provided in an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of the regression curve of the MSE loss function provided by existing technology.

[0027] Figure 3 This is a schematic diagram of the visualized detection results provided in an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram of the structure of the human acupoint detection device provided in an embodiment of the present invention.

[0029] Figure 5 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] To address the issues of low detection accuracy and stability in existing technologies for acupoint detection during slow-motion physical therapy scenarios, this invention provides a method for detecting human acupoints. Figure 1 This is a flowchart illustrating the human acupoint detection method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes steps 110 and 120.

[0032] Step 110: Obtain the human body image sequence corresponding to the user.

[0033] Specifically, the human body image sequence can be a sequence of image frames continuously captured by an image acquisition device such as a camera or camera mounted on the physiotherapy robot, showing at least one part of the user's body. Each frame in the human body image sequence can be a supine or prone image of the human body. The user's body parts can include the user's hands, feet, upper limbs, lower limbs, and torso.

[0034] The images in this human body image sequence are sequential in time, and the number of frames in the sequence is greater than or equal to a preset time interval. Images in this sequence may contain acupoints that are obscured.

[0035] Optionally, after the video stream is captured by the image acquisition device, the video stream can be extracted at equal intervals and arranged in order to obtain the human body image sequence.

[0036] Optionally, after acquiring the human image sequence, preprocessing operations such as image denoising, brightness adjustment, and size normalization can be performed on the human image sequence. The image resolution of the human image to be detected can be 256×256 or 512×512, and this embodiment of the invention does not limit this.

[0037] Step 120: Input the human image sequence into the acupoint detection model to obtain the acupoint detection results output by the acupoint detection model; the acupoint detection model is trained based on the sample balance focus loss function, the multi-frame heatmap corresponding to the human image sequence samples with a preset time interval, and the effective pixel points corresponding to the multi-frame heatmap.

[0038] Before executing step 120, an initial acupoint detection model can be constructed using a keypoint detection network architecture such as HRNet. A training dataset is built using human image samples from a supine human dataset and the corresponding acupoint coordinates for each frame of human image samples. Each sequence of human image samples in the training data includes three frames of human image samples at a preset time interval. For example, with a preset time interval of 8 frames, if the current frame is the 5th frame and the previous frame is the 2nd frame, then the next frame is the 10th frame. Subsequently, the ground truth heatmap corresponding to each frame of human image samples is determined based on the corresponding acupoint coordinates in the human image sample sequence. Next, the sequence of human image samples is input into the initial acupoint detection model. This model provides a wealth of temporal information for acupoint prediction based on the predicted and ground truth heatmaps corresponding to multiple frames of human image samples at preset time intervals, as well as the effective pixels corresponding to the predicted heatmaps. A sample balancing focus loss function is determined to balance the impact of positive and negative samples on gradient updates. For example, this function reduces the influence of negative samples (easily classifiable image samples) on gradient updates, enabling the initial acupoint detection model to more effectively learn the key features of positive samples (i.e., difficult-to-classify image samples). These positive samples can be human image samples where acupoints are occluded or blurred. The model parameters of the initial acupoint detection model are then updated using this sample balancing focus loss function until the model converges, resulting in a fully trained acupoint detection model.

[0039] Then, the sequence of human images is input into the trained acupoint detection model. Even if the acupoints in the current frame of the human image sequence are completely obscured by the robotic arm, the acupoint detection model can still accurately predict the acupoint detection results corresponding to the human image sequence by using the clear acupoint information in the adjacent frames of human images based on the temporal correlation between them.

[0040] It should be noted that the acupoint detection results include the coordinates, confidence level, acupoint name, and meridian to which the acupoint is located in the human image sequence, etc., and the embodiments of the present invention do not limit this.

[0041] The acupoint detection method provided in this invention trains an initial acupoint detection model using a sample balancing focus loss function, multi-frame heatmaps corresponding to human image sequence samples at preset time intervals, and effective pixels corresponding to the multi-frame heatmaps. The human image sequence is then input into the acupoint detection model to obtain the acupoint detection results output by the model. In this invention, temporal information from consecutive frames is introduced through human image sequence samples at preset time intervals, and the weights corresponding to the multi-frame heatmaps are dynamically adjusted based on the image quality reflected by the effective pixels of the multi-frame heatmaps. Even if acupoints in the image are occluded, the acupoint coordinates can still be accurately inferred from the temporal information, significantly improving the accuracy of acupoint detection in occluded scenarios. Simultaneously, the sample balancing focus loss function effectively reduces the weight of simple negative samples in gradient updates, allowing the model to focus on learning the acupoint features of difficult-to-classify positive samples, solving the problem of positive-negative sample imbalance, and further improving the accuracy of acupoint detection.

[0042] In one embodiment, the acupoint detection model is trained based on the following steps: The human image sequence samples are input into the initial acupoint detection model to obtain the current frame prediction heatmap, the previous frame prediction heatmap, and the next frame prediction heatmap; the frame interval between the previous frame prediction heatmap and the next frame prediction heatmap is the preset time interval; The current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame prediction heatmap are determined respectively, as well as their respective weights; A comprehensive deviation heatmap is determined based on the current frame deviation heatmap, the previous frame deviation heatmap, the subsequent frame deviation heatmap, and their respective weights. Based on the current frame predicted heatmap and the current frame ground truth heatmap, determine the sample balance focus loss function; The initial acupoint detection model is trained based on the comprehensive deviation heatmap and the sample balance focus loss function to obtain the acupoint detection model.

[0043] Specifically, in the field of keypoint detection using multi-frame joint heatmap generation, the PTM (Pose Temporal Merging) module significantly improves the performance of detection models by effectively utilizing temporal consistency information. However, in the specific application scenario corresponding to this invention, due to the dense acupoints and limited range of human movement, the keypoint heatmap undergoes a certain degree of decoupling, resulting in a reduction in the confidence region of the multi-frame joint heatmap expected by the PTM module. Furthermore, because the robotic arm moves extensively during the treatment process, compared to the fixed posture of the human body which only moves within a small range with the bed, the confidence region information provided by the multi-frame joint heatmap does not show a significant optimization effect compared to the single-frame heatmap. Therefore, considering that PTM is highly effective in dealing with motion blur caused by human movement and lens defocusing, this embodiment of the invention employs PTM heatmap generation to ensure the model maintains high robustness when handling dynamic scenes. It then utilizes temporal correlation between multiple frame heatmaps to construct a pose temporal merging strategy. Based on this strategy, acupoint detection is performed, and a sample balancing focus loss function is determined to balance positive and negative samples based on the predicted heatmap and the ground truth heatmap of the current frame, thus obtaining the trained acupoint detection model. The specific training includes the following steps.

[0044] 1. Input the human image sequence samples with a frame interval of a preset time interval in the training dataset into the initial acupoint detection model. The initial acupoint detection model predicts the current frame human image sample, the previous frame human image sample, and the next frame human image sample in the human image sequence sample. The predicted heatmap of the current frame human image sample, the predicted heatmap of the previous frame human image sample, and the predicted heatmap of the next frame human image sample are determined respectively.

[0045] It should be noted that during meridian therapy, the human body typically only moves within a small range with the bed, while the main dynamic changes come from the operation of the robotic arm. This low-speed movement makes the positional changes of key points relatively gradual, reducing the possibility of drastic changes in a short period of time. In existing technologies, the PTM module uses a fixed frame interval, for example, 1 to 4 frames, which performs well when handling fast-moving scenarios. However, in the slow-speed scenario of acupoint therapy corresponding to this embodiment of the invention, the frame interval may be too small, causing the same acupoints to still be occluded in the previous and subsequent frames, failing to provide effective supplementary information. Therefore, in this embodiment of the invention, the preset temporal interval between the previous frame human image sample and the subsequent frame human image sample is set to an integer greater than 4, enabling the model to effectively overcome the continuous occlusion caused by the slow movement of the robotic arm. For example, the preset temporal interval can be 8 frames, 10 frames, 12 frames, etc., and this embodiment of the invention does not limit this.

[0046] 2. Based on the predicted heatmaps of the current frame, the previous frame, and the next frame, determine the corresponding deviation heatmaps for each predicted heatmap. Then, dynamically adjust the weights of each deviation heatmap based on the similarity between adjacent frame predicted heatmaps to ensure that the initial acupoint detection model can flexibly adapt to changes in different scenarios. Next, perform a weighted sum of all deviation heatmaps and their corresponding weights to obtain a comprehensive deviation heatmap. This comprehensive deviation heatmap is represented as: L = α × Lcurrent + β × Lprev + γ × Lnext. Where L represents the comprehensive deviation heatmap, Lprev represents the current frame deviation heatmap, Lcurrent represents the previous frame deviation heatmap, Lnext represents the next frame deviation heatmap, α represents the weight of the previous frame deviation heatmap, β represents the weight of the current frame deviation heatmap, and γ represents the weight of the next frame deviation heatmap.

[0047] 3. Based on the predicted heatmap and the ground truth heatmap of the current frame, determine the sample balance focus loss function to balance the impact of positive and negative samples on gradient updates.

[0048] 4. The initial acupoint detection model performs acupoint detection based on the comprehensive deviation heatmap, and updates the model parameters based on the sample balance focus loss function until the model converges, thus obtaining the trained acupoint detection model.

[0049] In this embodiment of the invention, not only is a large amount of temporal information provided for the acupoint prediction of the current frame by weighting the heatmaps of three consecutive time points, but also invalid pixels with low confidence are initially screened by using a first preset threshold. The confidence of these invalid pixels will be lower than the initial value after weighting, ensuring that they are more likely to be eliminated in the NMS process. This reduces the number of low-confidence samples generated by the joint processing of multiple frames to a certain extent, thereby improving the model training efficiency.

[0050] In one embodiment, determining the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame prediction heatmap includes: Based on the current frame predicted heatmap and the current frame true heatmap, determine the current frame deviation heatmap; Based on the current frame predicted heatmap and the previous frame ground truth heatmap corresponding to the previous frame human image sample, determine the previous frame deviation heatmap; Based on the predicted heatmap of the current frame and the ground truth heatmap of the subsequent frame corresponding to the human image sample of the subsequent frame, the deviation heatmap of the subsequent frame is determined; the frame interval between the human image sample of the subsequent frame and the human image sample of the previous frame is the preset time interval.

[0051] Specifically, after determining the predicted heatmap for the current frame, the difference between the predicted heatmap and the ground truth heatmap for the current frame is calculated; this difference is the current frame deviation heatmap. Simultaneously, the difference between the predicted heatmap and the ground truth heatmap of the previous frame is calculated; this difference is the previous frame deviation heatmap. Finally, the difference between the predicted heatmap and the ground truth heatmap of the next frame is calculated; this difference is the next frame deviation heatmap.

[0052] In one embodiment, determining the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame prediction heatmap includes: Based on a first preset threshold, the effective pixel points of the current frame corresponding to the current frame prediction heatmap, the effective pixel points of the previous frame corresponding to the previous frame prediction heatmap, and the effective pixel points of the subsequent frame corresponding to the subsequent frame prediction heatmap are determined respectively. Based on the effective pixels of the current frame, the effective pixels of the previous frame, and the effective pixels of the next frame, the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the next frame deviation heatmap are determined respectively.

[0053] Specifically, to improve the adaptability of the initial acupoint detection model, in this embodiment of the invention, based on a first preset threshold, the effective pixels of the current frame corresponding to the current frame prediction heatmap, the effective pixels of the previous frame corresponding to the previous frame prediction heatmap, and the effective pixels of the subsequent frame corresponding to the subsequent frame prediction heatmap are identified. The number of these effective pixels reflects the image quality or the effectiveness of the image content of their respective human image samples; that is, the more effective pixels there are, the better the image quality of the corresponding human image sample, or the higher the effectiveness of the image content. Then, based on the effective pixels of the current frame, the previous frame, and the subsequent frame, the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap are determined respectively.

[0054] In this embodiment of the invention, the weights corresponding to the deviation heatmaps of each frame are dynamically adjusted by the image quality or the effectiveness of the image content of different frames, which significantly improves the model's adaptability under complex lighting and pose conditions.

[0055] In one embodiment, determining the effective pixels of the current frame corresponding to the current frame prediction heatmap, the effective pixels of the previous frame corresponding to the previous frame prediction heatmap, and the effective pixels of the subsequent frame corresponding to the subsequent frame prediction heatmap based on a first preset threshold includes: All pixels in the current frame prediction heatmap with a confidence level greater than or equal to the first preset threshold are determined as valid pixels in the current frame. All pixels in the predicted heatmap of the previous frame with a confidence level greater than or equal to the first preset threshold are determined as valid pixels of the previous frame. All pixels in the predicted heatmap of the next frame with a confidence level greater than or equal to the first preset threshold are determined as valid pixels in the next frame.

[0056] Specifically, the first preset threshold is a threshold value for distinguishing between valid and invalid pixels. If the confidence level corresponding to a pixel position is greater than or equal to the first preset threshold, the initial acupoint detection model returns 1, and the pixel corresponding to that position is a valid pixel. If the confidence level corresponding to a pixel position is less than the first preset threshold, the initial acupoint detection model returns 0, and the pixel corresponding to that position is an invalid pixel and does not participate in the weight adaptive adjustment of the corresponding deviation heatmap. Based on the above judgment conditions, the pixel positions in the current frame prediction heatmap, the previous frame prediction heatmap, and the next frame prediction heatmap are traversed and identified to obtain all valid pixels of the current frame in the current frame prediction heatmap, all valid pixels of the previous frame in the previous frame prediction heatmap, and all valid pixels of the next frame in the next frame prediction heatmap.

[0057] In one embodiment, determining the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap based on the current frame valid pixels, the previous frame valid pixels, and the subsequent frame valid pixels includes: The total number of valid pixels is determined based on the number of pixels corresponding to the current frame's valid pixels, the previous frame's valid pixels, and the next frame's valid pixels. The ratio of the number of current pixels corresponding to the current frame's valid pixels to the total number of valid pixels is determined as the weight corresponding to the current frame's deviation heatmap. The ratio of the number of previous frame pixels corresponding to the previous frame effective pixels to the total number of effective pixels is determined as the weight corresponding to the previous frame deviation heatmap. The ratio of the number of pixels in the subsequent frame corresponding to the effective pixels in the subsequent frame to the total number of effective pixels is determined as the weight corresponding to the subsequent frame deviation heatmap.

[0058] Specifically, after determining all valid pixels in the current frame, the previous frame, and the next frame, the number of current pixels corresponding to all valid pixels in the current frame, the number of pixels in the previous frame corresponding to all valid pixels in the previous frame, and the number of pixels in the next frame corresponding to all valid pixels in the next frame are counted. The sum of these three counts is then calculated; this sum represents the total number of valid pixels. Next, the ratio of the current pixel count to the total number of valid pixels is calculated. This ratio represents the weight corresponding to the current frame deviation heatmap, and this weight is expressed as... ,in, This represents the weight corresponding to the current frame's deviation heatmap, where Ncurrent represents the current number of pixels and Ntotal represents the total number of valid pixels. The ratio of the number of pixels in the previous frame to the total number of valid pixels is calculated; this ratio is the weight corresponding to the previous frame's deviation heatmap, and this weight is expressed as... ,in, This represents the weight corresponding to the previous frame's deviation heatmap, where Nprev represents the number of pixels in the previous frame. The ratio of the number of pixels in the subsequent frame to the total number of valid pixels is calculated; this ratio is the weight corresponding to the subsequent frame's deviation heatmap, and this weight is expressed as... ,in, This represents the weight corresponding to the deviation heatmap of the next frame, and Nnext represents the number of pixels in the next frame.

[0059] It should be noted that, This ensures that the confidence range of each pixel remains [0,1], which will not affect the logic of subsequent NMS and other processes.

[0060] In this embodiment of the invention, the weights are dynamically adjusted based on the image quality of each frame, so that the model can make full use of high-quality information when detecting acupoints and avoid being misled by low-quality information.

[0061] In one embodiment, determining the sample balance focus loss function based on the current frame predicted heatmap and the current frame ground truth heatmap includes: When the prediction probability of each effective pixel in the current frame in the current frame prediction heatmap is equal to the second preset threshold, the first sub-loss function is determined based on the ground truth probability of the effective pixel in the current frame in the current frame ground truth heatmap and the first adjustment factor. If the prediction probability corresponding to each effective pixel in the current frame prediction heatmap is not equal to the second preset threshold, a second sub-loss function is determined based on the prediction probability, the true probability, the first adjustment factor, and the second adjustment factor corresponding to the effective pixel in the current frame; the first adjustment factor and the second adjustment factor are used to balance the attention of the initial acupoint detection model to positive and negative samples; The current frame deviation heatmap is determined based on the total number of key points, all first sub-loss functions, and all second sub-loss functions; the total number of key points is the number of valid pixels in the current frame with a prediction probability equal to a second preset threshold.

[0062] In the existing technology, the MSE loss function and the MAE loss function are generally used in the field of key point detection. The MSE loss function is a method to measure the difference between the predicted value and the true value, which is obtained by calculating the average of the squared differences between the predicted value and the true value. The MSE loss function is shown in equation (1), which is: .

[0063] Where N represents the number of key points. This represents the true coordinates of the i-th key point. This represents the predicted coordinates of the i-th keypoint. In heatmap regression, this is transformed into calculating the coordinates of all pixels within the heatmap area.

[0064] The MSE loss function provides a continuously differentiable loss calculation method, which helps the gradient descent algorithm find the global optimum. However, due to the use of squaring operations, the MSE loss function is more sensitive to larger errors, amplifying the impact of outliers on the final keypoint detection results. Figure 2 This is a schematic diagram of the regression curve of the MSE loss function provided by existing technology, such as... Figure 2 As shown, when outliers such as the two rightmost data points exist, the overall trend of the regression curve of the MSE loss function changes and does not conform to the distribution of most data, especially for keypoint detection with a large number of samples.

[0065] The expression for the MAE loss function is very similar to that for the MSE loss function. However, the MAE loss function uses the absolute difference, so the overall loss will not be significantly affected by the particularly large error of a certain sample. This makes the MAE loss function more robust when dealing with datasets containing outliers, suitable for samples with asymmetric distributions, and effectively reduces the risk of model overfitting.

[0066] However, the two loss functions mentioned above have limited ability to model complex nonlinear relationships. When the relationship between the target variable and the input features is not linear or simply additive, such as in the case of meridian detection in a therapeutic setting, these two loss functions cannot fully capture the true structure of the data. More importantly, these loss functions lack modeling for uncertainty. For human keypoint detection tasks based on heatmap regression, understanding the confidence level of the prediction results is crucial due to the large number of invalid samples and the NMS process.

[0067] Therefore, in this embodiment of the invention, the concept of positive and negative sample balance loss (Focal Loss) is adopted. Focal Loss fully considers the characteristics of positive samples, making it very suitable for human acupoint detection tasks with a large number of imbalanced samples, and applicable to the meridian therapy scenario corresponding to this embodiment of the invention. Focal Loss is a loss function designed specifically to solve the class imbalance problem and is widely used in classification tasks. By introducing an adjustable parameter θ to reduce the weight of easily classified samples (such as a large number of negative samples), the model focuses more on difficult-to-classify samples (such as a few positive samples). The Focal Loss is shown in equation (2), which is: .

[0068] in, This represents the weighting factor, with a value range of (0,1), and pt represents the prediction probability. It is the adjustment factor of the focal loss function. In this way, Focal Loss can reduce the impact of simple negative samples on gradient updates, enabling the model to learn the key features of positive samples more effectively.

[0069] In this embodiment of the invention, a sample balancing focus loss function is obtained by improving upon equation (2), thereby realizing the transformation from a classification task to a heatmap-based regression task. This sample balancing focus loss function is shown in equation (3), which is: .

[0070] Where FL represents the sample balancing focus loss function, This represents the predicted probability of a valid pixel in the current frame at coordinates (x, y) in the current frame prediction heatmap. This represents the ground truth probability of a valid pixel at coordinates (x, y) in the current frame's ground truth heatmap. Indicates the first regulating factor. Indicates the second regulating factor. This represents the number of valid pixels in the current frame whose predicted probability equals the second preset threshold, i.e., the number of complete keypoints. The second preset threshold is 1. By adjusting the first and second adjustment factors, the weight of easily classified negative samples can be reduced, allowing the model to focus more on difficult-to-classify positive samples.

[0071] Preferably, , This is to distinguish between a large number of negative samples and a small number of positive samples.

[0072] In addition to the sample balance focus loss function mentioned above, other loss functions based on binary classification, such as Cross Entropy Loss (CE Loss), Balanced Cross Entropy Loss (BCE Loss), and Focal Loss, can also be used.

[0073] (1) The binary classification-based cross-entropy loss function can solve the problems of the MSE loss function and the MAE loss function. This binary classification-based cross-entropy loss function can handle the probability distribution between multiple categories and ensure that the model can correctly assign each pixel to the correct key point. In the key point detection process based on heatmap, the binary classification-based cross-entropy loss function is shown in Equation (4), which is: .

[0074] Where N represents the number of key points that meet the confidence criteria after threshold filtering, i.e., the number of valid pixels.

[0075] (2) This binary cross-entropy loss function is used to distinguish whether a pixel belongs to a key point, or to distinguish whether the pixel belongs to the foreground or the background. A weighting factor is introduced based on the binary classification-based cross-entropy loss function calculation method shown in equation (4). When the current pixel is a positive sample and its value is ∈[0,1], the weight factor is... When the current pixel is a negative sample, the weight factor is 1- After relevant experimental verification, The optimal value is 0.75. The binary cross-entropy loss function is shown in equation (5), which is: .

[0076] Furthermore, to ensure the effectiveness of model training and the reliability of experimental results, this embodiment of the invention was conducted using the PyTorch 1.11.0 deep learning framework and CUDA 11.6 environment, with two NVIDIA RTX 3090 graphics cards selected as the hardware platform. The Adam optimizer was used during training, with an initial learning rate set to 1×10⁻⁶. -3 The training process involved 140 epochs of iterative training, with validation performed every 10 epochs. A MultiStepLR strategy was introduced to decay the learning rate at epochs 90 and 120 to optimize convergence. To address the algorithm's dependence on the temporal information of the video stream, the experiment constructed multi-channel input data containing the current frame and its preceding and following frames. Considering the slow motion characteristics in the conditioning scene, the sampling interval between preceding and following frames was set to 1-8 frames to effectively supplement the occluded spatial information. To maintain the continuity of the temporal dimension, the data shuffle setting was turned off during training, and the batch size was adjusted to 24. This maximized the utilization of hardware resources while ensuring that the model could fully capture the spatiotemporal context information over long periods.

[0077] In the task of detecting acupoints on the human body, the dataset usually contains a large number of keypoint samples. There is a significant class imbalance between the positive samples (i.e., dozens of acupoints) that ultimately need to be precisely located and the negative samples (tens of thousands of non-acupoints) discarded by the model. This class imbalance may cause the model to overemphasize negative samples, thus affecting the detection accuracy of positive samples. To address this issue, this embodiment of the invention uses Focal Loss as the loss function, effectively balancing the impact of positive and negative samples on model training. The comparative experimental results of training guided by different loss functions are shown in Table 1.

[0078] Table 1

[0079] As shown in Table 1, on the complex human acupoint dataset, Focal Loss, by focusing more on learning from difficult samples, enables the model to achieve a better fit in fewer iterations, thus accelerating the convergence speed. Furthermore, by effectively suppressing the interference of a large number of negative samples, Focal Loss helps the model better capture the subtle features of positive samples, thereby significantly improving the final detection accuracy.

[0080] In this embodiment of the invention, the detection accuracy of the model in occluded scenarios is further improved by introducing temporal information and a dynamic weight adjustment mechanism. Traditional PTM modules enhance the prediction of the current frame by combining heatmap information from preceding and following frames. However, due to the relatively slow movements of the human body and robotic arm, close intervals between frames can lead to redundant occlusion information, failing to effectively supplement the missing information in the current frame. Therefore, this embodiment of the invention introduces a temporal information utilization method based on PTM and dynamically adjusts the weights of preceding and following frames to ensure that the model can fully utilize temporal information, especially in occluded scenarios where it can infer the location of occluded acupoints through supplementary information from preceding and following frames. This strategy not only effectively solves the information redundancy problem of the PTM module in occluded scenarios but also further enhances the model's inference ability for occluded regions by introducing a longer temporal window (1-8 frames). The ablation experiment results of this module are shown in Table 2.

[0081] Table 2

[0082] In Table 2, HRNet-w48 represents a high-resolution network with a depth of 48 layers. As shown in Table 2, the embodiments of this invention achieved accuracies of 90.35% and 89.78% on the supine and prone datasets, respectively, representing an improvement of over 1% compared to the traditional PTM module. Furthermore, subsequent experiments further confirmed that the accuracy of the model obtained using this method is more than 10% higher than the default single-frame base model.

[0083] To further verify the effectiveness of the method in the embodiments of the present invention, several methods that have performed well in key point detection tasks in recent years were selected for experimental comparison. The comparison results are shown in Table 3.

[0084] Table 3

[0085] As shown in Table 3, although Openpose's method of using affinity fields to handle occluded keypoint detection is very effective and widely used in practical detection tasks, it performs poorly in detecting occluded meridian acupoints. The AP of the supine and prone models is only about 60%, presumably because the number of keypoints in the meridian dataset increases sharply compared to public datasets, leading to overfitting. HigherHRNet and MobileNet methods, which are lightweight and have been applied to various client and mobile detection applications, perform only moderately, with AP of models obtained from various datasets ranging from 70% to 80%. Even the similarly performing RTMPose method, proposed in recent years, still suffers from some serious false detections under occlusion conditions. The CID method proposes a keypoint decoupling concept, exhibiting high robustness in complex multi-person scenarios. It achieves an accuracy of over 80% for meridian acupoint datasets and largely avoids false detections, but its specificity is not as strong as the human meridian acupoint detection method provided in this embodiment. DCPose focuses on keypoint detection in video streams and is the first to introduce the concept of temporal information-guided training. Benefiting from the strong guidance of temporal characteristics on occlusion problems, this method achieves the best performance among many mainstream methods, reaching 88.7% accuracy on the supine dataset. However, as a keypoint detection framework for video streams, its input and output formats only support the posetrack format. Reproducing the method requires converting the common COCO format annotation files to posetrack format, which is inconvenient for practical project use.

[0086] In comparison, the method provided in this embodiment of the invention achieves an optimized model mAP of 90.35% (supine) and 89.78% (prone) on a meridian acupoint test set containing dynamic occlusion, representing a 1.7% improvement over DCPose and over 10% improvement over the single-frame base model, demonstrating the effectiveness of the method. Furthermore, the method provided in this embodiment of the invention starts with a single-frame base model, supports discrete image sequence input, and does not rely on structured video stream data. By decoupling temporal processing from the single-frame detection framework, it achieves enhanced real-time and accurate detection, laying the foundation for subsequent adaptation to the real-time inference needs of edge devices.

[0087] Comparative experiments have verified that, through personalized data enhancement for meridian detection, the method provided in this embodiment of the invention can significantly improve the model's detection accuracy for acupoints. Targeted data enhancement effectively simulates occlusion conditions and further improves model accuracy. Figure 3 This is a schematic diagram of the visualized detection results provided in an embodiment of the present invention, such as... Figure 3 As shown, even if a part of the human body is obscured and the relevant acupoints are not visible during the operation of the robotic arm, the model can still detect their location relatively accurately.

[0088] The human acupoint detection device provided by the present invention is described below. The human acupoint detection device described below can be referred to in correspondence with the human acupoint detection method described above.

[0089] This invention provides a human acupoint detection device. Figure 4 This is a schematic diagram of the structure of the human acupoint detection device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the human acupoint detection device 400 includes: an acquisition module 410 and a detection module 420.

[0090] The acquisition module 410 is used to acquire the human body image sequence corresponding to the user.

[0091] The detection module 420 is used to input the human image sequence into the acupoint detection model and obtain the acupoint detection result output by the acupoint detection model; the acupoint detection model is trained based on the sample balance focus loss function, the multi-frame heatmap corresponding to the human image sequence samples with a preset time interval, and the effective pixel points corresponding to the multi-frame heatmap.

[0092] The acupoint detection device provided in this invention trains an initial acupoint detection model using a sample-balanced focus loss function, multi-frame heatmaps corresponding to human image sequence samples at preset time intervals, and effective pixels corresponding to the multi-frame heatmaps. The human image sequence is then input into the acupoint detection model to obtain the acupoint detection results output by the model. In this invention, temporal information from consecutive frames is introduced through human image sequence samples at preset time intervals, and the weights corresponding to the multi-frame heatmaps are dynamically adjusted based on the image quality reflected by the effective pixels of the multi-frame heatmaps. Even if acupoints in the image are occluded, the acupoint coordinates can still be accurately inferred from the temporal information, significantly improving the accuracy of acupoint detection in occluded scenarios. Simultaneously, the sample-balanced focus loss function effectively reduces the weight of simple negative samples in gradient updates, allowing the model to focus on learning the acupoint features of difficult-to-classify positive samples, solving the problem of positive-negative sample imbalance, and further improving the accuracy of acupoint detection.

[0093] Optionally, the human acupoint detection device 400 also includes a training module, which is specifically used for: The human image sequence samples are input into the initial acupoint detection model to obtain the current frame prediction heatmap, the previous frame prediction heatmap, and the next frame prediction heatmap; the frame interval between the previous frame prediction heatmap and the next frame prediction heatmap is the preset time interval; The current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame prediction heatmap are determined respectively, as well as their respective weights; A comprehensive deviation heatmap is determined based on the current frame deviation heatmap, the previous frame deviation heatmap, the subsequent frame deviation heatmap, and their respective weights. Based on the current frame predicted heatmap and the current frame ground truth heatmap, determine the sample balance focus loss function; The initial acupoint detection model is trained based on the comprehensive deviation heatmap and the sample balance focus loss function to obtain the acupoint detection model.

[0094] Optionally, this training module is specifically used for: Based on the current frame predicted heatmap and the current frame true heatmap, determine the current frame deviation heatmap; Based on the current frame predicted heatmap and the previous frame ground truth heatmap corresponding to the previous frame human image sample, determine the previous frame deviation heatmap; Based on the predicted heatmap of the current frame and the ground truth heatmap of the subsequent frame corresponding to the human image sample of the subsequent frame, the deviation heatmap of the subsequent frame is determined; the frame interval between the human image sample of the subsequent frame and the human image sample of the previous frame is the preset time interval.

[0095] Optionally, this training module is specifically used for: Based on a first preset threshold, the effective pixel points of the current frame corresponding to the current frame prediction heatmap, the effective pixel points of the previous frame corresponding to the previous frame prediction heatmap, and the effective pixel points of the subsequent frame corresponding to the subsequent frame prediction heatmap are determined respectively. Based on the effective pixels of the current frame, the effective pixels of the previous frame, and the effective pixels of the next frame, the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the next frame deviation heatmap are determined respectively.

[0096] Optionally, this training module is specifically used for: All pixels in the current frame prediction heatmap with a confidence level greater than or equal to the first preset threshold are determined as valid pixels in the current frame. All pixels in the predicted heatmap of the previous frame with a confidence level greater than or equal to the first preset threshold are determined as valid pixels of the previous frame. All pixels in the predicted heatmap of the next frame with a confidence level greater than or equal to the first preset threshold are determined as valid pixels in the next frame.

[0097] Optionally, this training module is specifically used for: The total number of valid pixels is determined based on the number of pixels corresponding to the current frame's valid pixels, the previous frame's valid pixels, and the next frame's valid pixels. The ratio of the number of current pixels corresponding to the current frame's valid pixels to the total number of valid pixels is determined as the weight corresponding to the current frame's deviation heatmap. The ratio of the number of previous frame pixels corresponding to the previous frame effective pixels to the total number of effective pixels is determined as the weight corresponding to the previous frame deviation heatmap. The ratio of the number of pixels in the subsequent frame corresponding to the effective pixels in the subsequent frame to the total number of effective pixels is determined as the weight corresponding to the subsequent frame deviation heatmap.

[0098] Optionally, this training module is specifically used for: When the prediction probability of each effective pixel in the current frame in the current frame prediction heatmap is equal to the second preset threshold, the first sub-loss function is determined based on the ground truth probability of the effective pixel in the current frame in the current frame ground truth heatmap and the first adjustment factor. If the prediction probability corresponding to each effective pixel in the current frame prediction heatmap is not equal to the second preset threshold, a second sub-loss function is determined based on the prediction probability, the true probability, the first adjustment factor, and the second adjustment factor corresponding to the effective pixel in the current frame; the first adjustment factor and the second adjustment factor are used to balance the attention of the initial acupoint detection model to positive and negative samples; The current frame deviation heatmap is determined based on the total number of key points, all first sub-loss functions, and all second sub-loss functions; the total number of key points is the number of valid pixels in the current frame with a prediction probability equal to a second preset threshold.

[0099] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a human acupoint detection method. This method includes: acquiring a human image sequence corresponding to the user; inputting the human image sequence into an acupoint detection model to obtain the acupoint detection result output by the acupoint detection model; the acupoint detection model is trained based on a sample balanced focus loss function, multi-frame heatmaps corresponding to human image sequence samples at a preset time interval, and effective pixel points corresponding to the multi-frame heatmaps.

[0100] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the human acupoint detection method provided by the above methods. The method includes: acquiring a human image sequence corresponding to the user; inputting the human image sequence into an acupoint detection model to obtain the acupoint detection result output by the acupoint detection model; the acupoint detection model is trained based on a sample balance focus loss function, a multi-frame heatmap corresponding to human image sequence samples with a preset time interval, and effective pixel points corresponding to the multi-frame heatmap.

[0102] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the human acupoint detection method provided by the above methods. The method includes: acquiring a human image sequence corresponding to a user; inputting the human image sequence into an acupoint detection model to obtain the acupoint detection result output by the acupoint detection model; wherein the acupoint detection model is trained based on a sample balance focus loss function, a multi-frame heatmap corresponding to human image sequence samples with a preset time interval, and effective pixel points corresponding to the multi-frame heatmap.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting acupoints in the human body, characterized in that, include: Obtain the human body image sequence corresponding to the user; The human image sequence is input into the acupoint detection model to obtain the acupoint detection results output by the acupoint detection model; The acupoint detection model is trained based on a sample balance focus loss function, multi-frame heatmaps corresponding to human image sequence samples with preset time intervals, and effective pixel points corresponding to the multi-frame heatmaps.

2. The method for detecting acupoints in the human body according to claim 1, characterized in that, The acupoint detection model was trained based on the following steps: The human image sequence samples are input into the initial acupoint detection model to obtain the current frame prediction heatmap, the previous frame prediction heatmap, and the next frame prediction heatmap; the frame interval between the previous frame prediction heatmap and the next frame prediction heatmap is the preset time interval; The current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame prediction heatmap are determined respectively, as well as their respective weights; A comprehensive deviation heatmap is determined based on the current frame deviation heatmap, the previous frame deviation heatmap, the subsequent frame deviation heatmap, and their respective weights. Based on the current frame predicted heatmap and the current frame ground truth heatmap, determine the sample balance focus loss function; The initial acupoint detection model is trained based on the comprehensive deviation heatmap and the sample balance focus loss function to obtain the acupoint detection model.

3. The method for detecting acupoints in the human body according to claim 2, characterized in that, The step of determining the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame prediction heatmap includes: Based on the current frame predicted heatmap and the current frame true heatmap, determine the current frame deviation heatmap; Based on the current frame predicted heatmap and the previous frame ground truth heatmap corresponding to the previous frame human image sample, determine the previous frame deviation heatmap; Based on the predicted heatmap of the current frame and the ground truth heatmap of the subsequent frame corresponding to the human image sample of the subsequent frame, the deviation heatmap of the subsequent frame is determined; the frame interval between the human image sample of the subsequent frame and the human image sample of the previous frame is the preset time interval.

4. The method for detecting acupoints in the human body according to claim 2, characterized in that, Determine the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap corresponding to the current frame prediction heatmap, including: Based on a first preset threshold, the effective pixel points of the current frame corresponding to the current frame prediction heatmap, the effective pixel points of the previous frame corresponding to the previous frame prediction heatmap, and the effective pixel points of the subsequent frame corresponding to the subsequent frame prediction heatmap are determined respectively. Based on the effective pixels of the current frame, the effective pixels of the previous frame, and the effective pixels of the next frame, the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the next frame deviation heatmap are determined respectively.

5. The method for detecting acupoints in the human body according to claim 4, characterized in that, The step of determining the effective pixel points of the current frame corresponding to the current frame prediction heatmap, the effective pixel points of the previous frame corresponding to the previous frame prediction heatmap, and the effective pixel points of the subsequent frame corresponding to the subsequent frame prediction heatmap based on a first preset threshold includes: All pixels in the current frame prediction heatmap with a confidence level greater than or equal to the first preset threshold are determined as valid pixels in the current frame. All pixels in the predicted heatmap of the previous frame with a confidence level greater than or equal to the first preset threshold are determined as valid pixels of the previous frame. All pixels in the predicted heatmap of the next frame with a confidence level greater than or equal to the first preset threshold are determined as valid pixels in the next frame.

6. The method for detecting acupoints in the human body according to claim 4, characterized in that, The step of determining the weights corresponding to the current frame deviation heatmap, the previous frame deviation heatmap, and the subsequent frame deviation heatmap based on the current frame's valid pixel points, the previous frame's valid pixel points, and the subsequent frame's valid pixel points includes: The total number of valid pixels is determined based on the number of pixels corresponding to the current frame's valid pixels, the previous frame's valid pixels, and the next frame's valid pixels. The ratio of the number of current pixels corresponding to the current frame's valid pixels to the total number of valid pixels is determined as the weight corresponding to the current frame's deviation heatmap. The ratio of the number of previous frame pixels corresponding to the previous frame effective pixels to the total number of effective pixels is determined as the weight corresponding to the previous frame deviation heatmap. The ratio of the number of pixels in the subsequent frame corresponding to the effective pixels in the subsequent frame to the total number of effective pixels is determined as the weight corresponding to the subsequent frame deviation heatmap.

7. The method for detecting acupoints in the human body according to any one of claims 2-6, characterized in that, The step of determining the sample balance focus loss function based on the predicted heatmap and the ground truth heatmap of the current frame includes: When the prediction probability of each effective pixel in the current frame in the current frame prediction heatmap is equal to the second preset threshold, the first sub-loss function is determined based on the ground truth probability of the effective pixel in the current frame in the current frame ground truth heatmap and the first adjustment factor. If the prediction probability corresponding to each effective pixel in the current frame prediction heatmap is not equal to the second preset threshold, a second sub-loss function is determined based on the prediction probability, the true probability, the first adjustment factor, and the second adjustment factor corresponding to the effective pixel in the current frame; the first adjustment factor and the second adjustment factor are used to balance the attention of the initial acupoint detection model to positive and negative samples; The current frame deviation heatmap is determined based on the total number of key points, all first sub-loss functions, and all second sub-loss functions; the total number of key points is the number of valid pixels in the current frame with a prediction probability equal to a second preset threshold.

8. A human acupoint detection device, characterized in that, include: The acquisition module is used to acquire the human body image sequence corresponding to the user; The detection module is used to input the human image sequence into the acupoint detection model and obtain the acupoint detection results output by the acupoint detection model. The acupoint detection model is trained based on the sample balance focus loss function, the multi-frame heatmap corresponding to the human image sequence samples with a preset time interval, and the effective pixel points corresponding to the multi-frame heatmap.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the human acupoint detection method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the human acupoint detection method as described in any one of claims 1 to 7.

Images