Human leg acupoint detection method based on improved UNet + + network

Abstract

The invention discloses a human leg acupoint detection method based on an improved UNet + + network, and the method employs the improved UNet + + network, introduces a triple attention mechanism in an encoder part to enhance the extraction capability of acupoint spatial features, integrates a focus modulation module in a decoding path to achieve the modulation of multi-scale context information, and achieves the detection of acupoints. And feature representation is optimized through weighted fusion of the SE attention mechanism module and the CBAM attention mechanism module. A dense jump connection structure is adopted in the network, characteristics of an encoder and a decoder are fused on multiple scales, and finally end-to-end training is carried out through a weighted loss function. According to the method, the positions of the leg acupuncture points can be automatically and accurately detected, the problems that traditional Chinese medicine acupuncture point positioning depends on artificial experience and is high in subjectivity are solved, and an objective and efficient intelligent solution is provided for acupuncture point detection.

Description

Technical Field

[0001] This invention relates to the technical field of acupuncture point location in traditional Chinese medicine, and in particular to a method for detecting acupuncture points on the human leg based on an improved UNet++ network. Background Technology

[0002] In traditional Chinese medicine (TCM) acupoint diagnosis and treatment, accurate acupoint location is crucial. Traditional acupoint location mainly relies on the experience and techniques of physicians, which suffers from high subjectivity and poor repeatability. Existing acupoint detection technologies are mostly based on two-dimensional image processing, lacking three-dimensional spatial positioning capabilities and having insufficient adaptability to complex backgrounds and individual differences.

[0003] UNet++, as an advanced medical image segmentation network, performs excellently in biomedical image segmentation. However, its application in acupoint detection has not been fully explored, especially when dealing with small targets and inconspicuous features like acupoints. Traditional UNet++ networks struggle to effectively capture subtle features of acupoint regions when processing leg acupoint detection, and their utilization of multi-scale information is insufficient.

[0004] Therefore, there is an urgent need for a highly accurate, lightweight, and robust method for locating acupoints to improve the standardization of TCM diagnosis and treatment. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a method for detecting acupoints on the human leg based on an improved UNet++ network. This method improves the U-Net++ network structure, introduces an attention mechanism and lightweight design, enhances the ability to extract acupoint features, and achieves high-precision and high-efficiency acupoint localization.

[0006] To achieve the above objectives, the technical solution provided by this invention is as follows: a method for detecting acupoints on the human leg based on an improved UNet++ network. The improved UNet++ network is based on the original UNet++ network with the following improvements: First, a triple attention mechanism module is introduced in the encoding path, and interactive relationships are established in three dimensions: channel-width, height-channel, and height-width, to enhance feature representation capabilities. Second, in the decoding path, a focus modulation module is used to optimize the multi-scale features extracted by the encoder of the original UNet++ network. Third, in the skip connections and decoder nodes of the original UNet++ network, the SE attention mechanism module and the CBAM attention mechanism module are integrated, and a gating fusion strategy is adopted. The outputs of the SE attention mechanism module and the CBAM attention mechanism module are adaptively integrated through learnable weights to achieve optimized fusion of feature information from different path sources. The specific implementation of this method for detecting acupoints on the human leg includes the following steps: 1) Acquire multiple original images containing the human leg region. Perform preprocessing operations such as size standardization, contrast enhancement, and noise filtering on each original image to obtain a standardized image. Perform data augmentation operations including translation, rotation, and flipping on the multiple standardized images to increase data diversity. Use the bone measurement method to perform pixel-level annotation on the acupoint regions in the data-augmented standardized images to obtain images with acupoint location labels. All images with acupoint location labels constitute an acupoint detection dataset. 2) The acupoint detection dataset was input into the improved UNet++ network for training. During training, the input images were first downsampled using a pre-trained VGG16 encoder to extract multi-scale features. Subsequently, a triple attention mechanism module was applied to the deepest features output by the encoder. Through the attention mechanism established in the three dimensions of channel-width, height-channel, and height-width, the network's ability to perceive the global spatial context information of acupoints was enhanced, resulting in a feature-enhanced deep representation, i.e., deep features. These deep features, along with the shallow features from the encoder passed through skip connections, were input into the decoder. During the decoding process, a focus modulation module was used to adaptively modulate features at different scales. This method utilizes multi-scale deep separable convolutions to capture key contextual information. At each feature fusion node of the decoder, shallow features from skip connections are concatenated with deep features from the decoder. These features are then sequentially weighted by channel attention via the SE attention mechanism and by hybrid attention via the CBAM attention mechanism. Finally, a gated fusion strategy is used to adaptively integrate the outputs of both, achieving optimized fusion of multi-path feature information. The network aims to minimize a combined loss function that combines cross-entropy loss, Dice loss, and Focal loss. Backpropagation is used to continuously optimize the parameters until convergence, resulting in a trained improved UNet++ network used as a human leg acupoint detection model. 3) Input the image of the human leg to be detected into the human leg acupoint detection model to obtain the final detection result image with accurate acupoint annotations.

[0007] Furthermore, in step 1), the specific method of marking using the bone measurement method is as follows: On the leg image, draw a baseline line with the Dubi acupoint as the starting point and the Jiexi acupoint as the ending point, and regard this baseline line as fifteen inches; on this basis, determine the positions of other acupoints proportionally along the direction of the baseline line: the Zusanli acupoint is located three inches below the Dubi acupoint, the Shangjuxu acupoint is located six inches, the Tiaokou acupoint is located eight inches, and the Xiajuxu acupoint is located nine inches.

[0008] Furthermore, in step 2), the improved UNet++ network is described in detail below: a) A triple attention mechanism module is applied to the deepest features output by the encoder. This module establishes feature dependencies of different dimensions of leg acupoints through three parallel attention branches: The first branch transforms the input leg features from (B,C,H,W) to (B,H,C,W), compresses them in the height dimension, calculates channel-width attention weights, and focuses on capturing the distribution pattern of lateral acupoints in the leg; the second branch transforms the input leg features from (B,C,H,W) to (B,W,H,C), compresses them in the width dimension, calculates height-channel attention weights, and focuses on capturing the distribution pattern of vertical acupoints in the leg; the third branch directly compresses them in the channel dimension, calculates height-width spatial attention weights, and focuses on capturing the spatial positional relationship of leg acupoints; finally, the outputs of the three branches are restored to their original dimensions and then weighted and averaged to achieve multi-dimensional feature fusion of leg acupoints; where B is the batch size, C is the number of channels, H is the height, and W is the width. b. During the decoding process, a focus modulation module is used to adaptively modulate features at different scales. The execution steps of the focus modulation module for leg acupoint detection are as follows: the input leg features are passed through a linear projection layer to generate a query vector, context features, and gating weights; multiple depthwise separable convolutional layers of different scales are used to extract context features at each scale. The formula for calculating the scale is: ; In the formula, focal_window is the base window size, focal_factor is the scale growth factor, and k is the current level index; Subsequently, the context features at each scale are multiplied by the corresponding gating weights and then summed to obtain the aggregated context representation; the global average pooling features are calculated and multiplied by the gating weights, and added to the context representation; a modulator is generated through 1×1 convolution, and multiplied element-wise with the query vector to achieve acupoint feature modulation; finally, the modulated leg acupoint features are output after linear projection and Dropout layers. c. At the feature fusion node, the SE attention mechanism module is applied for channel attention weighting, and the CBAM attention mechanism module is applied for hybrid attention weighting. The outputs of the two are adaptively integrated using the following gated fusion formula: ; In the formula, gate represents the gate weight, SE_out represents the weighted output of the SE attention mechanism module, CBAM_out represents the weighted output of the CBAM attention mechanism module, and the gate weight is generated by a 1×1 convolutional layer activated by Sigmoid. The formula for calculating the gate weight is as follows: ; In the formula, [SE_out, CBAM_out] represents the concatenation of the channel attention features output by the SE attention mechanism module and the hybrid attention features output by the CBAM attention mechanism module in the channel dimension; Conv1×1 is a 1×1 convolutional layer used to learn the fusion weights of the two attention features and halve the number of channels; The Sigmoid activation function maps the convolution output to the [0,1] interval as gate weights; d. The loss function used for training is a weighted combination of losses for leg acupoint detection, with a total loss of Represented as: ; In the formula, L CE For cross-entropy loss, L Dice For Dice's loss, L Focal The Focal loss is used, with α, β, and γ as balancing weights to address the imbalance between positive and negative samples in leg acupoints.

[0009] Further, in step 3), the human leg image to be detected is input into the human leg acupoint detection model. After forward propagation, the model outputs a leg acupoint segmentation map. The post-processing of the leg acupoint segmentation map includes: first, performing binarization segmentation by setting a threshold to initially obtain the acupoint region; then, performing morphological opening operation to remove noise and smooth the region boundaries; finally, extracting the centroid coordinates of each acupoint through connected component analysis, marking the position on the image accordingly, and outputting the final detection result map with accurate acupoint annotations.

[0010] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. An improved UNet++ network is proposed to address the problem that the leg acupoint region occupies a small proportion in the image and its features are not obvious. A focus modulation module is introduced to adaptively aggregate multi-scale contextual information and focus on key features through a gating mechanism, making the network more sensitive to the detailed features of small targets such as leg acupoints, thereby improving the segmentation accuracy.

[0011] 2. By introducing a triple attention mechanism module, the model can simultaneously establish feature dependencies from multiple dimensions such as channel, height, and width, more comprehensively capturing the relative positions and distribution patterns between acupoints and enhancing feature representation capabilities.

[0012] 3. In the decoding path, the channel attention features of the SE attention mechanism module and the hybrid attention features of the CBAM attention mechanism module are adaptively integrated through a gating fusion mechanism. This avoids information redundancy or conflict that may be caused by simple splicing or addition, and enables the network to dynamically adjust the feature selection strategy according to the input content, thereby optimizing the final segmentation effect.

[0013] In summary, this invention significantly improves the accuracy and robustness of automatic detection of acupoints on the human leg while maintaining model efficiency, providing an effective technical means for the objectification and intelligentization of acupoint location in traditional Chinese medicine. Attached Figure Description

[0014] Figure 1 This is a flowchart of the improved UNet++ network in this embodiment of the invention; where Backbone represents the feature extraction backbone of the network; feat1-5 represent five downsampling stages respectively; reduce_conv(1×1) represents a 1×1 convolutional layer; x 4 ,0 This indicates the starting node of the decoder path; Triplet Attention indicates the triple attention mechanism module; UNet++Decoder indicates the original U-Net++ decoder.

[0015] Figure 2 This is a structural diagram of the focus modulation module in an embodiment of the present invention; in the diagram, q represents the query vector; ctx represents the context feature; and gate represents the gate weight.

[0016] Figure 3 This is a structural diagram of the triple attention mechanism module in an embodiment of the present invention; in the diagram, Permute represents changing the dimensional order and changing the observation angle; AttentionGate represents generating attention weights to highlight important areas; Identity represents keeping the original information unchanged to prevent information loss.

[0017] Figure 4 This is a schematic diagram of the gating fusion mechanism in an embodiment of the present invention; in the figure, SE_Block and CBAMLayer represent the SE attention mechanism module and the CBAM attention mechanism module, respectively; Sigmoid represents the activation function. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0019] like Figures 1 to 4As shown, this embodiment discloses a method for detecting acupoints on the human leg based on an improved UNet++ network. The improved UNet++ network is based on the original UNet++ network and has the following improvements: First, a triple attention mechanism module is introduced in the encoding path, and interactive relationships are established in three dimensions: channel-width, height-channel, and height-width, to improve feature representation capability. Second, in the decoding path, a focus modulation module is used to optimize the multi-scale features extracted by the encoder of the original UNet++ network. Third, in the skip connections and decoder nodes of the original UNet++ network, the SE attention mechanism module and the CBAM attention mechanism module are integrated, and a gating fusion strategy is adopted to adaptively integrate the outputs of the SE attention mechanism module and the CBAM attention mechanism module through learnable weights to achieve optimized fusion of feature information from different path sources. The specific implementation of this method for detecting acupoints on the human leg includes the following steps: 1) Acquire multiple original images containing the human leg region. Perform preprocessing operations such as size normalization, contrast enhancement, and noise filtering on each original image sequentially to obtain a normalized image. Perform data augmentation operations including translation, rotation, and flipping on the multiple normalized images to increase data diversity. Use the bone-weight measurement method to perform pixel-level annotation of acupoint regions in the data-augmented normalized images, obtaining images with acupoint location labels. All images with acupoint location labels constitute an acupoint detection dataset. The specific method of annotation using the bone-weight measurement method is as follows: On the leg image, draw a baseline line starting from the Dubi acupoint and ending at the Jiexi acupoint, and regard this baseline line as 15 inches. Based on this, determine the positions of other acupoints proportionally along the direction of this baseline line: Zusanli acupoint is located 3 inches below Dubi acupoint, Shangjuxu acupoint is located 6 inches below, Tiaokou acupoint is located 8 inches below, and Xiajuxu acupoint is located 9 inches below.

[0020] 2) The acupoint detection dataset was input into the improved UNet++ network for training. During training, the input images were first downsampled using a pre-trained VGG16 encoder to extract multi-scale features. Subsequently, a triple attention mechanism module was applied to the deepest features output by the encoder. Through the attention mechanism established in the three dimensions of channel-width, height-channel, and height-width, the network's ability to perceive the global spatial context information of acupoints was enhanced, resulting in a feature-enhanced deep representation, i.e., deep features. These deep features, along with the shallow features from the encoder passed through skip connections, were input into the decoder. During the decoding process, a focus modulation module was used to adaptively modulate features at different scales. This paper describes a method for detecting acupoints on the human leg using a multi-scale, deep separable convolutional network. At each feature fusion node of the decoder, shallow features from skip connections are concatenated with deep features from the decoder. These concatenated features are then weighted by channel attention via the SE attention mechanism and by hybrid attention via the CBAM attention mechanism. Finally, a gated fusion strategy adaptively integrates the outputs of both mechanisms, achieving optimized fusion of multi-path feature information. The network aims to minimize a combined loss function that combines cross-entropy loss, Dice loss, and Focal loss. Backpropagation is used to continuously optimize the parameters until convergence, resulting in a trained improved UNet++ network used as a model for detecting acupoints on the human leg. The improved UNet++ network is described in detail below: a) A triple attention mechanism module is applied to the deepest features output by the encoder. This module establishes feature dependencies of different dimensions of leg acupoints through three parallel attention branches: The first branch transforms the input leg features from (B,C,H,W) to (B,H,C,W), compresses them in the height dimension, calculates channel-width attention weights, and focuses on capturing the distribution pattern of lateral acupoints in the leg; the second branch transforms the input leg features from (B,C,H,W) to (B,W,H,C), compresses them in the width dimension, calculates height-channel attention weights, and focuses on capturing the distribution pattern of vertical acupoints in the leg; the third branch directly compresses them in the channel dimension, calculates height-width spatial attention weights, and focuses on capturing the spatial positional relationship of leg acupoints; finally, the outputs of the three branches are restored to their original dimensions and then weighted and averaged to achieve multi-dimensional feature fusion of leg acupoints; where B is the batch size, C is the number of channels, H is the height, and W is the width. b. During the decoding process, a focus modulation module is used to adaptively modulate features at different scales. The execution steps of the focus modulation module for leg acupoint detection are as follows: the input leg features are passed through a linear projection layer to generate a query vector, context features, and gating weights; multiple depthwise separable convolutional layers of different scales are used to extract context features at each scale. The formula for calculating the scale is: ; In the formula, focal_window is the base window size, focal_factor is the scale growth factor, and k is the current level index; Subsequently, the context features at each scale are multiplied by the corresponding gating weights and then summed to obtain the aggregated context representation; the global average pooling features are calculated and multiplied by the gating weights, and added to the context representation; a modulator is generated through 1×1 convolution, and multiplied element-wise with the query vector to achieve acupoint feature modulation; finally, the modulated leg acupoint features are output after linear projection and Dropout layers. c. At the feature fusion node, the SE attention mechanism module is applied for channel attention weighting, and the CBAM attention mechanism module is applied for hybrid attention weighting. The outputs of the two are adaptively integrated using the following gated fusion formula: ; In the formula, gate represents the gate weight, SE_out represents the weighted output of the SE attention mechanism module, CBAM_out represents the weighted output of the CBAM attention mechanism module, and the gate weight is generated by a 1×1 convolutional layer activated by Sigmoid. The formula for calculating the gate weight is as follows: ; In the formula, [SE_out, CBAM_out] represents the concatenation of the channel attention features output by the SE attention mechanism module and the hybrid attention features output by the CBAM attention mechanism module in the channel dimension; Conv1×1 is a 1×1 convolutional layer used to learn the fusion weights of the two attention features and halve the number of channels; The Sigmoid activation function maps the convolution output to the [0,1] interval as gate weights; d. The loss function used for training is a weighted combination of losses for leg acupoint detection, with a total loss of Represented as: ; In the formula, L CE For cross-entropy loss, L Dice For Dice's loss, L Focal The Focal loss is used, with α, β, and γ as balancing weights to address the imbalance between positive and negative samples in leg acupoints.

[0021] 3) Input the human leg image to be detected into the human leg acupoint detection model. After forward propagation, the model outputs a leg acupoint segmentation map. The post-processing of the leg acupoint segmentation map includes: first, performing binarization segmentation by setting a threshold to initially obtain the acupoint region; then, performing morphological opening operation to remove noise and smooth the region boundary; finally, extracting the centroid coordinates of each acupoint through connected component analysis, marking the position on the image accordingly, and outputting the final detection result map with accurate acupoint annotations.

[0022] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for detecting acupoints on the human leg based on an improved UNet++ network, characterized in that, The improved UNet++ network is based on the original UNet++ network and has the following improvements: First, a triple attention mechanism module is introduced into the encoding path, and interactive relationships are established in three dimensions: channel-width, height-channel, and height-width, to improve feature representation capabilities; Second, in the decoding path, the focus modulation module is used to optimize the multi-scale features extracted by the encoder of the original UNet++ network. Third, the SE attention mechanism module and the CBAM attention mechanism module are integrated into the skip connections and decoder nodes of the original UNet++ network. A gating fusion strategy is adopted to adaptively integrate the outputs of the SE attention mechanism module and the CBAM attention mechanism module through learnable weights, so as to achieve optimized fusion of feature information from different path sources. The specific implementation of this method for detecting acupoints on the human leg includes the following steps: 1) Acquire multiple original images containing the human leg region. Perform preprocessing operations such as size standardization, contrast enhancement, and noise filtering on each original image to obtain a standardized image. Perform data augmentation operations including translation, rotation, and flipping on the multiple standardized images to increase data diversity. Use the bone measurement method to perform pixel-level annotation on the acupoint regions in the data-augmented standardized images to obtain images with acupoint location labels. All images with acupoint location labels constitute an acupoint detection dataset. 2) The acupoint detection dataset was input into the improved UNet++ network for training. During training, the input images were first downsampled using a pre-trained VGG16 encoder to extract multi-scale features. Subsequently, a triple attention mechanism module was applied to the deepest features output by the encoder. Through the attention mechanism established in the three dimensions of channel-width, height-channel, and height-width, the network's ability to perceive the global spatial context information of acupoints was enhanced, resulting in a feature-enhanced deep representation, i.e., deep features. These deep features, along with the shallow features from the encoder passed through skip connections, were input into the decoder. During the decoding process, a focus modulation module was used to adaptively modulate features at different scales. This method utilizes multi-scale deep separable convolutions to capture key contextual information. At each feature fusion node of the decoder, shallow features from skip connections are concatenated with deep features from the decoder. These features are then sequentially weighted by channel attention via the SE attention mechanism and by hybrid attention via the CBAM attention mechanism. Finally, a gated fusion strategy is used to adaptively integrate the outputs of both, achieving optimized fusion of multi-path feature information. The network aims to minimize a combined loss function that combines cross-entropy loss, Dice loss, and Focal loss. Backpropagation is used to continuously optimize the parameters until convergence, resulting in a trained improved UNet++ network used as a human leg acupoint detection model. 3) Input the image of the human leg to be detected into the human leg acupoint detection model to obtain the final detection result image with accurate acupoint annotations.

2. The method for detecting acupoints on the human leg based on an improved U-Net++ network according to claim 1, characterized in that, In step 1), the specific method of marking using the bone measurement method is as follows: On the leg image, draw a baseline line with the Dubi acupoint as the starting point and the Jiexi acupoint as the ending point, and regard this baseline line as fifteen inches; on this basis, determine the positions of other acupoints proportionally along the direction of the baseline line: the Zusanli acupoint is located three inches below the Dubi acupoint, the Shangjuxu acupoint is located six inches, the Tiaokou acupoint is located eight inches, and the Xiajuxu acupoint is located nine inches.

3. The method for detecting acupoints on the human leg based on an improved U-Net++ network according to claim 1, characterized in that, In step 2), the improved UNet++ network is described in detail below: a) A triple attention mechanism module is applied to the deepest features output by the encoder. This module establishes feature dependencies of different dimensions of leg acupoints through three parallel attention branches: The first branch transforms the input leg features from (B,C,H,W) to (B,H,C,W), compresses them in the height dimension, calculates channel-width attention weights, and focuses on capturing the distribution pattern of lateral acupoints in the leg; the second branch transforms the input leg features from (B,C,H,W) to (B,W,H,C), compresses them in the width dimension, calculates height-channel attention weights, and focuses on capturing the distribution pattern of vertical acupoints in the leg; the third branch directly compresses them in the channel dimension, calculates height-width spatial attention weights, and focuses on capturing the spatial positional relationship of leg acupoints; finally, the outputs of the three branches are restored to their original dimensions and then weighted and averaged to achieve multi-dimensional feature fusion of leg acupoints; where B is the batch size, C is the number of channels, H is the height, and W is the width. b. During the decoding process, a focus modulation module is used to adaptively modulate features at different scales. The execution steps of the focus modulation module for leg acupoint detection are as follows: the input leg features are passed through a linear projection layer to generate a query vector, context features, and gating weights; multiple depthwise separable convolutional layers of different scales are used to extract context features at each scale. The formula for calculating the scale is: ； In the formula, focal_window is the base window size, focal_factor is the scale growth factor, and k is the current level index; Subsequently, the context features at each scale are multiplied by the corresponding gating weights and then summed to obtain the aggregated context representation; the global average pooling features are calculated and multiplied by the gating weights, and added to the context representation; a modulator is generated through 1×1 convolution, and multiplied element-wise with the query vector to achieve acupoint feature modulation; finally, the modulated leg acupoint features are output after linear projection and Dropout layers. c. At the feature fusion node, the SE attention mechanism module is applied for channel attention weighting, and the CBAM attention mechanism module is applied for hybrid attention weighting. The outputs of the two are adaptively integrated using the following gated fusion formula: ； In the formula, gate represents the gate weight, SE_out represents the weighted output of the SE attention mechanism module, CBAM_out represents the weighted output of the CBAM attention mechanism module, and the gate weight is generated by a 1×1 convolutional layer activated by Sigmoid. The formula for calculating the gate weight is as follows: ； In the formula, [SE_out, CBAM_out] represents the concatenation of the channel attention features output by the SE attention mechanism module and the hybrid attention features output by the CBAM attention mechanism module in the channel dimension; Conv1×1 is a 1×1 convolutional layer used to learn the fusion weights of the two attention features and halve the number of channels; The Sigmoid activation function maps the convolution output to the [0,1] interval as gate weights; d. The loss function used for training is a weighted combination of losses for leg acupoint detection, with a total loss of Represented as: ； In the formula, L CE For cross-entropy loss, L Dice For Dice's loss, L Focal The Focal loss is used, with α, β, and γ as balancing weights to address the imbalance between positive and negative samples in leg acupoints.

4. The method for detecting acupoints on the human leg based on an improved U-Net++ network according to claim 1, characterized in that, In step 3), the human leg image to be detected is input into the human leg acupoint detection model. After forward propagation, the model outputs a leg acupoint segmentation map. The post-processing of the leg acupoint segmentation map includes: first, performing binarization segmentation by setting a threshold to initially obtain the acupoint region; then, performing morphological opening operation to remove noise and smooth the region boundaries; finally, extracting the centroid coordinates of each acupoint through connected component analysis, marking the position on the image accordingly, and outputting the final detection result map with accurate acupoint annotations.

Images