Human back acupoint automatic recognition method and system based on key point detection

Abstract

The application discloses a human back acupoint automatic recognition method and system based on key point detection. The human back image to be analyzed is input into a trained target detection network to obtain acupoint recognition information. The trained target detection network comprises a Focus module and a plurality of C3STR modules. The Focus module is located at the first layer, performs slice reorganization and feature extraction on the input image, can finely process a local area, enhances the attention to the back and acupoint area, and makes the target detection network better capture key features. The C3STR module extracts features of the output image of the previous layer through a hierarchical window self-attention mechanism, effectively improves the feature extraction efficiency, and can more accurately locate the acupoint. Therefore, after the Focus module and the C3STR module are introduced, the target detection network can achieve good detection effect when detecting the back acupoint with high-density distribution and small target characteristics.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to an automatic identification method and system for acupoints on the human back based on key point detection. Background Technology

[0002] The location of acupoints on the human body is crucial in traditional Chinese medicine and other fields of physiotherapy. Acupoints on the back are closely related to the body's meridians and are of great significance for therapies such as massage. With the rapid development of artificial intelligence technology, automated acupoint recognition has become an important direction for improving the intelligence level of physiotherapy.

[0003] Deep learning has made significant progress in object detection and keypoint detection tasks. While the YOLO series of models are widely used, directly applying them to acupoint recognition on the human back still faces technical challenges, such as difficulty adapting to the distribution characteristics of acupoints and inferring unknown acupoints. Special optimization of the algorithm structure and data characteristics is needed. Object detection technology can locate specific parts of the human body, but its performance is poor for high-density acupoints and small target characteristics on the back, especially in complex backgrounds or when there are significant changes in human posture. Keypoint detection technology is mainly used for motion capture or pose estimation, but when dealing with highly densely distributed points like acupoints on the back, it is easily limited by data distribution and annotation accuracy. Furthermore, without considering the specific anatomical features of acupoint distribution, it is difficult to model and infer the positional relationships between acupoints.

[0004] Furthermore, there is currently a lack of research on human back acupoint datasets, and there is a lack of publicly available datasets specifically for human back acupoints. This scarcity of datasets leads to insufficient adaptability of existing models when detecting back acupoints.

[0005] Therefore, existing technologies still need improvement and development. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide an automatic identification method and system for human back acupoints based on key point detection, in order to address the above-mentioned deficiencies of the prior art. The aim is to solve the problem that the existing detection algorithms have poor detection performance when dealing with back acupoints with high density distribution and small target characteristics.

[0007] The technical solution adopted by this invention to solve the problem is as follows:

[0008] In a first aspect, embodiments of the present invention provide an automatic identification method for acupoints on the human back based on key point detection, the method comprising:

[0009] Obtain an image of the human back to be analyzed;

[0010] The human back image is input into a trained target detection network; wherein the trained target detection network includes a Focus module and several C3STR modules; the Focus module is located in the first layer and is used to slice and reconstruct the input image and extract features; the C3STR modules are used to extract features from the output image of the previous layer through a hierarchical window self-attention mechanism;

[0011] The acupoint recognition information corresponding to the human back image is obtained through the trained target detection network; wherein, the acupoint recognition information includes the back detection box, the location and category of key acupoints.

[0012] Secondly, embodiments of the present invention also provide an automatic identification system for acupoints on the human back based on key point detection, the system comprising:

[0013] The image acquisition module is used to acquire images of the human back to be analyzed.

[0014] An image input module is used to input the human back image into a trained target detection network; wherein, the trained target detection network includes a Focus module and several C3STR modules; the Focus module is located in the first layer and is used to slice and reconstruct the input image and extract features; the C3STR modules are used to extract features from the output image of the previous layer through a hierarchical window self-attention mechanism;

[0015] The image detection module is used to obtain acupoint recognition information corresponding to the human back image through the trained target detection network; wherein, the acupoint recognition information includes a back detection box, the location and category of key acupoints.

[0016] Thirdly, embodiments of the present invention also provide a terminal, the terminal including a memory and one or more processors; the memory stores one or more programs; the programs include instructions for executing the automatic identification method for human back acupoints based on key point detection as described above; the processor is used to execute the programs.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a plurality of instructions, which are adapted to be loaded and executed by a processor to implement the steps of the automatic identification method for human back acupoints based on key point detection as described above.

[0018] The beneficial effects of this invention are as follows: This embodiment introduces a Focus module and a C3STR module into the target detection network. The Focus module refines local regions, enhancing attention to the back and acupoint areas, enabling the target detection network to better capture key features. The C3STR module effectively improves feature extraction efficiency by introducing a spatial attention mechanism, enabling more accurate acupoint localization. With the introduction of the Focus and C3STR modules, the target detection model achieves better detection results when detecting acupoints on the back with high density distribution and small target characteristics. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the automatic identification method for acupoints on the human back based on key point detection provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the model training process provided in an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the data enhancement combination provided in an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of Mosaic enhancement provided in an embodiment of the present invention.

[0024] Figure 5 This is a diagram of the improved target detection network structure provided in the embodiments of the present invention.

[0025] Figure 6 This is a Focus network structure diagram provided in an embodiment of the present invention.

[0026] Figure 7 This is a network structure diagram of the C3STR module provided in an embodiment of the present invention.

[0027] Figure 8 This is a diagram of the Swing Transformer Layer network structure provided in an embodiment of the present invention.

[0028] Figure 9 This is a diagram illustrating the model prediction effect provided in an embodiment of the present invention.

[0029] Figure 10This is a schematic diagram of a module of an automatic acupoint recognition system for the human back based on key point detection provided in an embodiment of the present invention.

[0030] Figure 11 This is a schematic diagram of the terminal provided in the embodiment of the present invention. Detailed Implementation

[0031] This invention discloses an automatic identification method and system for human back acupoints based on key point detection. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention. Unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. “Comprising” as used herein means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The term “and / or” as used herein includes all or any unit and all combinations of one or more associated listed items.

[0032] To address the aforementioned shortcomings of existing technologies, this invention provides an automatic identification method for acupoints on the human back based on key point detection, such as... Figure 1 As shown, the method specifically includes the following steps:

[0033] Step S100: Obtain the image of the human back to be analyzed.

[0034] Specifically, the first step is to acquire an image of the human back. This can be done using an RGBD depth camera or other image acquisition devices, capturing the image from directly above the human body.

[0035] Step S200: Input the human back image into a trained target detection network; wherein, the trained target detection network includes a Focus module and several C3STR modules; the Focus module is located in the first layer and is used to slice and reconstruct the input image and extract features; the C3STR module is used to extract features from the output image of the previous layer through a hierarchical window self-attention mechanism.

[0036] Specifically, an image of the human back is input into a trained object detection network (or object detection model). To improve model performance, Focus and C3STR modules are further introduced, enhancing the attention to local regions and spatial feature extraction capabilities, respectively. The Focus module, located in the first layer, slices the input image in height and width and stitches them together in the channel dimension, achieving the conversion of spatial information to channel information while preserving all pixel information, which helps improve the efficiency and accuracy of subsequent feature extraction. The C3STR module combines convolutional neural networks and self-attention mechanisms, extracting features from the output image of the previous layer through a layered window self-attention mechanism. This effectively captures key points and global information in the image, enhancing the model's ability to detect objects at different scales. In this embodiment, the Focus and C3STR modules are added to the object detection network. The Focus module achieves efficient downsampling while maintaining information integrity, while the C3STR module effectively integrates the advantages of traditional convolution and modern Transformers, allowing the object detection network to significantly reduce the number of model parameters while maintaining computational efficiency, ensuring a balance between performance and computational resources.

[0037] For example, the basic architecture of an object detection network can be YOLOv11. YOLOv11 is a high-efficiency version of the YOLO series, redefining the performance boundaries of real-time object detection through architectural optimization and functional expansion. Based on optimizations of YOLOv9 and YOLOv10, it introduces an improved feature extraction module, enhancing the ability to capture image details, especially performing better in complex scenes. It adopts a lightweight design, achieving higher computational efficiency by reducing the number of parameters while maintaining accuracy. More post-processing optimization methods are added, improving the model's inference speed. Through multi-dimensional optimization, YOLOv11 sets a new benchmark in speed, accuracy, and functional versatility, becoming a widely adopted general-purpose vision model in industry and academia.

[0038] Step S300: Obtain acupoint recognition information corresponding to the human back image through the trained target detection network; wherein, the acupoint recognition information includes a back detection box, the location and category of key acupoints.

[0039] Specifically, with the help of the Focus and C3STR modules, the network extracts image features layer by layer, gradually abstracting richer feature representations from low to high levels. Predicting the extracted feature maps yields acupoint recognition information, including three types of information: back bounding boxes, the location and category of key acupoints. The back bounding boxes are used to locate the back region, the location of key acupoints is used to precisely locate the acupoints, and the category is used to distinguish different acupoints.

[0040] In one implementation, the method for constructing the training dataset of the trained object detection network includes:

[0041] Several raw images of the human back were acquired using a depth camera;

[0042] For each original human back image, an enhanced image corresponding to the original human back image is obtained through data augmentation operations, and annotation data corresponding to the enhanced image is generated.

[0043] A dataset is generated based on all the enhanced images and the labeled data, and a training dataset is obtained from the dataset.

[0044] Specifically, depth cameras can capture images and simultaneously acquire depth information, providing depth information for each pixel in the image, which is crucial for object detection tasks. In practical applications, depth cameras are used to acquire several raw human back images as the basis for subsequent processing. These raw human back images are unprocessed data and may contain noise, lighting variations, background interference, and other issues. Data augmentation brings significant advantages to object detection models, especially in enhancing their generalization ability, improving small object detection capabilities, enhancing adaptability to complex backgrounds, improving robustness to changes in target position and scale, and reducing overfitting. By appropriately using different data augmentation methods, the performance of object detection models can be effectively improved, especially when facing variable real-world environments and limited training data. For each raw human back image, one or more data augmentation operations are applied to generate one or more augmented images. Annotated data includes, but is not limited to, back detection boxes, and the location and category of key acupoints. In practical applications, if the labeled data is generated before the data augmentation operation, the labeled data needs to be transformed synchronously during the data augmentation operation to ensure consistency between the augmented images and the labeled data. The dataset is a collection of augmented images and corresponding labeled data. The dataset is divided into training, validation and test sets to evaluate the model's performance at different stages.

[0045] For example, such as Figure 2As shown, data acquisition utilized a Deptrum Aurora 930 series RGBD depth camera, capturing images from directly above the human body to ensure a full back view and capture details of the back. A total of 1500 images were acquired, covering a variety of body types, skin tones, poses, and lighting conditions. During annotation, in addition to marking the overall detection bounding box of the back, 27 key acupoints were precisely labeled, ensuring high accuracy and consistency in the location of each acupoint. To improve annotation accuracy, depth information was incorporated to ensure correct identification of the three-dimensional spatial location of the back. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio to ensure the effectiveness of model training and the comprehensiveness of validation. In practical applications, model training employed a deep convolutional neural network architecture, using an SGD optimizer for parameter tuning. For the acupoint recognition task, a multi-task learning strategy was designed to ensure that the model could not only detect the back region but also accurately locate the 27 acupoints. After training, the model performed well in terms of accuracy, recall, and mAP through evaluation on the validation and test sets, achieving the design goals.

[0046] In one implementation, the data augmentation operation includes at least one of the following: random cropping, random rotation, horizontal flipping, brightness adjustment, image merging, and depth map enhancement.

[0047] In real-world scenarios, targets are often intertwined with complex backgrounds. To enhance the model's generalization ability and help it better adapt to such complex backgrounds, reducing background interference with detection results, this embodiment employs various data augmentation techniques during training. This allows the model to access diverse training data, enabling it to perform better inference when encountering unfamiliar images and improving its generalization ability across different scenarios. Figure 3 As shown, this embodiment combines various data augmentation techniques, including random cropping, rotation, flipping, brightness adjustment, and Mosaic technology. It also incorporates depth map enhancement techniques, such as adding noise and depth distortion, to improve the model's generalization ability and enable stable performance in complex scenes. Furthermore, it can simulate back images under different environments, further enhancing the model's robustness to target space transformations and its performance in complex scenes. When training data is limited, data augmentation effectively expands the training set by generating diverse samples, reducing the risk of overfitting.

[0048] For example, consider random cropping: Random cropping involves randomly cropping a small region from the original image to simulate targets of different sizes and locations. This data augmentation operation increases the model's robustness to changes in target location. Assume the original image is W×H, and a sub-image of size w×h is cropped from it. The mathematical formula for cropping can be expressed as:

[0049] I crop (x,y)=I(x+Δx,y+Δy);

[0050] Where I(x,y) is the value of pixel (x,y) in the original image; Δx and Δy are the cropping offsets, which are randomly selected.

[0051] For random rotation: Random rotation involves rotating an image within a certain angular range. By rotating, the target in the image can be trained at different angles, enhancing the model's rotation invariance. Assuming the rotation angle is θ, the rotated image I... rot The pixel value can be calculated using the following transformation:

[0052]

[0053] Where (x′,y′) are the coordinates after rotation, (x,y) are the coordinates before rotation, and θ is the rotation angle.

[0054] For horizontal flipping: Horizontal flipping enhances the model's ability to learn the left-right symmetry of objects by reversing the image horizontally. The expression for horizontal flipping is:

[0055] I flip (x,y)=I(Wx,y);

[0056] Where W is the width of the image, x is the x-coordinate of the current pixel, and I(x,y) is the pixel value of the original image.

[0057] Regarding brightness adjustment: Brightness adjustment simulates image performance under different lighting conditions by weighting the pixel values ​​of the image. The brightness adjustment formula is as follows:

[0058] I bright (x,y)=I(x,y)×α+β;

[0059] Where α and β are randomly selected coefficients that control the scaling factor and offset of the brightness, respectively, and I(x,y) is the pixel value of the original image.

[0060] Mosaic enhancement: The basic idea of ​​the Mosaic technique is to stitch multiple images together at random positions to form a new large image. This increases the diversity of training samples and allows target objects to appear at different proportions and positions in the image, thereby improving the model's ability to detect targets under various backgrounds and scales (e.g., ...). Figure 4 (As shown). The Mosaic process specifically includes: random cropping: randomly cropping a region from each original image; the size of the cropped region will generally vary randomly (the cropping size can be adjusted as needed); image stitching: stitching the cropped image patches into a new large image in a certain way. Assuming four images are stitched together using a 2x2 grid, the size of the stitched image will be:

[0061] W′=W1+W2, H′=H1+H2;

[0062] Where W1, W2, H1, and H2 are the width and height of each stitched image.

[0063] Target annotation update: Because the image position has changed, the coordinates of the target objects need to be adjusted accordingly. This adjustment can be achieved by calculating the relative position of each target in the stitched image.

[0064] In one implementation, to further improve the efficiency and accuracy of model training, this embodiment provides several more efficient training methods / strategies:

[0065] By employing adaptive learning rates (such as AdamW, RAdam, and other optimization algorithms), the learning rate is automatically adjusted based on the model's performance during training. This helps to achieve rapid convergence in the early stages of training while avoiding overfitting in the later stages, thereby improving the model's stability and generalization ability.

[0066] Alternatively, a multi-stage training approach can be adopted. First, coarse training is performed using a large dataset and strong data augmentation strategies, followed by refined training on a smaller target dataset. This training strategy allows the model to gradually adapt to different training tasks, avoiding premature entrapment in local optima and thus improving overall training performance.

[0067] In one implementation, the target detection network includes a backbone network, an intermediate network, and a detection head module. The specific structure and function of each part are as follows:

[0068] The backbone network includes a series of cascaded Focus modules, several feature extraction modules, a spatial pyramid pooling fast module, and a cross-stage local self-attention module; each feature extraction module includes a standard convolution module and a C3STR module; the backbone network is used to extract features from the input image to obtain an initial feature map.

[0069] Specifically, regarding the Focus module: For computational efficiency, this embodiment introduces a Focus module in the first layer of the object detection network. Traditional downsampling methods (such as max pooling and average pooling) lose image information. The Focus module retains all pixel information through recombination operations while simultaneously achieving downsampling, reducing the computational load of subsequent network layers and improving overall efficiency. Spatial information is converted into channel information, which is beneficial for feature extraction. Regarding the feature extraction module: Each module includes a standard convolutional module (Conv module) and a C3STR module. The standard convolutional module contains convolutional layers, batch normalization, and activation functions, and is the most basic feature extraction unit in the network. The C3STR module effectively improves the efficiency of feature extraction by introducing a spatial attention mechanism, especially in complex backgrounds and pose variations, enabling more accurate acupoint localization. Regarding the Spatial Pyramid Pooling Fast Module (SPPF module): This module uses multiple pooling kernels of different sizes to extract multi-scale features, enhancing the model's ability to detect objects at different scales. For the Cross-Stage Local Self-Attention Module (C2PSA Module): This module combines the advantages of cross-stage local networks and self-attention mechanisms, enabling the model to capture contextual information more effectively across multiple layers, thereby improving object detection accuracy.

[0070] For example, such as Figure 5 As shown, the specific structure of the backbone network includes: a Focus module in the first layer; four feature extraction modules are cascaded after the Focus module, each of which includes a standard convolutional module (Conv module) and a C3STR module; an SPPF module is connected after the fourth feature extraction module; and a C2PSA module is connected after the SPPF module.

[0071] The intermediate network (Neck) includes a plurality of first feature extraction modules and a plurality of second feature extraction modules cascaded in sequence. Each first feature extraction module includes a cascaded upsampling module, a concatenation module, and a C3STR module. The input image of the concatenation module in the first feature extraction module includes the output image of the previous layer and the output image of a specific C3STR module in the backbone network. Each second feature extraction module includes a cascaded standard convolution module and a concatenation module, and adjacent second feature extraction modules are connected through a C3STR module. The input image of the concatenation module in the second feature extraction module includes the output image of the previous layer and the output image of a specific C3STR module in the first feature extraction module, or the output image of the previous layer and the output image of the cross-stage local self-attention module. The intermediate network is used to perform feature enhancement and fusion on the initial feature map output by the backbone network to obtain a target feature map.

[0072] Specifically, such as Figure 5 As shown, this embodiment divides the intermediate network into two modules: a first feature extraction module and a second feature extraction module. Adjacent second feature extraction modules are connected by a C3STR module.

[0073] For the first feature extraction module: The first feature extraction modules are cascaded sequentially. Each first feature extraction module includes a cascaded upsampling module, a concatenation module, and a C3STR module. The upsampling module upsamples (enlarges) the input feature map to increase its resolution. The concatenation module in the first feature extraction module concatenates the output image of the previous layer with the output image of a specific C3STR module in the backbone network. A specific C3STR module in the backbone network can be selected based on the size of the output image of the previous layer, thereby fusing feature information from different levels to enhance feature expressiveness. The C3STR module in the first feature extraction module further extracts features from the concatenated feature map output by the concatenation module. For the second feature extraction module: The second feature extraction modules follow the first feature extraction modules and are also cascaded sequentially. Each second feature extraction module includes a cascaded standard convolution module and a concatenation module. The standard convolution module extracts features from the input feature map. The stitching module in the second feature extraction module is used to stitch the output image of the previous layer with the output image of a specific C3STR module in the first feature extraction module, or to stitch the output image of the previous layer with the output image of a cross-stage local self-attention module, to enhance the diversity and expressive power of features. The C3STR module in the second feature extraction module is used to connect two adjacent second feature extraction modules, further enhancing feature fusion.

[0074] For example, such as Figure 5 As shown, the specific structure of the intermediate network includes two cascaded first feature extraction modules. Each first feature extraction module includes a cascaded standard convolutional module and a concatenation module. A second feature extraction module is connected to the second first feature extraction module, which in turn is connected to a C3STR module, which is then connected to another second feature extraction module. Each second feature extraction module includes a cascaded standard convolutional module and a concatenation module.

[0075] The detection head module is used to predict the target feature map output by the intermediate network to obtain the acupoint recognition information.

[0076] Specifically, the detection head module is the last layer of the object detection network. It is used to predict the target feature map output by the intermediate network and generate the final prediction result, including the object's category, location, and key points.

[0077] For example, such as Figure 5 As shown, the detection head module fuses information from three different feature levels for bounding box detection (target location, target category confidence) and keypoint information (keypoint coordinates, keypoint visibility). All necessary information is output in a single forward propagation.

[0078] In one implementation, the Focus module includes:

[0079] The slicing module is used to slice the input image in the length and width dimensions to obtain sub-slice images;

[0080] The stitching module is used to stitch the sub-slice images together along the channel dimension to obtain a stitched image;

[0081] A standard convolutional module is used to extract features from the stitched image to obtain the feature map output by the Focus module.

[0082] Specifically, the Focus module's structure includes a slicing module, a stitching module, and a standard convolutional module. The data flow within the Focus module is as follows: the slicing module receives the feature map input to the Focus module and slices it along both the length (height) and width (width) dimensions to obtain multiple sub-slice images. Slicing divides a large image into smaller images, reducing the complexity of subsequent processing and improving computational efficiency. The stitching module receives the multiple sub-slice images output from the slicing module and stitches them together along the channel dimension to form a stitched image, further enriching the image's feature representation. The standard convolutional module receives the stitched image output from the stitching module and uses convolution operations to extract features from it, ultimately generating the final feature map output by the Focus module.

[0083] For example, such as Figure 6 As shown, in the Focus module, each input feature map is first sliced ​​along the H and W dimensions, then concatenated along the channel dimension, then passed through a standard convolutional module (Conv), and finally the feature image is output.

[0084] The input tensor dimension transformation process is as follows:

[0085] Assume the input is: x(B,C,H,W); after slicing: x(B,4C,H / 2,W / 2).

[0086] The results of the slicing operations are as follows: Slice 1: x1 = x[...,::2,::2]; Slice 2: x2 = x[...,1::2,::2]; Slice 3: x3 = x[...,::2,1::2]; Slice 4: x4 = x[...,1::2,1::2]. Where, ... indicates that the first two dimensions (B and C) remain unchanged; ::2 indicates that a value is taken every two elements along that dimension (step size of 2); x[...,::2,::2] indicates that elements with even indices in both height and width are selected; x[...,1::2,::2] indicates that elements with odd indices in height and even indices in width are selected; x[...,::2,1::2] indicates that elements with even indices in height and odd indices in width are selected; x[...,1::2,1::2] indicates that elements with odd indices in both height and width are selected.

[0087] The calculation formula for the splicing operation is as follows: x cat =Concat([1,xx2,x3,x4],dim=1);

[0088] The formula for calculating convolution is as follows: y = Conv(x) cat).

[0089] In this embodiment, the Focus module achieves downsampling without information loss through clever slicing and recombination operations, converting spatial dimension information into channel dimension, increasing feature richness, improving computational efficiency, and reducing the computational load of subsequent processing.

[0090] In one implementation, the C3STR module includes:

[0091] The first standard convolutional module is used to extract features from the input image to obtain the first feature map;

[0092] The second standard convolutional module is used to extract features from the input image to obtain a second feature map;

[0093] The Swin Transformer module is used to extract features from the first feature map using a hierarchical window self-attention mechanism to obtain a third feature map;

[0094] The splicing module is used to splice the second feature map and the third feature map to obtain a spliced ​​feature map;

[0095] The third standard convolutional module is used to extract features from the concatenated feature map to obtain the feature map output by the C3STR module.

[0096] Specifically, acupoints on the back (such as Feishu and Xinshu) typically exhibit small (approximately 1-3 cm in diameter) and similar texture features, and their distribution follows the meridian pathways (e.g., the Bladder Meridian runs longitudinally along the back). Traditional CNNs struggle to capture such cross-regional connections. To achieve fine-grained feature extraction and enhance robustness against interference, this embodiment introduces the SwinTransformer module into the C3STR module. The SwinTransformer module's window self-attention mechanism can focus on locally high-response areas. For example, by calculating attention weights for skin surface convexities and color differences (such as redness and swelling), it enhances sensitivity to acupoint boundaries. Secondly, the SwinTransformer module's global context modeling can distinguish acupoint features from non-target areas, effectively addressing interference in real-world scenarios (such as scars, birthmarks, and clothing occlusion): for example, by comparing the smoothness differences of surrounding skin to suppress interference from occlusions.

[0097] like Figure 7As shown, the specific structure of the C3STR module includes three standard convolutional modules, one SwinTransformer module, and one concatenation module. The data flow within the C3STR module is as follows: the first standard convolutional module receives the feature map input to the C3STR module, performs further feature extraction, and outputs a first feature map; the second standard convolutional module and the Swin Transformer module each receive the first feature map and perform further feature extraction, with the second standard convolutional module outputting a second feature map and the Swin Transformer module outputting a third feature map. The concatenation module receives the second and third feature maps, performs feature concatenation / fusion, and outputs a concatenated feature map; the third standard convolutional module receives the concatenated feature map and performs further feature extraction to obtain the final feature map output by the C3STR module. The three standard convolutional modules perform basic feature extraction, while the Swin Transformer Block uses a window multi-head self-attention mechanism to calculate attention weights within a local window and achieves cross-window information interaction through window shifting, taking into account both global and local features, significantly improving the model's ability to capture complex textures and long-distance dependencies. In summary, the C3STR in this embodiment interacts through a hierarchical window: it captures local textures (such as the arrangement of pores around a single acupoint) in the shallow network; and it establishes topological relationships between acupoint groups (such as the regularity of the spacing between adjacent acupoints) in the deep network, which can help correct positioning deviations caused by individual body shape differences.

[0098] In one implementation, the Swin Transformer module includes a plurality of SwinTransformer layers cascaded in sequence.

[0099] Specifically, such as Figure 7 As shown, the Swing Transformer Block is composed of multiple Swing Transformer Layers cascaded sequentially. Figure 8As shown, the structure of each Swin Transformer Layer includes: a first channel dimension normalization layer (Layer Norm), a multi-stage window partitioning module (Window Partition), a position embedding module (PositionEmbedding), a windowed multi-head self-attention module, a window fusion module (Window Reverse), a first fusion module, a second channel dimension normalization layer, a multilayer perceptron (MLP), and a second fusion module. The data flow in each Swin Transformer Layer is as follows: The first channel dimension normalization layer receives the initial feature map input to the Swin Transformer Layer and normalizes its channel dimensions to obtain the first normalized data. The multi-stage window partitioning module receives the first normalized data and performs multi-stage window partitioning, gradually expanding the receptive field to adapt to the multi-scale characteristics of acupoint regions. The position embedding module receives the partitioned image windows output by the multi-stage window partitioning module and explicitly models the spatial relative positional relationships between acupoints, solving the problem of traditional CNNs being sensitive to geometric transformations. The window fusion module receives the image windows with modeled spatial relative positional relationships and fuses the various image windows. The first fusion module receives the initial feature map input to the Swin Transformer Layer and the image window fusion data output by the window fusion module, and outputs the first fused data. The second channel dimension normalization layer receives the first fused data and normalizes its channel dimensions to obtain the second normalized data. The second normalized data passes through a multilayer perceptron and enters the second fusion module. At the same time, the first fused data output by the first fusion module also enters the second fusion module. The two types of data are merged to obtain the output data of the Swin Transformer Layer.

[0100] Furthermore, the Window Multi-Head Self Attention (W-MSA) in the Swin Transformer can solve the problem of high computational complexity in high-resolution image processing using traditional Transformers. The specific calculation process of W-MSA includes:

[0101] Query, Key, Value Calculation: Q = XW q K = XW k V = XW v ;

[0102] Wherein, Query(Q): represents the information to be queried at the current position; Key(K): represents the key information of each position, used to match the Query; Value(V): represents the actual information content contained in each position; Wq represents the query weight matrix; Wk represents the key weight matrix; Wv represents the value weight matrix.

[0103] Attention weight calculation:

[0104] Where T denotes the transpose of the matrix; d k The dimension of the K matrix is ​​represented by B; B represents the relative position encoding.

[0105] Relative position code: B i,j =B[(ij) x ,(ij) y ];

[0106] Among them, B i，j It represents the relative positional relationship between position i and position j in the feature map. It is a learnable bias term used to enhance the attention mechanism's perception of positional information.

[0107] The final MSA output: MSA(X) = [head1; head2; ...; head...] h W O ;

[0108] Among them, head h W represents the h-th attention head; O This represents the corresponding weight matrix.

[0109] The overall calculation process of the C3STR module: Y1 = Conv1(X); Y2 = Conv2(X); Y STR =SwinTransformer(Y1); Output=Conv3(Concat[Y STR ,Y2]);

[0110] Where Conv1 represents the first standard convolutional module, and Y1 represents the first feature map; Conv2 represents the second standard convolutional module, and Y2 represents the second feature map; Conv3 represents the third standard convolutional module, and Y... STR This represents the third feature map.

[0111] In this embodiment, W-MSA reduces computational complexity while preserving the global modeling potential of the Transformer through local window partitioning and relative position encoding. Combined with the window offset strategy of SW-MSA, the Swing Transformer achieves efficient long-range dependency capture, becoming a mainstream architecture to replace CNNs in visual tasks. Its design philosophy has been further extended by subsequent work (such as frequency domain enhancement and cross-modal attention), driving the continuous evolution of visual Transformers.

[0112] To demonstrate the technical effectiveness of this invention, relevant experiments were conducted using the technical solution proposed in this invention. Experimental results show that the method proposed in this invention has high accuracy and robustness in the task of identifying acupoints on the human back, and can accurately identify 27 acupoints in various practical environments. It can be widely applied in the fields of intelligent healthcare, rehabilitation assistance, and health management.

[0113] In acupoint recognition tasks, selecting appropriate evaluation metrics is crucial for model optimization. Commonly used evaluation metrics include accuracy, precision, recall, F1-score, IoU, localization error, and mAP, which help comprehensively evaluate model performance. Depending on the focus of different tasks, these metrics can be comprehensively selected for evaluation based on requirements such as precision, recall, and localization accuracy, thereby better optimizing and improving model performance. This invention evaluates model performance using three metrics: precision, recall, and mean precision.

[0114] Precision: Precision measures the percentage of acupoints predicted by the model that are actually correct. High precision means the model does not make too many incorrect predictions when predicting acupoints.

[0115]

[0116] TP (True Positive) represents the number of correctly identified acupoints, while FP (False Positive) represents the number of incorrectly predicted acupoints.

[0117] Recall: Recall measures the proportion of actual acupoints that the model identifies. A high recall means the model can identify more actual acupoints, reducing missed detections.

[0118]

[0119] FN (False Negative) represents the number of acupoints that were missed, i.e., acupoints that actually existed but were not identified by the model.

[0120] Mean Average Precision (mAP): Mean Average Precision (mAP) is a commonly used evaluation metric in object detection tasks, suitable for multi-class recognition tasks. For acupoint recognition, the precision can be calculated for each acupoint category, and finally, the average precision for all acupoint categories is obtained. The higher the mAP value, the better the model's recognition performance.

[0121]

[0122] Where N is the total number of acupoint categories, AP i It represents the average accuracy for each acupoint category.

[0123] Experimental Results: The evaluation of keypoints typically focuses on the accuracy and precision of their predictions. Bounding box evaluation involves not only correctly identifying the location of the target acupoint, but also considering the degree of overlap between the predicted bounding box and the actual target area.

[0124] Table 1. Comparison of parameters for multiple evaluation indicators

[0125]

[0126] Analysis of the bounding box evaluation results shows that the model of this invention has strong accuracy and robustness in acupoint recognition tasks, can efficiently identify most target acupoints, and the prediction results have a high degree of overlap with the true values. The high performance of mAP and other evaluation metrics also indicates excellent overall model performance, with a 0.5% improvement in precision, a 0.6% improvement in recall, and a 0.4% improvement in mAP-50, making it suitable for practical acupoint recognition applications.

[0127] In this invention, the Focus and C3STR modules are introduced to optimize the model and improve the performance of acupoint recognition tasks. When comparing the number of model parameters, the Focus and C3STR modules effectively reduce computational load while maintaining high model accuracy. The advantages of this technical solution are not only reflected in the model's recognition accuracy but also in the significantly reduced computational resource requirements, making it suitable for deployment on edge devices and in low-resource environments.

[0128] Table 2. Comparison of Parameter Quantity and Effect

[0129]

[0130] Through the innovative technologies of this invention, particularly the introduction of the Focus and C3STR modules, model performance is preserved or even improved while significantly reducing the number of parameters. This makes the model more efficient and scalable, and also suitable for deployment on resource-constrained devices. Specifically, the number of model parameters is reduced by 59%, while recognition accuracy is almost unaffected. This characteristic gives this invention a strong advantage in practical applications, especially for scenarios requiring real-time processing and efficient computation. The comparison results of the prediction performance of the target detection model in this invention with traditional models are as follows: Figure 9 As shown.

[0131] In summary, the main innovations / advantages of this invention are:

[0132] 1. By combining automatic identification and localization technology of acupoints on the human back, and through a series of processes such as data collection and annotation, data augmentation, model improvement and training, the accuracy, robustness and efficiency of acupoint identification are improved.

[0133] 2. Building upon traditional deep learning models, this paper combines the Focus module with the C3STR module. By enhancing the model's feature extraction capabilities and reducing model parameters, it significantly improves the accuracy of keypoint recognition while reducing computational resource consumption. The Focus module strengthens the focus on features of important regions, while the C3STR module improves the network's feature learning method, enabling the model to better focus on target acupoints when dealing with complex human backgrounds.

[0134] 3. Dedicated data augmentation techniques are provided for acupoint recognition tasks. High-quality annotations of acupoints on the human back are achieved, and various innovative data augmentation methods, such as rotation, translation, scaling, mirroring, and Mosaic techniques, are employed to enhance the model's generalization ability and robustness by combining depth map information. In particular, the use of Mosaic technology enables the simultaneous processing of image data from multiple different perspectives, enhancing the model's adaptability to acupoint detection in complex environments.

[0135] Possible variations of this invention are as follows:

[0136] 1. Use lighter-weight deep learning models: In the future, lightweight neural networks (such as MobileNet and EfficientNet) may be designed to reduce computational resource requirements. These networks have a more compact architecture, making them suitable for embedded devices and mobile applications, and offering advantages for scenarios with high real-time processing and low-power consumption requirements.

[0137] 2. Multimodal Fusion: To improve the accuracy and robustness of acupoint recognition, future research may consider fusing data from multiple sensors, such as combining infrared or thermal imaging data. These sensors can help identify the thermal distribution characteristics of the human back and further refine the model's recognition capabilities by combining RGBD information. Through multimodal data fusion, the model can more comprehensively understand the geometric structure and temperature distribution of the human body, thereby improving the overall recognition effect.

[0138] 3. Adaptive Augmentation Strategy: Future development may consider introducing adaptive data augmentation strategies, such as dynamically adjusting the method and intensity of data augmentation based on the current training stage or the model's error distribution. A stronger augmentation strategy could be used in the early training phase, while the interference from augmentation could be reduced during later fine-tuning to improve the final model's generalization ability.

[0139] 4. Optimization methods to replace the Focus and C3STR modules: The Focus and C3STR modules reduce the number of model parameters by simplifying the network structure and feature extraction methods. Future development may consider introducing other convolutional optimization methods, such as the Squeeze-and-Excitation module (SE module) or attention mechanisms (such as SE-Net, CBAM, etc.). These methods weight features, allowing the network to focus more on regions that significantly contribute to the final task, further improving the model's performance and efficiency.

[0140] 5. Combining Traditional Computer Vision Methods with Deep Learning: Future development may combine the advantages of traditional computer vision techniques with those of deep learning. For example, features can be extracted using traditional methods such as HOG+SVM and image pyramids, and then combined with deep learning models for feature classification and regression. This can reduce the model's reliance on large datasets, allowing it to work effectively even when data is scarce.

[0141] 6. Real-time Performance Optimization: Considering the real-time processing requirements of this invention's application scenarios, the following suggestions for real-time performance optimization are proposed: Edge computing and model compression: To meet real-time requirements, the model can be deployed to edge computing devices. Compression techniques such as quantization, pruning, and distillation can reduce the model's storage space and computational demands while maintaining its accuracy. This is crucial for improving response speed and reducing latency in real-time applications. Low-latency inference framework: Low-latency inference frameworks (such as TensorRT or ONNX Runtime) are used for inference acceleration. By optimizing the computation graph and parallelizing the inference process, the model's running efficiency on edge devices can be maximized. This can significantly reduce the latency from data input to model output, ensuring the system can complete the task within a limited time.

[0142] 7. Application Scenario Integration: Integrate automatic detection of human acupoints with actual application scenarios. For example, customize designs for practical applications such as massage robots, enhance the linkage with automated equipment, and enable direct application to scenarios such as intelligent massage, thereby expanding the practical value and promotion potential of the technology.

[0143] Based on the above embodiments, the present invention also provides an automatic identification system for acupoints on the human back based on key point detection, such as... Figure 10 As shown, the system includes:

[0144] Image acquisition module 01 is used to acquire an image of the human back to be analyzed;

[0145] Image input module 02 is used to input the human back image into a trained target detection network; wherein, the trained target detection network includes a Focus module and several C3STR modules; the Focus module is located in the first layer and is used to perform slice reconstruction and feature extraction on the input image; the C3STR module is used to extract features from the output image of the previous layer through a hierarchical window self-attention mechanism;

[0146] Image detection module 03 is used to obtain acupoint recognition information corresponding to the human back image through the trained target detection network; wherein, the acupoint recognition information includes back detection box, the location and category of key acupoints.

[0147] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 11 As shown, the terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an automatic recognition method for human back acupoints based on key point detection. The display screen can be an LCD screen or an e-ink screen.

[0148] Those skilled in the art will understand that Figure 11 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] In one implementation, the terminal's memory stores one or more programs, and these programs are configured to be executed by one or more processors. The programs include instructions for performing an automatic identification method for human back acupoints based on key point detection.

[0150] In summary, this invention discloses an automatic identification method and system for acupoints on the human back based on keypoint detection. By introducing a Focus module and a C3STR module into the target detection network, the Focus module refines local regions, enhancing attention to the back and acupoint areas, enabling the target detection network to better capture key features. The C3STR module effectively improves feature extraction efficiency by introducing a spatial attention mechanism, enabling more accurate acupoint location. With the introduction of the Focus and C3STR modules, the target detection model achieves better detection results when detecting acupoints on the back with high density distribution and small target characteristics.

[0151] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A human back acupoint automatic recognition method based on key point detection, characterized in that, The method comprises: acquiring a human back image to be analyzed; inputting the human back image into a trained target detection network; wherein the trained target detection network comprises a Focus module and a plurality of C3STR modules; the Focus module is located at the first layer and is used for slice reorganization and feature extraction of the input image; the C3STR module is used for feature extraction of the output image of the previous layer through a hierarchical window self-attention mechanism; obtaining acupoint recognition information corresponding to the human back image through the trained target detection network; the acupoint recognition information comprises a back detection frame, the position and category of key acupoint positions; The target detection network comprises: a backbone network comprising a Focus module, a plurality of feature extraction modules, a spatial pyramid pooling fast module and a cross-stage local self-attention module which are sequentially cascaded; each feature extraction module comprises a standard convolution module and a C3STR module; the backbone network is used for feature extraction of the input image to obtain an initial feature map; an intermediate network comprising a plurality of first feature extraction modules and a plurality of second feature extraction modules which are sequentially cascaded; each first feature extraction module comprises an up-sampling module, a splicing module and a C3STR module which are sequentially cascaded; the input image of the splicing module in the first feature extraction module comprises the output image of the previous layer and the output image of the C3STR module in the backbone network; each second feature extraction module comprises a standard convolution module and a splicing module which are sequentially cascaded, and two adjacent second feature extraction modules are connected through a C3STR module; the input image of the splicing module in the second feature extraction module comprises the output image of the previous layer and the output image of the C3STR module in the first feature extraction module, or the output image of the previous layer and the output image of the cross-stage local self-attention module; the intermediate network is used for feature enhancement and fusion of the initial feature map output by the backbone network to obtain a target feature map; a detection head module used for predicting the target feature map output by the intermediate network to obtain the acupoint recognition information.

2. The automatic recognition method of human back acupoints based on key point detection according to claim 1, characterized in that, The training data set of the trained target detection network comprises: obtaining a plurality of original human back images through a depth camera; for each original human back image, obtaining an enhanced image corresponding to the original human back image through a data enhancement operation, and generating label data corresponding to the enhanced image; generating a data set according to all the enhanced images and the label data, and dividing the training data set from the data set.

3. The automatic recognition method of human back acupoints based on key point detection according to claim 2, characterized in that, The data enhancement operation comprises at least one of random cropping, random rotation, horizontal flipping, brightness adjustment, image merging and depth map enhancement.

4. The automatic recognition method of human back acupoints based on key point detection according to claim 1, characterized in that, The Focus module comprises: a slice module used for slicing the input image in the length and width dimensions to obtain sub-slice images; a splicing module used for splicing the sub-slice images in the channel dimension to obtain a spliced image; a standard convolution module used for feature extraction of the spliced image to obtain a feature map output by the Focus module.

5. The automatic recognition method of human back acupoints based on key point detection according to claim 1, characterized in that, The C3STR module comprises: a first standard convolution module, configured to perform feature extraction on an input image to obtain a first feature map; a second standard convolution module, configured to perform feature extraction on the input image to obtain a second feature map; a Swin Transformer module, configured to perform feature extraction on the first feature map through a hierarchical window self-attention mechanism to obtain a third feature map; a splicing module, configured to splice the second feature map and the third feature map to obtain a spliced feature map; a third standard convolution module, configured to perform feature extraction on the spliced feature map to obtain a feature map output by the C3STR module.

6. The automatic recognition method of human back acupoints based on key point detection according to claim 5, characterized in that, The Swin Transformer module comprises a plurality of Swin Transformer layers connected in sequence.

7. A human back acupoint automatic recognition system based on key point detection, characterized in that, The system comprises: an image acquisition module, configured to acquire a human back image to be analyzed; an image input module, configured to input the human back image into a trained target detection network; wherein the trained target detection network comprises a Focus module and a plurality of C3STR modules; the Focus module is located at the first layer and is configured to perform slice reorganization and feature extraction on an input image; and the C3STR module is configured to perform feature extraction on an output image of a previous layer through a hierarchical window self-attention mechanism; an image detection module, configured to acquire acupoint recognition information corresponding to the human back image through the trained target detection network; wherein the acupoint recognition information comprises a back detection frame, positions and categories of key acupoint positions. The target detection network comprises: a backbone network comprising a Focus module, a plurality of feature extraction modules, a spatial pyramid pooling fast module and a cross-stage local self-attention module connected in sequence; each feature extraction module comprises a standard convolution module and a C3STR module; the backbone network is configured to perform feature extraction on an input image to obtain an initial feature map; an intermediate network comprising a plurality of first feature extraction modules and a plurality of second feature extraction modules connected in sequence; each first feature extraction module comprises an upsampling module, a splicing module and a C3STR module connected in sequence; the input image of the splicing module in the first feature extraction module comprises an output image of a previous layer and an output image of the C3STR module in the backbone network; each second feature extraction module comprises a standard convolution module and a splicing module connected in sequence, and two adjacent second feature extraction modules are connected through a C3STR module; the input image of the splicing module in the second feature extraction module comprises an output image of a previous layer and an output image of the C3STR module in the first feature extraction module, or an output image of a previous layer and an output image of the cross-stage local self-attention module; the intermediate network is configured to perform feature enhancement and fusion on the initial feature map output by the backbone network to obtain a target feature map; a detection head module, configured to perform prediction on the target feature map output by the intermediate network to obtain the acupoint recognition information.

8. A terminal, characterized by comprising: The terminal comprises a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for executing the automatic identification method of back acupoints of human body based on key point detection according to any one of claims 1-6; and the processors are used for executing the programs.

9. A computer-readable storage medium storing a plurality of instructions thereon, characterized in that, The instructions are suitable for being loaded and executed by the processors to implement the steps of the automatic identification method of back acupoints of human body based on key point detection according to any one of claims 1-6.

Images