Robot massage-oriented back acupuncture point identification method

Abstract

The invention relates to the technical field of image processing, in particular to a robot massage-oriented back acupuncture point recognition method, which comprises the following steps of: acquiring a human body back image, and preprocessing the image; segmenting the back image by adopting YOLOv8 to obtain a back image; inputting the RGB image and the Depth image into an improved YOLOv8 network, and identifying the acupuncture points on the back; the improved YOLOv8 network adopts a double backbone as a feature extraction network. According to the method, the problem that the robustness of an existing model is poor due to interference of skin color difference, scars, illumination and the like in application scenes in an acupoint recognition task is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a back acupuncture point recognition method for robot massage. Background Art

[0002] For some clinical diseases, massage is a relatively simple, inexpensive and non-invasive auxiliary intervention measure; massage needs to act on certain special parts of the body, which are called meridians and acupoints in traditional Chinese medicine.

[0003] Zhang Huakai et al. proposed an acupoint localization and tracking method based on template matching. However, due to its lack of intelligence, it has poor robustness to receptor differences.

[0004] Liu et al. proposed the FasterPFLD-L model, which can quickly and accurately locate facial acupoints; Wang et al. proposed a hand acupoint localization method based on a dual attention mechanism and a cascade network model; Liu et al. used the KeypointRCNN model to automatically locate the Du meridian acupoints on the back, and on this basis, corrected the obtained Du meridian acupoint positions according to prior knowledge, and finally located other acupoints on the foot-taiyang bladder meridian based on the corrected Du meridian acupoints; however, the acupoint recognition accuracy and robustness of the above methods need to be further improved.

[0005] In addition, the existing human back acupoint recognition methods for robot massage do not take into account individual differences in practical applications and the actual relative position of the receptor and the camera, which affects the recognition of acupoints. Summary of the invention

[0006] In view of the shortcomings of the existing methods, the present invention solves the problem that in the task of identifying acupoints, the application scenarios have interferences such as skin color differences, scars, and lighting, which leads to poor robustness of the existing models.

[0007] The technical solution adopted by the present invention is: a back acupoint recognition method for robot massage comprises the following steps:

[0008] Step 1: Collect human back images and preprocess the images;

[0009] As a preferred embodiment of the present invention, the preprocessing includes: changing brightness, contrast, grayscale, hue, saturation, flipping, translation, scaling and Gaussian blur.

[0010] Step 2: Use YOLOv8 to segment the back image to obtain the back image; the back image includes RGB image and Depth image, and the RGB image and Depth image are input into the improved YOLOv8 network to identify the acupuncture points on the back; the improved YOLOv8 network uses a double backbone as a feature extraction network;

[0011] As a preferred embodiment of the present invention, the improved YOLOv8 network includes a first backbone branch, a second backbone branch and a MIFM branch; the RGB image is input into the first backbone branch, the Depth image is input into the second backbone branch, the primary fusion feature is input into the Conv of the MIFM branch, the first Conv of the two backbone branches and the Conv of the MIFM branch are input into the first MIFM module; three MIFM modules are respectively connected between the three pairs of C2f+Conv combinations of the two backbone branches, and two adjacent MIFM modules are cascaded; the last C2f of the two backbone branches and the fourth MIFM module are input into the fifth MIFM module; the fifth MIFM module is connected to the SPPF, and the third and fourth MIFM modules are connected to the first and second Concat operations of the Neck network.

[0012] As a preferred implementation of the present invention, the MIFM module includes: a SAM_MIM module and a SE_MFM module, and the SAM_MIM module inputs the SE_MFM module.

[0013] As a preferred embodiment of the present invention, the mechanism of the SAM_MIM module is to convert D n and R n The feature ω after cross injection with the spatial attention weight R D n and ω D R n Add element by element to get the output features and The formula is:

[0014]

[0015] Among them, D n is the Depth feature, R n is the RGB feature, ω D and ω R They are feature D n and R n The spatial attention weights are:

[0016] ω D =σ(Conv[AvgPool(D n );MaxPool[D n ]]) (2)

[0017] ω R =σ(Conv[AvgPool(R n );MaxPool[Rn ]]) (3)

[0018] Among them, σ represents the Sigmoid activation layer, Conv represents the convolutional layer, AvgPool and MaxPool represent the maximum pooling and average pooling respectively.

[0019] As a preferred embodiment of the present invention, the SE_MFM module works by using three AFF modules to transform the interacted features into and And the fusion feature F n Hierarchical fusion, output fusion feature F n+1 , the formula is:

[0020]

[0021] The mechanism of action of the AFF module is as follows:

[0022] AFF(I1,I2)=S 1+2 I1+(1-S 1+2 )I2 (5)

[0023] Among them, I1 and I2 are two feature inputs Input1 and Input2 respectively. The AFF module adds I1 and I2 as the initial integral and sends it to the SE module to obtain the fusion weight

[0024] The SE module consists of the Squeeze module and the Excitation module in SE-Net, and its mechanism of action is:

[0025]

[0026] S=Excitation(Z)=σ(W2δ(W1Z)) (7)

[0027] Among them, X in the Squeeze module represents the input feature, X k is the k-th dimension channel feature of X; the feature vector Z after Squeeze compression is [z1,z2,…,z C ]; In the Excitation module, W1 and W2 represent two fully connected layers and S is a weight vector.

[0028] As a preferred implementation of the present invention, the camera shooting position is pre-adjusted when acquiring the image of the human back.

[0029] As a preferred embodiment of the present invention, the machine vision-based servo control is pre-adjusted to minimize the image error.

[0030] As a preferred embodiment of the present invention, a back acupoint recognition system for robot massage includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement a back acupoint recognition method for robot massage.

[0031] As a preferred embodiment of the present invention, a computer readable medium stores computer program codes, and when the computer program codes are executed by a processor, a back acupuncture point recognition method for robot massage is implemented.

[0032] Beneficial effects of the present invention:

[0033] 1. Improve the YOLOv8 network and propose a dual-modal acupoint recognition model. By integrating RGB features and depth features, the accuracy and robustness of the back acupoint recognition model are improved;

[0034] 2. Design a dual-backbone YOLOv8 network; and design a MIFM module between the dual backbones; the SAM_MIM module uses the spatial attention mechanism to explore the synergistic relationship between the Depth stream and the RGB stream, establish the interaction between the modal branches, and thus improve the expressiveness of each modal branch; at the same time, the SAM_MIM module also introduces a residual structure to enhance feature reuse and reduce the risk of overfitting; the SE_MFM module uses the channel attention mechanism to hierarchically integrate the interacted Depth stream and RGB stream as well as the upper-layer fusion features to obtain new fusion features;

[0035] 3. A self-adjustment method of shooting angle based on visual servoing is proposed to improve the quality of visual information acquisition and ensure the robustness of the human back acupoint recognition model in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is the improved YOLOv8 network structure diagram of the present invention;

[0037] Figure 2 are examples of labeled and unlabeled datasets;

[0038] Figure 3 It is a SAM_MIM module structure diagram of the present invention;

[0039] Figure 4 It is a structural diagram of the SE_MFM module of the present invention;

[0040] Figure 5 Schematic diagram of visual servo control; (a) is image-based visual servo, (b) is position-based visual servo;

[0041] Figure 6It is the back spatial position solution diagram; (a) is the back segmentation, (b) is the OBB bounding box;

[0042] Figure 7 It is the experimental result diagram; (a) is the loss value, (b) is R, mAP50, and mAP50-95;

[0043] Figure 8 This is the servo control effect diagram;

[0044] Fig. 9 The following are the acupoint recognition effect diagrams; (a) is the test set result diagram, and (b) is the actual scene diagram. DETAILED DESCRIPTION

[0045] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0046] like Figure 1 As shown, a back acupoint recognition method for robot massage includes the following steps:

[0047] Step 1: Collect human back images and preprocess the images;

[0048] To avoid loss of generality, the objects in the dataset cover various body characteristics, including differences in height, body shape, and skin color. The dataset construction plan is:

[0049] 1. Human body posture: Most people take the prone position, with their hands hanging naturally at the sides of the body. In this position, the acupoints are not easily offset, and the human body is not likely to feel tired or uncomfortable;

[0050] 2. Shooting equipment: Intel Realsense D435 depth camera;

[0051] 3. Data capture: First, the camera is fixed above the human body, and then the TCM doctor uses the bone measurement positioning method to mark the acupoints on the back of the subject. The acupoints involved are mainly Dazhui and Yaoyang points on the Governor Vessel, Dazhui, Feishu, Ganshu, Gaohuang and Tianzong points on the Foot-Taiyang Bladder Meridian.

[14] Then keep the human body position unchanged and shoot Figure 2 (a) shows the labeled data, and Figure 2 (b) shows the data after the mark is removed, and the captured data includes RGB image and depth image;

[0052] 4. Data labeling: First, Labelme is used to label the RGB images in the labeled data to obtain labels, and then the obtained labels and the data after removing the labels together constitute the data set;

[0053] 5. Dataset enhancement: Expand the dataset by changing brightness, contrast, grayscale, hue, saturation, flipping, translation, scaling, Gaussian blur, etc.

[0054] Step 2: Use YOLOv8 to segment the back image to obtain the back image; the back image includes RGB image and Depth image, and the RGB image and Depth image are input into the improved YOLOv8 network to identify the acupuncture points on the back; the improved YOLOv8 network uses a double backbone as a feature extraction network;

[0055] The improved YOLOv8 network includes the first backbone branch, the second backbone branch and the MIFM branch; the RGB image is input into the first backbone branch, the Depth image is input into the second backbone branch, the primary fusion feature is input into the Conv of the MIFM branch, the first Conv of the two backbone branches and the Conv of the MIFM branch are input into the first MIFM module; three pairs of C2f+Conv combinations of the two backbone branches are respectively connected with three MIFM modules, and two adjacent MIFM modules are cascaded; the last C2f and the fourth MIFM module of the two backbone branches are input into the fifth MIFM module; the fifth MIFM module is connected to the SPPF, and the third and fourth MIFM modules are connected to the first and second Concat operations of the Neck network.

[0056] The YOLOv8 model has the advantages of fast detection speed and strong generalization ability. Its structure can be divided into four parts according to its functions: Input, Backbone, Neck and Head, which are used for image preprocessing, image feature extraction, image feature fusion enhancement and image prediction output respectively. YOLOv8 is a single-input model, and RGB images are mainly used as input objects in practical applications. Therefore, in the task of identifying acupoints, the model has poor robustness to various interferences in the application scenarios, such as skin color differences, scars, and lighting.

[0057] Therefore, the present invention is improved based on the YOLOv8 model, takes the two modalities of Depth image and RGB image as input, uses two backbones as feature extraction networks, and extracts features to form Depth stream and RGB stream; different modalities carry different information, and the data characteristics and processing requirements between modalities vary greatly. Therefore, the present invention embeds a modal interaction fusion module (MIFM) between the Depth stream and the RGB stream, so that the information between different modalities can fully interact and effectively integrate, thereby obtaining a unified and comprehensive representation and sending it to the subsequent Neck and Head for further feature fusion and prediction, so that the model can make more accurate inferences and decisions.

[0058] The 640×640×6 primary fusion feature is obtained by concatenating the 640×640×3 initial depth feature and the 640×640×3 RGB feature, and then undergoes a 3×3 convolution;

[0059] Improvements to the YOLOv8 network include:

[0060] The RGB images are input into the first Conv layer respectively; the Depth images are input into the seventeenth Conv layer;

[0061] The RGB image and the Depth image are input into the tenth Concat operation and then into the eleventh Conv layer;

[0062] The first Conv layer, the eleventh Conv layer and the seventeenth Conv layer input the twelfth MIFM module;

[0063] The twelfth MIFM module inputs the second Conv layer, the thirteenth MIFM module, and the eighteenth Conv layer respectively;

[0064] The second Conv layer is connected to the third C2f layer; the eighteenth Conv layer is connected to the nineteenth C2f layer; the third C2f layer, the nineteenth C2f layer and the twelfth MIFM module input the thirteenth MIFM module;

[0065] The thirteenth MIFM module inputs the fourth Conv layer, the fourteenth MIFM module, and the twentieth Conv layer respectively;

[0066] The fourth Conv layer is connected to the fifth C2f layer; the twentieth Conv layer is connected to the twenty-first C2f layer; the fifth C2f layer, the twenty-first C2f layer and the thirteenth MIFM module input the fourteenth MIFM module;

[0067] The fourteenth MIFM module inputs the sixth Conv layer, the fifteenth MIFM module and the twenty-second Conv layer respectively;

[0068] The sixth Conv layer is connected to the seventh C2f layer; the twenty-second Conv layer is connected to the twenty-third C2f layer; the seventh C2f layer, the twenty-third C2f layer and the fourteenth MIFM module input the fifteenth MIFM module;

[0069] The fifteenth MIFM module inputs the eighth Conv layer, the sixteenth MIFM module and the twenty-fourth Conv layer respectively;

[0070] The eighth Conv layer is connected to the ninth C2f layer; the twenty-fourth Conv layer is connected to the twenty-fifth C2f layer; the ninth C2f layer, the twenty-fifth C2f layer and the fifteenth MIFM module input the sixteenth MIFM module;

[0071] The sixteenth MIFM module inputs the twenty-sixth SPPF layer; the fifteenth MIFM module inputs the twenty-eighth Concat operation; and the fourteenth MIFM module inputs the thirty-first Concat operation.

[0072] The MIFM module consists of two parts: Spatial Attention Mechanism_Modal interaction module, SAM_MIM and Squeeze and Excitation_Modal fusion module, SE_MFM.

[0073] First, the SAM_MIM module uses the spatial attention mechanism to explore the synergistic relationship between the Depth stream and the RGB stream, establishes the interaction between the modal branches, and thus improves the expressiveness of each modal branch. Then, the SE_MFM module uses the channel attention mechanism to hierarchically integrate the interacted Depth stream, RGB stream, and upper-layer fusion features to obtain new fusion features.

[0074] Depth data provides distance information of the scene, while RGB data provides rich color and texture information. There is a certain complementary relationship between the two. The Spatial Attention Mechanism (SAM) can identify the key areas in the image or feature map and dynamically adjust the degree of attention to this important area in a weighted manner; Figure 3 The SAM_MIM module shown uses the method of spatial attention cross injection to enable the Depth stream and RGB stream to integrate effective information from both sides, focus on the key parts of the data, and give full play to the synergy between modal data; at the same time, the SAM_MIM module also introduces a residual structure to enhance feature reuse and reduce the risk of overfitting; the mechanism of the SAM_MIM module is as follows:

[0075]

[0076] Where D n is the Depth feature, R n is the RGB feature, and D n and R n The feature ω after cross injection with the spatial attention weight R D n and ω D R n Add element by element to get the output features and ω D and ω R The dimensions [H, W, C] are features D n and R n The spatial attention weights are:

[0077] ω D =σ(Conv[AvgPool(D n );MaxPool[D n ]]) (2)

[0078] ω R =σ(Conv[AvgPool(R n );MaxPool[R n ]]) (3)

[0079] Among them, σ represents the Sigmoid activation layer, Conv represents the convolutional layer, AvgPool and MaxPool represent the maximum pooling and average pooling respectively. First, two attention maps of [H, W, 1] dimensions are obtained through AvgPool and MaxPool, and the two are concatenated to obtain a tensor of dimension [H, W, 2]. Then, Conv is used to reduce the tensor dimension to [H, W, 1]. Finally, the weights are normalized.

[0080] The main goal of the SE_MFM module is to effectively integrate the information after the interaction of each modality to generate a unified and diverse information representation to facilitate subsequent decision-making and prediction tasks.

[0081] The SE attention mechanism (Squeeze-and-Excitation Network, SE-Net) can adaptively learn the importance of each channel of the feature map, and dynamically adjust the feature responses of different channels in a weighted manner, enhance the information channels in each modality that contribute to the final decision, and suppress the feature channels that are not very useful for the current task.

[0082] like Figure 4 As shown, the SE_MFM module works as follows:

[0083]

[0084] The SE_MFM module uses three AFF modules to transform the interacted features and And the previous layer fusion feature F n Perform hierarchical fusion and finally output the fusion feature F n+1 The AFF module is an Attentional Feature Fusion (AFF) module, and its working mechanism is as follows:

[0085] AFF(I1,I2)=S 1+2 I1+(1-S 1+2 )I2 (5)

[0086] Among them, I1 and I2 are two feature inputs Input1 and Input2 respectively. The AFF module adds I1 and I2 as the initial integral and sends it to the SE module to obtain the fusion weight The dotted lines in the AFF module represent Finally, the output is obtained by weighting I1 and I2 for soft selection or weighted averaging. The SE module consists of the Squeeze module and the Excitation module in SE-Net, and its mechanism is as follows:

[0087]

[0088] S=Excitation(Z)=σ(W2δ(W1Z)) (7)

[0089] Among them, X in the Squeeze module represents the input feature of size H×W×C, X k is the k-th dimension channel feature of X, and the k-th dimension channel feature is compressed into an element z by averaging. k ,in The feature vector after Squeeze compression is Z = [z1,z2,…,z C ]. In the Excitation module, W1 and W2 represent two fully connected layers. W1 compresses the C-dimensional channel into a C / r (r refers to the compression ratio)-dimensional channel to reduce the amount of computation, and W2 restores the number of channels back to C; δ represents the RELU activation layer, and S is the final 1×1×C weight vector.

[0090] Most massage robots on the market use a mechanical structure with eyes on hands. In daily massage scenes, the relative position of the massage robot and the human body is often affected by external factors, which in turn affects the shooting angle; traditional fixed-position shooting cannot flexibly adapt to changes in relative position, which can lead to problems such as missing objects and tilted field of view. A good shooting position is a prerequisite for accurate acupoint identification and positioning. The present invention proposes to pre-adjust the camera shooting position using a visual servo control method before acupoint identification.

[0091] Robot visual servo is an organic combination of machine vision and robot control, and its content involves research fields such as image processing, robot kinematics and dynamics, and control theory. The purpose of the servo control method based on machine vision is to minimize an image error, which can be defined as:

[0092] e(t)=s(m(t),a)-s * (8)

[0093] Where m(t) is a series of image features, such as image point features, shape features, posture features or frequency domain features; s(m(t),a) represents the mapping function formed by the feature m(t) and some additional system parameters a; the extracted feature m(t) is mapped to the space of the target feature s* and compared with s*. When the difference between the two is zero, that is, the system error e(t) is zero, and the system reaches a stable convergence position.

[0094] Visual servoing methods can be divided into image-based visual servoing and position-based visual servoing according to the different target image features m(t); Figure 5 (a) The image-based visual servo method directly controls the movement of the robot arm by obtaining feature points from the image; Figure 5 (b) The position-based servo method first calculates the target's position and posture, and then plans the motion plan based on the relative position of the robot arm and the target object.

[0095] Since there are fewer features on the back, the present invention adopts position-based visual servoing; Figure 6 As shown in (a), the back mask is first obtained by the YOLOv8 segmentation algorithm, and the depth information is obtained based on the mask to obtain the back point cloud, where the spatial coordinates of each pixel point in the camera coordinate system can be expressed as:

[0096]

[0097] Where (u,v) represents the pixel coordinates, f x ,f yThe four parameters u0, v0 are the internal parameters of the camera, fx = F / dx, fy = F / dy, where F is the focal length of the camera, dx and dy are the horizontal and vertical physical sizes of a single pixel, that is, the actual width and height of a pixel. u0, v0 are half of the horizontal and vertical resolutions of the image, respectively.

[0098] Next, construct the minimum bounding box (OBB) as Figure 6 As shown in (b), the pose of the OBB bounding box is used to represent the spatial coordinate pose feature m(t) of the back point cloud.

[0099] Experimental environment and parameters:

[0100] This experiment was conducted on the Windows 10 operating system, and the experimental platform is shown in Table 1. During the experiment, the resolution of the sample image was set to 640 × 640 pixels, the training batch was 16, the initial learning rate was 0.01, and the number of early stopping rounds was 10.

[0101] Table 1 Test platform

[0102]

[0103]

[0104] Evaluation Metrics

[0105] In order to comprehensively evaluate the performance of the deep model on the posture detection image, this paper adopts the mAP (meanAverage Precision) which is widely recognized in the evaluation of detection algorithms.

[0106] This is a key indicator. mAP is the average of various types of precision in the data set, which aims to measure the accuracy of the model in the target detection task. It combines accuracy and recall and is an important criterion for evaluating model performance in the pose estimation task. The formula for the specific threshold mean precision (mAP@0.5, mAP@0.5:0.95) is defined as follows:

[0107]

[0108] Among them, P represents the accuracy, R represents the recall rate, AP represents the average precision, TP represents the correct target box successfully predicted by the model, FP represents the box incorrectly predicted by the model, FN represents the target box that the model failed to predict correctly, and IOU(th) represents the specific intersection-over-union ratio threshold between the prior box and the true box.

[0109] Model evaluation and analysis:

[0110] mAP@0.5 is the average value of the proportion of correctly predicted object boxes to all predicted boxes when the IOU value is greater than or equal to 0.5; mAP@0.5:0.95 is a more comprehensive indicator, which is the average accuracy of the model under 10 different IOU thresholds from 0.5 to 0.95. This indicator more comprehensively reflects the performance of the model in the object detection task. Therefore, three indicators are mainly used to evaluate the performance of the deep model in the acupoint recognition task: Recall, mAP@0.5, and mAP@0.5:0.95. The model training results are shown in Figure 2. Figure 7 As shown, the final model has a recall of 96.9%, mAP@0.5 of 99.4%, and mAP@0.5:0.95 of 85.0%.

[0111] Ablation experiment:

[0112] In order to verify the effectiveness of the improved method, an ablation experiment was designed. By comparing the experimental indicators of YOLOv8s, YOLOv8s+SE_MFM and YOLOv8s+SE_MFM+SAM_MIM, the influence of SAM-MIM and SE_MFM on the model recognition effect was evaluated. The experimental details are shown in Table 2. The results show that the addition of SAM-MIM and SE_MFM has a positive effect on improving the acupoint recognition performance of the model.

[0113] Table 2 Ablation test

[0114]

[0115] Comparative experiment:

[0116] YOLOv8n_MIFM is further compared with YOLOv5n_pose, YOLOv7_pose, and YOLOv8n models on the back acupuncture point training dataset. As shown in Table 4, the mAP@0.5:0.95 of the YOLOv8n-MIFM model is 85%, which is 5.7%, 4.8%, and 0.7% higher than that of the YOLOv5n_pose, YOLOv7_pose, and YOLOv8n models, respectively. It can be seen that the YOLOv8n-MIFM model has the best overall performance on the back acupuncture point dataset compared to the control model.

[0117] Table 3 Comparison of detection performance of each model

[0118]

[0119] Verification experiment:

[0120] Figure 8In order to simulate the situation in which the relative position of the human body and the camera deviates from the expected position due to objective reasons in daily use, the video frames of the depth camera are first extracted at a frequency of 15Hz, where each frame includes an RGB image and a depth image. Then, the minimum bounding box method is used to estimate the human body posture. Finally, according to the visual servoing theory, the robotic arm is controlled to move slowly to adjust the shooting angle.

[0121] After the robot reaches the target position, it uses its equipped depth camera to capture the RGB image and depth image of the back. Then, the captured RGB image and depth image are input into the dual-modal acupoint recognition model for prediction. The final result is as follows: Fig. 9 (a) shown. Fig. 9 (b) is the prediction result of the test set under ideal conditions, where the dots represent the prediction results and × represents the actual label.

[0122] In the field of human back acupoint recognition for robot massage, there are problems such as difficulty in data set collection and annotation, poor robustness of single-modal model recognition, and non-standard actual input information, which limit the popularization and development of the technology. This study performs annotation based on manual acupoint marking, making acupoint annotation more convenient and reliable. The improved dual-modal acupoint recognition model improves the accuracy of the back acupoint recognition model by fusing RGB features and depth features. The proposed method of adjusting the shooting angle based on visual servoing makes the input of the model more standardized and unified, solves problems such as back tilt in shooting, and improves the robustness of the acupoint recognition model to a certain extent.

[0123] In addition, the research and application of acupoint recognition urgently requires a large-scale, diverse, standardized database to train and verify the recognition system, and currently the resources in this area are still insufficient. There is a lack of monitoring of human feedback during the follow-up massage process, and the issue of how to ensure the safety and comfort of the human body during the massage cannot be ignored. Finally, the vast majority of users are still skeptical about robot physiotherapy, and its safety and comfort are still in the testing stage.

[0124] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A back acupoint recognition method for robot massage, characterized in that: The following steps are involved: Step 1: Collect human back images and preprocess the images; Step 2: Use YOLOv8 to segment the back image; input the RGB image and Depth image into the improved YOLOv8 network to identify the acupoints on the back; the improved YOLOv8 network uses a dual backbone as the feature extraction network.

2. The back acupoint recognition method for robot massage according to claim 1, characterized in that: The improved YOLOv8 network includes the first backbone branch, the second backbone branch and the MIFM branch; the RGB image is input into the first backbone branch, the Depth image is input into the second backbone branch, the primary fusion feature is input into the Conv of the MIFM branch, the first Conv of the two backbone branches and the Conv of the MIFM branch are input into the first MIFM module; three pairs of C2f+Conv combinations of the two backbone branches are respectively connected with three MIFM modules, and two adjacent MIFM modules are cascaded; the last C2f and the fourth MIFM module of the two backbone branches are input into the fifth MIFM module; the fifth MIFM module is connected to the SPPF, and the third and fourth MIFM modules are connected to the first and second Concat operations of the Neck network.

3. The back acupoint recognition method for robot massage according to claim 2, characterized in that: The MIFM module includes: a SAM_MIM module and a SE_MFM module, and the SAM_MIM module inputs the SE_MFM module.

4. The back acupoint recognition method for robot massage according to claim 3, characterized in that: The working mechanism of the SAM_MIM module is to convert D n and R n The feature ω after cross injection with the spatial attention weight R D n and ω D R n Add element by element to get the output features and The formula is: Among them, Dn is the Depth feature, Rn is the RGB feature, ωωD and ωωR are the spatial attention weights of features Dn and Rn respectively, ω D =σ(Conv[AvgPool(D n );MaxPool[D n ]]) (2) ω R =σ(Conv[AvgPool(R n );MaxPool[R n ]]) (3) Among them, σ represents the Sigmoid activation layer, Cony represents the convolutional layer, AvgPool and MaxPool represent the maximum pooling and average pooling respectively.

5. The back acupuncture point recognition method for robot massage according to claim 4, characterized in that: The mechanism of SE_MFM module is to use three AFF modules to transform the interacted features and And the fusion feature F n Hierarchical fusion, output fusion feature F n+1 , the formula is: The mechanism of action of the AFF module is as follows: AFF(I1,I2)=S 1+2 I1+(1-S 1+2 )I2 (5) Among them, I1 and I2 are two feature inputs Input1 and Input2 respectively. The AFF module adds I1 and I2 as the initial integral and sends it to the SE module to obtain the fusion weight The SE module consists of the Squeeze module and the Excitation module in SE-Net, and its mechanism of action is: S=Excitation(Z)=σ(W2δ(W1Z)) (7) Among them, X in the Squeeze module represents the input feature, X k is the k-th dimension channel feature of X; the feature vector Z after Squeeze compression is [z1, z2, …, z C ]; In the Excitation module, W1 and W2 represent two fully connected layers, and S is a weight vector.

6. The back acupuncture point recognition method for robot massage according to claim 1, characterized in that: Preprocessing includes: By changing brightness, contrast, grayscale, hue, saturation, flip, translate, zoom and Gaussian blur.

7. The back acupuncture point recognition method for robot massage according to claim 1, characterized in that: Pre-adjust the camera shooting position when acquiring images of the human back.

8. The back acupuncture point recognition method for robot massage according to claim 7, characterized in that: Pre-adjust machine vision-based servo control to minimize image errors.

9. The back acupuncture point recognition system for robot massage is characterized by: include: a memory for storing instructions executable by a processor; A processor, configured to execute instructions to implement the back acupoint recognition method for robot massage as described in any one of claims 1 to 8.

10. A computer readable medium storing computer program code, characterized in that: When the computer program code is executed by a processor, the computer program code implements the back acupoint recognition method for robot massage as described in any one of claims 1 to 8.

Images