Dynamic acupoint positioning moxibustion system fused with multi-modal vision and control method

Abstract

The invention relates to the field of traditional Chinese medicine physiotherapy, and particularly discloses a dynamic acupoint positioning moxibustion system fused with multi-modal vision and a control method, the system comprises an image acquisition module used for acquiring RGB images and thermodynamic images of the body surface of a patient, the RGB images are acquired through a high-definition full-color camera, and the thermodynamic images are acquired through an infrared thermal camera; the preprocessing module is used for carrying out normalization preprocessing on the acquired RGB image and the thermodynamic image and splicing the preprocessed image data in a channel dimension; by fusing the multi-modal vision technology, the moxibustion system can simultaneously obtain the RGB image and the thermodynamic image of the body surface of the patient, and the advantages of the two images in the aspects of vision and thermal imaging are fully utilized; rich color and texture information is provided by the RGB image, so that body surface features can be identified; the thermodynamic image reflects the temperature distribution of the body surface, and has unique value for positioning areas with special physiological significance, such as acupuncture points and the like.

Description

Technical Field

[0001] This invention belongs to the field of traditional Chinese medicine physiotherapy, specifically a dynamic acupoint positioning moxibustion system and control method that integrates multimodal vision. Background Technology

[0002] In the field of Traditional Chinese Medicine (TCM) physiotherapy, moxibustion, as a traditional therapy with a long history and significant efficacy, has always received widespread attention and application. It uses the warmth generated by burning mugwort to stimulate specific acupoints, aiming to unblock meridians, harmonize qi and blood, and prevent and treat diseases. However, the traditional moxibustion treatment process is highly dependent on the experience and skills of the physician, especially in the acupoint location stage. The accuracy is greatly affected by human factors, and differences in location may exist between different physicians. This not only affects the stability and repeatability of the treatment effect but also, to some extent, limits the promotion and popularization of moxibustion therapy.

[0003] With the advancement of technology, especially the rapid development of computer vision and artificial intelligence, new possibilities have emerged for the modernization of moxibustion therapy. Multimodal vision technology, combining the advantages of visible light imaging and infrared thermal imaging, can simultaneously capture morphological features and temperature distribution information of the body surface, providing richer and more comprehensive data support for acupoint location. Deep learning, particularly convolutional neural networks (CNNs) and their variants such as U-Net and HRNet, has demonstrated outstanding performance in image segmentation and object detection, effectively processing complex image data, extracting key features, and achieving high-precision acupoint recognition and location.

[0004] Although some studies have attempted to apply computer vision technology to acupuncture point location, most solutions only use image data of a single modality, ignoring the complementarity between body surface morphology and temperature information, which limits the accuracy of location. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a dynamic acupoint positioning and moxibustion system that integrates multimodal vision, thereby solving the problem in the prior art where relying solely on single-modal image data and failing to fully utilize the complementarity of body surface morphology and temperature information leads to limited acupoint positioning accuracy.

[0006] A dynamic acupoint positioning and moxibustion system integrating multimodal vision, comprising:

[0007] Image acquisition module: used to acquire RGB images and thermal images of the patient's body surface. The RGB images are acquired by a high-definition full-color camera, and the thermal images are acquired by an infrared thermal camera.

[0008] Preprocessing module: Normalizes the acquired RGB and thermal images and stitches the preprocessed image data along the channel dimension; at the same time, it preprocesses the marked acupoint coordinates to generate corresponding soft labels.

[0009] Data partitioning module: Divides the preprocessed image data and corresponding soft labels into training dataset and test dataset;

[0010] Deep learning model module: The deep learning model is constructed using a U-Net+HRNet parallel structure. The input is the stitched RGB and thermal images, and the output is the coordinates and classification information of the detected acupoints.

[0011] Training and Evaluation Module: Trains the deep learning model using the training dataset, employing mean squared error (MSE) as the loss function; evaluates the accuracy of the trained model using the test dataset to determine whether the model's accuracy meets the preset requirements.

[0012] Dynamic positioning and moxibustion execution module: including a multi-joint collaborative robot and an automatic smoke-exhausting moxibustion tube. The end joints of the collaborative robot are equipped with high-definition full-color cameras and infrared thermal cameras. Based on the prediction results of the trained deep learning model, the position of the end joints is dynamically adjusted to achieve precise positioning of acupoints and start the moxibustion tube to perform moxibustion therapy.

[0013] Control module: Responsible for the coordinated operation of the entire system, including image acquisition control, model prediction control, robot motion control, and moxibustion tube start / stop control.

[0014] Preferably, the preprocessing module further includes:

[0015] Image normalization unit: Normalizes the pixel value of each pixel in the RGB image and thermal image by dividing it by 255;

[0016] Coordinate preprocessing unit: Generates soft labels based on the marked acupoint coordinates. The soft labels are filled with linear interpolation within a preset radius, centered on the acupoint coordinates. The fill value at the edge is 0.3, and the fill value at the center is 1.0.

[0017] Preferably, the U-Net+HRNet parallel structure in the deep learning model module further includes:

[0018] Four parallel feature extraction branches: the original image is 1x, 2x, 4x and 8x larger than the feature map, respectively. Each branch performs independent loss calculation after outputting the feature map.

[0019] Feature fusion and output unit: The outputs of the four parallel branches are superimposed in the channel dimension, and the output shape is (B, C, H, W), where B is the batch size, C is the number of acupoints to be detected, and H and W are the height and width of the original image, respectively.

[0020] Preferably, the training and evaluation module further includes:

[0021] Loss calculation unit: The loss function for each parallel structure uses mean squared error (MSE), and the total loss is the sum of the losses of the four parallel structures;

[0022] Model evaluation unit: Based on the prediction model evaluation index, the accuracy of the model is evaluated using test set data, and the maximum deviation of each acupoint is limited to 25 pixels to be considered a correct prediction.

[0023] Preferably, the dynamic positioning and moxibustion execution module further includes:

[0024] Multi-joint collaborative robot: Two cameras are installed inside the end joint, and RGB images and thermal images are captured separately by rotating the end joint;

[0025] Automatic smoke-exhausting moxibustion tube: Installed at the end of the collaborative robot, it performs moxibustion on accurate acupoints based on model predictions, and also has an automatic smoke exhaust function.

[0026] A control method for a dynamic acupoint positioning moxibustion system integrating multimodal vision as described above includes the following steps:

[0027] The image acquisition module is activated to obtain RGB and thermal images of the patient's body surface;

[0028] The acquired image data is transmitted to the preprocessing module for normalization and stitching, and the coordinates of the marked acupoints are preprocessed to generate soft labels.

[0029] The preprocessed image data and soft labels are transmitted to the data partitioning module and divided into training dataset and test dataset.

[0030] The training dataset is input into the deep learning model module for training, and the trained model is evaluated using the test dataset;

[0031] Based on the evaluation results, determine whether the model accuracy meets the preset requirements. If it does not, return to the preprocessing module to adjust the preprocessing parameters or re-collect data.

[0032] If the model accuracy meets the requirements, the dynamic positioning and moxibustion execution module is activated. Based on the model prediction results, the position of the end joint of the collaborative robot is dynamically adjusted to achieve precise positioning of acupoints, and the moxibustion tube is activated to perform moxibustion therapy.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] By integrating multimodal vision technology, this moxibustion system can simultaneously acquire RGB images and thermal images of the patient's body surface, making full use of the advantages of both types of images in visual and thermal imaging. RGB images provide rich color and texture information, which helps to identify body surface features; thermal images reflect the temperature distribution of the body surface, which has unique value for locating acupoints and other areas with special physiological significance.

[0035] A deep learning model is constructed using a parallel structure of U-Net and HRNet, which fully leverages the advantages of both network structures in feature extraction and image segmentation. This parallel structure can more effectively capture multi-scale features in images and improve the accuracy of acupoint detection. At the same time, the model is trained using the mean squared error loss function and its accuracy is evaluated using a test dataset, ensuring the reliability and stability of the model in practical applications.

[0036] The collaborative work of the dynamic positioning, moxibustion execution module, and control module enables a fully automated process from image acquisition to acupoint location and moxibustion therapy. The flexibility and precision of the multi-joint collaborative robot, combined with the design of the automatic smoke-exhausting moxibustion cylinder, not only improves the efficiency and comfort of moxibustion treatment but also reduces the risks and errors of human operation. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the process of the present invention;

[0038] Figure 2 This is a schematic diagram of the network structure of the present invention;

[0039] Figure 3 This is a schematic diagram of the physical device structure of the present invention;

[0040] Figure 4 This is a schematic diagram of the camera structure of the end effector joint of the collaborative robot of the present invention.

[0041] In the picture: 1. Massage bed; 2. Automatic smoke extraction moxibustion tube; 3. Multi-joint collaborative robot; 4. Cabinet; 5. High-definition full-color camera; 6. Infrared thermal camera. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] like Figure 1 and Figure 2 As shown:

[0044] Example 1: This invention provides a dynamic acupoint positioning and moxibustion system integrating multimodal vision, comprising:

[0045] Image acquisition module: includes a high-definition full-color camera and an infrared thermal camera, used to acquire RGB images and thermal images of the patient's body surface, respectively.

[0046] Preprocessing module: Normalizes the acquired RGB and thermal images using the following formula: Where I raw I represents the original image pixel values. normalized These are the normalized pixel values. Simultaneously, the labeled acupoint coordinates are preprocessed to generate soft labels. The soft label generation formula is as follows: within a region centered at the acupoint coordinates (x, y) with a radius of R (generally 6–12 for a 600x800 image), linear interpolation is used for filling, with a fill value of 0.3 at the edges and 1.0 at the center.

[0047] Data partitioning module: The preprocessed image data and soft labels are divided into training dataset and test dataset in a ratio of 80% and 20%, respectively.

[0048] The deep learning model module adopts a U-Net+HRNet parallel structure. The input is the stitched RGB and heatmap images, and the output is the coordinates and classification information of the detected acupoints. The model output shape is (B, C, H, W), where B is the batch size, C is the number of acupoints to be detected (362, the standard number of acupoints), and H and W are the height and width of the original image, respectively.

[0049] Training and Evaluation Module: The model is trained using the training dataset. The loss function for each parallel architecture uses the mean squared error (MSE), as shown in the formula: Where y i For the true value, Let N be the predicted value and N be the number of samples. The total loss is the sum of the losses of the four parallel structures. The accuracy of the model is evaluated using a test dataset, with a maximum deviation of 25 pixels for each acupoint considered a correct prediction.

[0050] Dynamic positioning and moxibustion execution module: This includes a multi-joint collaborative robot and an automatic smoke-exhausting moxibustion cylinder. The collaborative robot dynamically adjusts the position of its end joints based on model predictions to achieve precise positioning of acupoints and activates the moxibustion cylinder for moxibustion therapy.

[0051] Control module: Responsible for the coordinated operation of the entire system, including image acquisition control, model prediction control, robot motion control, and moxibustion tube start / stop control.

[0052] As shown above, this system, by integrating a high-definition full-color camera and an infrared thermal camera, achieves simultaneous acquisition of RGB and thermal images of the patient's body surface, providing a rich data source for subsequent high-precision acupoint location. The normalization processing and soft label generation technology adopted by the system effectively improves the quality of image data and the efficiency of model training, providing more accurate and robust input features for deep learning models.

[0053] The deep learning model module adopts a U-Net+HRNet parallel structure, cleverly integrating information from RGB images and thermal images to output the detected acupoint coordinates and classification information. Its output shape is rationally designed to meet the needs of actual acupoint localization. The training and evaluation module ensures the model's prediction accuracy and stability through a mean squared error loss function and rigorous accuracy evaluation criteria, providing a reliable basis for the dynamic localization and moxibustion execution module.

[0054] The dynamic positioning and moxibustion execution module leverages the flexibility of a multi-joint collaborative robot to dynamically adjust the position of the end joints based on model predictions, achieving precise acupoint positioning and automated moxibustion therapy. The control module, acting as the "brain" of the entire system, effectively coordinates image acquisition, model prediction, robot movement, and the start and stop of the moxibustion cylinder, ensuring the overall performance and safety of the system and bringing a revolutionary change to moxibustion therapy.

[0055] Example 2: This example is basically the same as the previous example, except that it also includes a control method for a dynamic acupoint positioning moxibustion system that integrates multimodal vision, comprising the following steps:

[0056] S1. Obtain RGB image samples and the corresponding acupoint coordinates and heatmaps. The image samples (RGB images and heatmaps) are obtained through actual data collection, and the corresponding acupoints are manually labeled.

[0057] S2. Preprocess the original image and the acupoint coordinates.

[0058] S2.1 Normalize the RGB image and heatmap, i.e., divide the pixel value of each pixel by 255.

[0059] Ax' i,j,k =x i,j,k / 255

[0060] i, j, k represent the width, height, and depth positions of the RGB image or heatmap. The heatmap is a single-channel image.

[0061] B. When inputting, the RGB and heatmap are concatenated along the channel dimension.

[0062] S2.2. Coordinate preprocessing. The coordinate values ​​are used to generate corresponding labels. This is similar to the label creation method for U-Net networks.

[0063] A. For a certain RGB image, the corresponding label is assumed to be: (x1, y1, c1), (x2, y2, c2), (x2, y2, c2), ..., where x1 and y1 are the coordinates of the acupoint in the width and height directions in the image, and c1 is the category label corresponding to this acupoint.

[0064] B. Generate a blank label L (all values ​​are 0), with the shape: 1, C, H, W. (C is 362, the standard number of acupoints).

[0065] C. Fill labels to blank labels. For label (x, y, c), fill the area within a radius R (a hyperparameter, typically 6-12 for a 600x800 image) centered at x and y. The filling rule is linear interpolation along the radius, with 0.3 filling at the outermost edge and 1.0 filling at the center; at channel C, this corresponds to c.

[0066] S3. Divide the data obtained from step S2 into two parts: a training dataset and a test dataset.

[0067] First, shuffle the data and then randomly divide all the preprocessed data into training dataset C1 and test dataset C2, with a ratio of 80% and 20% respectively.

[0068] S4. Construct a deep learning model; the model feature map output is as follows: Figure 2 As shown. The network adopts a U-Net+HRNet parallel structure, and the RGB image and heat map are superimposed on the channel and input into the network. The final output shape is (B, C, H, W), where C is the number of acupoints to be detected (C is 362, the standard number of acupoints), and HW are the height and width of the original image.

[0069] S4.1, such as Figure 2 The four parallel structures shown have relative feature map sizes of 1x, 2x, 4x, and 8x, respectively.

[0070] S4.2, Number of 1x Feature Map Channels:

[0071] 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 32, 32, 32, C.

[0072] S4.3, 2x Feature Map Channel Count:

[0073] 4, 8, 8, 16, 16, 32, 32, 64, 64, 64, 128, 128.

[0074] S4.4, 4x Feature Map Channel Count:

[0075] 16, 32, 32, 64, 64, 128, 128, 256.

[0076] S4.5, 8x Feature Map Channel Count:

[0077] 32, 128, 256, 512.

[0078] S4.6, the four parallel structures all output at the end: output, output1, output2, and output3. The shapes of all of them are (B, C, H, W).

[0079] S5. Train the model obtained in step S4 using the training data obtained in step S3. The loss function for each parallel structure is MSE, and the total loss is the sum of the losses of the four parallel structures.

[0080] The output and label formats of each branch of the network are consistent, so the loss is calculated directly using MSE.

[0081] S6. Decode the output prediction to obtain the acupoint coordinates.

[0082] A threshold of 0.8 is set, and the maximum value is taken from the output at the channel layer to determine the acupoint classification. Since adjacent points on a single acupoint will also be detected, a weighted average is used to obtain the final result points.

[0083] Let the coordinates of a certain acupoint be (xi, yi, pi), where x and y are pixel coordinates and p is the corresponding probability.

[0084] but:

[0085] S7. Based on the prediction model evaluation metrics, use test set data to assess the model's accuracy and obtain the model evaluation results. A maximum deviation of 25px is defined as a correct prediction (i.e., the distance between the predicted coordinates and the label coordinates) for each acupoint; otherwise, the prediction is incorrect. Output can be used during the inference phase.

[0086] S8. Determine if the model accuracy meets the requirements. If yes, use the prediction model to predict the RGB image and heatmap. Otherwise, return to step S2.

[0087] S9. After obtaining an accurate model, start the collaborative robot, start the moxibustion tube, and perform moxibustion on the corresponding acupoints.

[0088] As shown above, this method starts with image acquisition, simultaneously acquiring RGB and thermal images of the patient's body surface using a high-definition full-color camera and an infrared thermal camera, providing a comprehensive and accurate data foundation for subsequent processing. In the image preprocessing stage, not only is the image normalized, but soft labels are also innovatively generated. Label values ​​are filled into specific regions using linear interpolation, effectively improving the targeting and effectiveness of model training.

[0089] The data partitioning step in the control method ensures a reasonable allocation of training and testing datasets, providing strong support for model construction and training. In terms of model construction, a U-Net+HRNet parallel structure is adopted, fully utilizing the advantages of both networks to improve the accuracy of acupoint localization. During training, the model is optimized using the MSE loss function, with the total loss being the sum of the losses of the four parallel structures, ensuring the overall performance of the model. The model evaluation step uses rigorous accuracy evaluation criteria to verify the trained model. If the accuracy does not meet the requirements, preprocessing parameters are adjusted promptly or data is re-collected, ensuring the reliability and stability of the model.

[0090] Once the model accuracy meets the requirements, the control method enters the dynamic positioning and moxibustion execution phase. The multi-joint collaborative robot dynamically adjusts the position of its end joints based on the model's predictions, achieving precise acupoint positioning and providing accurate target points for moxibustion. Subsequently, the automatic smoke-exhausting moxibustion cylinder is activated, performing moxibustion on the accurate acupoints, ensuring both therapeutic efficacy and improved patient comfort. The entire control method is logically clear and rigorously executed, providing strong support for the practical application of a dynamic acupoint positioning moxibustion system integrating multimodal vision.

[0091] like Figure 3 and Figure 4 As shown:

[0092] Example 3: This example is basically the same as the previous example, except that an intelligent moxibustion device is provided, comprising:

[0093] Massage bed 1: Provides a platform for patients to lie down, ensuring their comfort and stability during moxibustion.

[0094] Multi-joint collaborative robot 3: Equipped with an automatic smoke extraction moxibustion tube, it has multi-degree-of-freedom motion capabilities and can precisely adjust the position and posture of the moxibustion tube.

[0095] Automatic smoke extraction moxibustion tube 2: Installed at the end of the multi-joint collaborative robot 3, it is responsible for performing moxibustion on accurately positioned acupoints and has an automatic smoke extraction function to reduce the interference of smoke generated during moxibustion on the patient.

[0096] Cabinet 4: The multi-joint collaborative robot 3 is installed on cabinet 4. Cabinet 4 contains necessary components such as electrical controllers and has four wheels at the bottom for easy movement and deployment of the equipment.

[0097] High-definition full-color camera 5: Installed inside the end joint of the multi-joint collaborative robot 3, it is responsible for acquiring high-definition full-color RGB images of the patient's body surface.

[0098] Infrared thermal camera 6: Installed inside the end joint of the multi-joint collaborative robot 3, arranged side by side with high-definition full-color camera 5, responsible for collecting infrared thermal images.

[0099] Detailed usage instructions:

[0100] Step 1: Equipment Initialization and Patient Preparation

[0101] Move the smart moxibustion device to the designated treatment area and connect it to a power source for initialization.

[0102] The patient lies on the massage bed, adjusts to a comfortable position, and prepares to receive moxibustion treatment.

[0103] Step 2: Image Acquisition

[0104] Start the collaborative robot and control its end joints to rotate to an appropriate angle so that the high-definition full-color camera and the infrared thermal camera are respectively aimed at the areas of the patient's body surface that need moxibustion.

[0105] A high-definition full-color camera captures RGB images of the patient's body surface, while an infrared thermal camera simultaneously captures thermal images of the same area.

[0106] Step 3: Image Preprocessing and Data Fusion

[0107] The acquired RGB and thermal images were normalized.

[0108] The normalized RGB image and the thermal image are stitched together along the channel dimension to form fused image data.

[0109] Step 4: Acupoint Location Algorithm Processing

[0110] The fused image data is input into a pre-trained deep learning model, which uses a U-Net+HRNet parallel structure to efficiently extract visual and thermal imaging features from the image.

[0111] The model output consists of the detected acupoint coordinates and classification information. The decoding algorithm converts the model output into the actual acupoint coordinates.

[0112] Step 5: Dynamic Tracking and Attitude Compensation

[0113] During the moxibustion process, the collaborative robot continuously collects image data of the patient's body surface through a camera and inputs it into a deep learning model for processing in real time.

[0114] The model dynamically adjusts the acupoint coordinates based on newly acquired image data, enabling real-time tracking and posture compensation for minor adjustments to the patient's lying posture, ensuring that the moxibustion tube is always aligned with the accurate acupoint.

[0115] Step 6: Moxibustion therapy is performed.

[0116] Based on the acupoint coordinates output by the model, the collaborative robot precisely adjusts the position of its end joints to align the automatic smoke-exhausting moxibustion tube with the target acupoint.

[0117] Turn on the automatic smoke extraction moxibustion cylinder and begin moxibustion at the precise acupoints. During the moxibustion process, the automatic smoke extraction system effectively removes smoke, maintaining a clean and comfortable treatment environment.

[0118] Step 7: End of treatment and return equipment to its proper place

[0119] After the preset moxibustion time is reached, the moxibustion tube is automatically turned off, and the collaborative robot returns its end joints to their initial positions.

[0120] Inform the patient that the treatment is over and the patient should get up and leave the massage bed.

[0121] Move the smart moxibustion device to the storage area and perform necessary cleaning and maintenance.

[0122] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0123] The accompanying drawings of the embodiments disclosed in this invention only involve structures relevant to the embodiments disclosed in this invention. Other structures can be referred to with common designs. Unless otherwise specified, the same embodiment and different embodiments of this invention can be combined with each other.

[0124] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic acupoint positioning and moxibustion system integrating multimodal vision, characterized in that, include: Image acquisition module: used to acquire RGB images and thermal images of the patient's body surface. The RGB images are acquired by a high-definition full-color camera, and the thermal images are acquired by an infrared thermal camera. Preprocessing module: Normalizes the acquired RGB and thermal images and stitches the preprocessed image data along the channel dimension; at the same time, it preprocesses the marked acupoint coordinates to generate corresponding soft labels. Data partitioning module: Divides the preprocessed image data and corresponding soft labels into training dataset and test dataset; Deep learning model module: The deep learning model is constructed using a U-Net+HRNet parallel structure. The input is the stitched RGB and thermal images, and the output is the coordinates and classification information of the detected acupoints. Training and Evaluation Module: Trains the deep learning model using the training dataset, employing mean squared error (MSE) as the loss function; evaluates the accuracy of the trained model using the test dataset to determine whether the model's accuracy meets the preset requirements. Dynamic positioning and moxibustion execution module: including a multi-joint collaborative robot and an automatic smoke-exhausting moxibustion tube. The end joints of the collaborative robot are equipped with high-definition full-color cameras and infrared thermal cameras. Based on the prediction results of the trained deep learning model, the position of the end joints is dynamically adjusted to achieve precise positioning of acupoints and start the moxibustion tube to perform moxibustion therapy. Control module: Responsible for the coordinated operation of the entire system, including image acquisition control, model prediction control, robot motion control, and moxibustion tube start / stop control.

2. The dynamic acupoint positioning and moxibustion system integrating multimodal vision as described in claim 1, characterized in that, The preprocessing module further includes: Image normalization unit: Normalizes the pixel value of each pixel in the RGB image and thermal image by dividing it by 255; Coordinate preprocessing unit: Generates soft labels based on the marked acupoint coordinates. The soft labels are filled with linear interpolation within a preset radius, centered on the acupoint coordinates. The fill value at the edge is 0.3, and the fill value at the center is 1.

0.

3. The dynamic acupoint positioning and moxibustion system integrating multimodal vision as described in claim 1, characterized in that, The U-Net+HRNet parallel structure in the deep learning model module further includes: Four parallel feature extraction branches: the original image is 1x, 2x, 4x and 8x larger than the feature map, respectively. Each branch performs independent loss calculation after outputting the feature map. Feature fusion and output unit: The outputs of the four parallel branches are superimposed in the channel dimension, and the output shape is (B, C, H, W), where B is the batch size, C is the number of acupoints to be detected, and H and W are the height and width of the original image, respectively.

4. The dynamic acupoint positioning and moxibustion system integrating multimodal vision as described in claim 1, characterized in that, The training and evaluation module further includes: Loss calculation unit: The loss function for each parallel structure uses mean squared error (MSE), and the total loss is the sum of the losses of the four parallel structures; Model evaluation unit: Based on the prediction model evaluation index, the accuracy of the model is evaluated using test set data, and the maximum deviation of each acupoint is limited to 25 pixels to be considered a correct prediction.

5. The dynamic acupoint positioning and moxibustion system integrating multimodal vision as described in claim 1, characterized in that, The dynamic positioning and moxibustion execution module further includes: Multi-joint collaborative robot: Two cameras are installed inside the end joint, and RGB images and thermal images are captured separately by rotating the end joint; Automatic smoke-exhausting moxibustion tube: Installed at the end of the collaborative robot, it performs moxibustion on accurate acupoints based on model predictions, and also has an automatic smoke exhaust function.

6. A control method for a dynamic acupoint positioning moxibustion system integrating multimodal vision as described in any one of claims 1 to 5, characterized in that, Includes the following steps: The image acquisition module is activated to obtain RGB and thermal images of the patient's body surface; The acquired image data is transmitted to the preprocessing module for normalization and stitching, and the coordinates of the marked acupoints are preprocessed to generate soft labels. The preprocessed image data and soft labels are transmitted to the data partitioning module and divided into training dataset and test dataset. The training dataset is input into the deep learning model module for training, and the trained model is evaluated using the test dataset. Based on the evaluation results, determine whether the model accuracy meets the preset requirements. If it does not, return to the preprocessing module to adjust the preprocessing parameters or re-collect data. If the model accuracy meets the requirements, the dynamic positioning and moxibustion execution module is activated. Based on the model prediction results, the position of the end joint of the collaborative robot is dynamically adjusted to achieve precise positioning of acupoints, and the moxibustion tube is activated to perform moxibustion therapy.

Images