Intelligent Moxibustion Heat Therapy Robot Based on Infrared-Visible Light Fusion Positioning and Its Control Method

Abstract

The present invention relates to an intelligent moxibustion thermotherapy robot based on infrared-visible light fusion positioning and its control method, belonging to the fields of medical equipment and image processing. The robot includes an infrared thermal imaging scanner, a visible light optical camera, an intelligent moxibustion thermotherapy instrument, and a control method installed inside the robot. The robot adopts infrared-visible light image fusion registration and positioning technology. According to the characteristic regions and characteristic points that need to be treated determined manually, combined with intelligent electromechanical measurement and control software and hardware components for moxibustion thermotherapy, through technologies such as three-dimensional surface self-positioning of the human body, automatic detection of human body surface temperature, automatic ash shaking of moxibustion burning strips, automatic switching of treatment points with controllable trajectories, visual interaction throughout the treatment process, and abnormal warning safety protection, it realizes automatic moxibustion thermotherapy for the meridian acupoints or designated parts of the patient, and has functions such as intelligent moxibustion thermotherapy, autonomous controllability, and customizable design.

Description

Technical Field

[0001] The present invention belongs to the fields of medical devices and image processing, and relates to technologies such as infrared thermal imaging, image processing, computer vision, and mechanical control. In particular, it relates to an intelligent moxibustion thermotherapy robot based on infrared-visible light fusion positioning and its control method. Background Art

[0002] With the continuous development of medical technology and the increasing attention of people to health, the application prospect of infrared thermal imaging technology in the medical field has become broader. However, traditional infrared thermal imaging technology has some limitations, such as being sensitive to the patient's body position, unable to adjust the body position in real time, and lacking automated treatment means, etc. These limitations restrict its efficiency and accuracy in clinical applications. At the same time, as a traditional Chinese medical treatment method, moxibustion has a long history and rich experience in promoting health, relieving pain, and improving blood circulation. However, traditional moxibustion means are limited by the doctor's experience and manual operation, lacking personalized and precise treatment methods.

[0003] For example, the prior art with the publication number CN117503582A proposes a control method for a moxibustion device. The control method includes: collecting the thermal imaging information of the moxibustion target to obtain a first target image; controlling the moxibustion device to perform moxibustion operations on the moxibustion target; collecting the thermal imaging information of the moxibustion target to obtain a second target image; and performing data comparison and analysis based on the first target image and the second target image to generate a data comparison result.

[0004] Traditional infrared thermal imaging technology has certain limitations in the medical application of moxibustion, which is highlighted by the insufficient accuracy of simply using infrared light to locate the target point. This is mainly because infrared light cannot provide enough details to accurately locate the treatment area. Especially in areas with dense tissues or small pores, its resolution is not sufficient to accurately capture the target point. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an intelligent moxibustion thermotherapy robot based on infrared-visible light fusion positioning and its control method, aiming to overcome the limitations of traditional infrared thermal imaging technology in medical applications, as well as the problems of doctor experience dependence and insufficient manual operation in traditional moxibustion treatment. It realizes personalized and precise treatment for patients by developing an intelligent moxibustion thermotherapy robot based on infrared-visible light fusion positioning, combining the whole-body infrared imaging image of the human body and the pathological analysis of traditional Chinese and Western medicine. The robot uses intelligent algorithms and a three-axis mechanical motion platform to automatically adjust treatment parameters and positions to ensure the smooth progress of the moxibustion treatment process. Its purpose is to provide an efficient, precise, and personalized medical service to promote the improvement of people's life and health levels.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An intelligent moxibustion heat therapy robot based on infrared-visible light fusion positioning, the robot includes: an infrared thermal imaging scanner, a visible light optical camera, a bed frame, a three-axis mechanical motion platform, and an information processing module; the bed frame includes a camera bracket disposed beside the bed frame, the three-axis mechanical motion platform is disposed on the bed frame, and the information processing module is disposed on the three-axis mechanical motion platform; the infrared thermal imaging scanner and the visible light optical camera are mounted on the camera bracket, and both the infrared thermal imaging scanner and the visible light optical camera are directed towards the end of the moxa stick; the information processing module performs key point detection on the infrared thermal imaging image obtained by the infrared thermal imaging scanner to obtain a human body sparse key point distribution map and a human body dense key point distribution map; the information processing module performs semantic segmentation and sparse key point detection on the optical image obtained by the visible light optical camera to obtain a human body part segmentation map and a human body sparse key point distribution map; the information processing module compares the sparse key point distribution maps of the infrared thermal imaging image and the optical image, uses the segmentation area as a judgment criterion, and adjusts the patient's posture according to the posture adjustment strategy; the visible light optical camera obtains the optical image again, and the information processing module performs key point detection to obtain a human body dense key point distribution map; the information processing module compares the human body dense key point distribution maps in the infrared thermal imaging image and the optical image, determines the two-dimensional coordinates of the target area according to the key points, maps the two-dimensional coordinates from the infrared thermal imaging image to the optical image, and then converts them into the coordinates of the end effector of the three-axis mechanical motion platform.

[0008] Further, the robot further includes a moxa stick clamping device, and the moxa stick clamping device is disposed on the three-axis mechanical motion platform; the robot further includes a distance sensing and calibration module, a temperature sensing module, a voice interaction module, and a drive control module, which are also disposed on the three-axis mechanical motion platform; the distance sensing and calibration module detects the distance between the end of the moxa stick and the plane where the patient is located in real time, calculates the deviation between the end of the three-axis mechanical motion platform and the target area, and then performs automatic calibration according to the deviation; the temperature sensing module monitors the temperature change of the human body surface in the moxibustion area in real time; when the voice interaction module detects an abnormal event, it reminds the user or operator by sending an alarm signal; the drive control module is used to control the actuator in the three-axis mechanical motion platform. The three-axis mechanical motion platform is a three-axis gantry structure, and each axis is controlled by a stepper motor; the three-axis mechanical motion platform drives the end of the moxa stick on the moxa stick clamping device to move and position in three directions of the three-axis mechanical motion platform; the stepper motor is driven by a controller and realizes the motion control of the mechanical structure according to the received instruction; the serial communication part adopts the serial communication protocol and realizes the connection with the controller through the CH340G module. The data transmission adopts the UART and PWM protocols. The controller receives and parses the data from the server, and then sends the corresponding instruction to the stepper motor.

[0009] Furthermore, the robot further includes an automatic ash shaking module and an ash receiving tray disposed at the end of the head of the bed. A pressure sensor is provided at the bottom of the ash receiving tray, and a mica sheet is placed on top of the pressure sensor. During ash shaking, the moxa stick is pressed against the mica sheet to make the ash fall. The pressure sensor detects the poking force. When the pressure sensor senses a change in force, the moxa holding device stops descending. The drive control module calculates the length of the moxa stick by calculating the descending height, so as to adjust the descending height of the device during moxibustion.

[0010] The present invention also proposes a control method for an intelligent moxibustion thermotherapy robot based on infrared-visible light fusion positioning applicable to the aforementioned robot. The control method includes the following steps:

[0011] S1: Obtain the infrared thermal image and optical image of the human body through an infrared thermal imager and a visible light optical camera respectively, and send the infrared thermal image and optical image to the information processing module and the interactive diagnosis and treatment module;

[0012] S2: Perform key point detection on the obtained infrared thermal image through the information processing module to obtain an infrared sparse key point distribution map and an infrared dense key point distribution map; perform semantic segmentation and sparse key point detection on the obtained optical image through the information processing module to obtain a human body part segmentation map and an optical sparse key point distribution map;

[0013] S3: By comparing the key points in the infrared sparse key point distribution map and the optical sparse key point distribution map, and using the segmentation area as the judgment criterion, give a patient posture adjustment strategy;

[0014] S4: Obtain the optical image of the patient after posture adjustment through the visible light optical camera, and upload the obtained optical image to the information processing module for human body dense key point detection to obtain an optical dense key point distribution map;

[0015] S5: The information processing module compares the key points of the infrared dense key point distribution map and the optical dense key point distribution map, calculates the position relationship between the target area in the artificially selected infrared thermal image and the key points to determine the two-dimensional coordinates of the target area; then maps the target area in the infrared thermal image to the optical image through coordinate mapping to obtain the two-dimensional coordinates of the target area in the optical camera coordinate system, and converts it into the coordinates of the end effector of the three-axis mechanical motion platform;

[0016] S6: The drive control module obtains the coordinates of the end effector of the three-axis mechanical motion platform from the information processing module, and controls the moxa holding device to move to the corresponding position.

[0017] Further, in step S1, after receiving the infrared thermal image and the optical image, the information processing module first preprocesses the infrared thermal image and the optical image; the infrared image preprocessing includes denoising, enhancement, and normalization; the optical image preprocessing includes correction, alignment, and denoising.

[0018] Further, in step S2, the information processing module constructs the key point distribution maps of the front and back of the patient's body under the infrared thermal image through the key point detection algorithm, and then optimizes the key point positions through the multi-stage optimization strategy;

[0019] The information processing module divides the human body parts in the optical image through the semantic segmentation algorithm. The semantic segmentation algorithm divides the human body into at least the following regions: left upper arm (aa), left lower arm (ab), left hand (ac), right upper arm (ba), right lower arm (bb), right hand (bc), left thigh (ca), left lower leg (cb), left foot (cc), right thigh (da), right lower leg (db), right foot (dc), torso (e), head (f); the area of each part is represented by S i denoted.

[0020] Further, in step S3, based on the positions of several key points on the infrared image and the line segment angles, the offsets of several key points on the optical image relative to the reference and the angle deviation of the line connecting two points are calculated respectively, and the position of the human body part is scored according to the preset standard. The score value is used as the criterion for judging the pose accuracy, and the score value threshold is used to judge whether the pose meets the standard; the comparison of the areas of each limb part of the human body in the same view is added, and the key point positions and the connecting line angles are used as the guiding method for the patient's pose adjustment. After the offset between the measured value and the standard value is less than a certain threshold, the semantic segmentation area coincidence degree is used as the final standard.

[0021] Further, in step S5, the information processing module realizes coordinate conversion by combining the target point positioning algorithm and the coordinate mapping algorithm, which specifically includes the following processes:

[0022] Obtain the dense key point distribution maps of the human body of the infrared thermal image and the optical image;

[0023] Select the human body key points on the infrared image as the target points, and locate the target area according to the target points;

[0024] Select two key points adjacent to the target point in the target area to form a triangle;

[0025] Calculate the position of the target point on the optical image through the properties of similar triangles,

[0026] Convert the two-dimensional optical coordinates of the target point on the optical image into the end device coordinates;

[0027] Move the stepping motor by a certain number of steps, record the actual moving distance, and use the calibration relationship of the stepping motor to convert the number of motor steps into the actual three-dimensional space distance;

[0028] According to the relationship between the number of motor steps and the actual three-dimensional space distance, convert the coordinates of the end device corresponding to the target point into the actual three-dimensional space coordinates of the end device.

[0029] Furthermore, in step S5, mapping the selected area on the infrared imaging map to the real body coordinate system of the patient requires using the positions of the key points and the mapping relationship between the infrared image and the visible light image; select n key points of the human body, and use the infrared thermal image key point detection model to detect the n key points of the human body on the front and back of the body respectively; use the optical image key point detection model to detect the n key points of the human body on the front or back of the body in real time; the key points in the infrared image and each key point in the optical image have numbers and correspond one by one (k i , p i ), and the position coordinate information of the key points is represented by the two-dimensional coordinates (x i , y i ); artificially select the target point, and the position information of the target point on the thermal imaging image can be obtained; calculate the relative position relationship between the target point in the infrared thermal imaging map and each human body key point, and this relationship is represented by the Euclidean distance α i between the target point O and each key point k i and the direction angle α i ; map this relationship directly to the real-time optical image, starting from p i , find the mapped point A i at a distance of d i in the direction of α i , and all the A i form a set A; use the data fitting algorithm, by setting the loss and reducing the loss to the minimum, so as to obtain the two-dimensional coordinates of the final target point OA.

[0030] Furthermore, in step S6, the process of controlling the actuator of the three-dimensional mechanical motion platform for moxibustion according to the target point includes the following contents:

[0031] Obtain the actual three-dimensional space coordinates of the end of the device corresponding to the target point;

[0032] Perform orthogonal direction solution on the actual three-dimensional space coordinates of the end of the device corresponding to the target point to determine the relative position of the target point on the three-dimensional coordinate axes;

[0033] Plan the motion path of the three-dimensional mechanical motion platform;

[0034] Control the moxibustion clamping device of the three-dimensional mechanical motion platform to reach the actual three-dimensional space coordinates of the end of the device corresponding to the target point and perform the moxibustion operation.

[0035] The beneficial effects of the present invention are as follows:

[0036] The present invention improves the accuracy and efficiency of moxibustion treatment. By using infrared thermal imaging technology to monitor the patient's body temperature distribution and lesion conditions in real time, and combining machine learning algorithms for data analysis, it can accurately identify diseases and lesion sites, achieving personalized and precise treatment target positioning. The intelligent robot has the function of automatically adjusting moxibustion treatment parameters and positions, and can perform personalized treatment according to the actual situation of the patient, improving the degree of personalization of treatment. At the same time, by combining the infrared imaging image of the whole human body and the distribution of human meridians, the combined traditional Chinese and Western medicine pathological analysis is carried out, providing a more scientific and effective basis for treatment. Using advanced medical imaging technology and artificial intelligence algorithms to provide personalized and precise treatment services for patients promotes the improvement of the people's life and health level.

[0037] Other advantages, objectives and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. Brief Description of the Drawings

[0038] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be described in preferred detail below in conjunction with the drawings, where:

[0039] Figure 1 is a schematic structural diagram of the robot of the present invention;

[0040] Figure 2 is an overall framework diagram of the robot of the present invention;

[0041] Figure 3 is a flowchart of the robot control method of the present invention;

[0042] Figure 4 is a detailed flowchart of the robot control method under an embodiment;

[0043] Figure 5 is a schematic diagram of the division of human body parts after semantic segmentation;

[0044] Figure 6 is a schematic diagram of the process of posture adjustment;

[0045] Figure 7 is a flowchart of the coordinate mapping algorithm;

[0046] Figure 8 is a schematic diagram of the process of coordinate mapping;

[0047] Figure 9 Schematic diagram of key points of the human body proposed by the present invention;

[0048] Figure 10 Schematic diagram of key point matching in an embodiment;

[0049] Figure 11 Drive control flowchart of a three-dimensional mechanical motion platform. Specific embodiments

[0050] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0051] Among them, the drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and should not be construed as a limitation to the present invention; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0052] In the drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the drawings are only for illustrative purposes and should not be construed as a limitation to the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0053] Please refer to Figures 1 to 11 , an intelligent moxibustion thermotherapy robot based on infrared-visible light fusion positioning and its control method.

[0054] Such as Figure 1An intelligent moxibustion heat therapy robot based on infrared-visible light fusion positioning is shown. The robot includes: an infrared thermal imaging scanner, a visible light optical camera, a bed frame, a three-axis mechanical motion platform, and an information processing module; the bed frame includes a camera support arranged beside the bed frame, the three-axis mechanical motion platform is arranged on the bed frame, and the information processing module is arranged on the three-axis mechanical motion platform; the infrared thermal imaging scanner and the visible light optical camera are installed on the camera support, and both the infrared thermal imaging scanner and the visible light optical camera are directed towards the end of the moxa stick.

[0055] The information processing module performs key point detection on the infrared thermal imaging image obtained by the infrared thermal imaging scanner to obtain a human body sparse key point distribution map and a human body dense key point distribution map; the information processing module performs semantic segmentation and sparse key point detection on the optical image obtained by the visible light optical camera to obtain a human body part segmentation map and a human body sparse key point distribution map; the information processing module compares the sparse key point distribution maps of the infrared thermal imaging image and the optical image, uses the segmentation area as a judgment criterion, and adjusts the patient's posture according to the posture adjustment strategy; the visible light optical camera obtains the optical image again, and the information processing module performs key point detection to obtain a human body dense key point distribution map; the information processing module compares the human body dense key point distribution maps in the infrared thermal imaging image and the optical image, determines the two-dimensional coordinates of the target area according to the key points, maps the two-dimensional coordinates from the infrared thermal imaging image to the optical image, and converts them into the coordinates of the end effector of the three-axis mechanical motion platform.

[0056] As Figure 2 Shown in the overall framework diagram of the robot, the robot adopts infrared-visible light image fusion registration and positioning technology, uses the pathological analysis of integrated traditional Chinese and Western medicine to discover the characteristic regions and characteristic points that need treatment of the patient, combines the intelligent electromechanical measurement and control software and hardware components of moxibustion heat therapy, and through technologies such as three-dimensional surface self-positioning of the human body, automatic detection of human body surface temperature, automatic ash shaking of the moxa burning stick, automatic switching of treatment points with controllable trajectories, visual interaction throughout the treatment process, and abnormal warning safety protection, realizes the automatic moxibustion heat therapy of the meridian acupoints or specified parts of the patient, has the characteristics of intelligent moxibustion heat therapy, autonomous controllability, customizable design, etc., is a new method for realizing intelligent, safe, and efficient moxibustion treatment of human diseases, and has broad application prospects in the field of medical health.

[0057] As Figure 3 Shown is a control method for an intelligent moxibustion heat therapy robot based on infrared-visible light fusion positioning. The control method includes the following steps:

[0058] S1: Respectively obtain the infrared thermal imaging and optical image of the human body through the infrared thermal imager and the visible light optical camera, and send the infrared thermal imaging and the optical image to the information processing module and the interactive diagnosis and treatment module;

[0059] S2: Detect key points of the acquired infrared thermal imaging through the information processing module to obtain an infrared sparse key point distribution map and an infrared dense key point distribution map; perform semantic segmentation and sparse key point detection on the acquired optical image through the information processing module to obtain a human body part segmentation map and an optical sparse key point distribution map;

[0060] S3: Compare the key points in the infrared sparse key point distribution map and the optical sparse key point distribution map, and use the segmentation area as the judgment criterion to give a patient posture adjustment strategy;

[0061] S4: Obtain the optical image of the patient after posture adjustment through the visible light optical camera, and upload the obtained optical image to the information processing module for human body dense key point detection to obtain an optical dense key point distribution map;

[0062] S5: Compare the key points of the infrared dense key point distribution map and the optical dense key point distribution map through the information processing module, calculate the positional relationship between the target area and the key points in the artificially selected infrared thermal imaging image to determine the two-dimensional coordinates of the target area; then map the target area in the infrared thermal imaging image to the optical image through coordinate mapping to obtain the two-dimensional coordinates of the target area in the optical camera coordinate system, and convert it into the coordinates of the end effector of the three-axis mechanical motion platform;

[0063] S6: The drive control module obtains the coordinates of the end effector of the three-axis mechanical motion platform from the information processing module and controls the movement of the moxibustion clamping device to the corresponding position.

[0064] Preferably, in the intelligent robot and its control method of the present invention, it at least includes an image processing algorithm, a key point detection algorithm, a semantic segmentation algorithm, a feature extraction algorithm, a coordinate mapping algorithm, and a mechanical device control algorithm. The image processing algorithm is used to detect and identify abnormal temperature regions in the image, realizing real-time monitoring of infrared images. In addition, infrared images at multiple time points can be compared to help doctors analyze the trend of temperature changes; the key point detection algorithm is used to identify and calibrate the key points of physiological structures in the image, such as human joints. Through the calibration of key points, the device can more accurately locate the position of the patient; the semantic segmentation algorithm is used to identify and segment the human body region in the infrared thermal imaging image, including parts such as the head, torso, and limbs. Thus, the human body is separated from the background or environment, providing a clearer human body contour; the feature extraction algorithm is used to extract the features of temperature information from the infrared thermal imaging image, including the distribution and intensity of hot spots and heat sources; in addition, the feature extraction algorithm can capture the spatial distribution information of each region in the image, such as the relative positions, distances, and relationships of different regions; the coordinate mapping algorithm is used to align the coordinate systems of the infrared thermal imaging image and the visible light image, enabling the device to accurately determine the position of a specific location in the infrared image in the visible light image, realizing image matching and positioning; the mechanical device control algorithm is used to receive instructions from the system to control and manage mechanical devices related to treating patients, including: intelligent treatment device positioning, intelligent treatment device movement, real-time feedback, etc. Through precise mechanical control, the correct positioning and stability of the patient during the treatment process are ensured.

[0065] Embodiment 1

[0066] This embodiment proposes an intelligent moxibustion thermotherapy robot based on infrared-visible light fusion positioning, as Figure 1 shown. The intelligent robot includes: an infrared thermal imaging scanner, a visible light optical camera, a driving device, and an intelligent information module. Among them, the driving device at least includes a bed frame, a three-axis mechanical motion platform, and a moxibustion clamping device; the intelligent information module at least includes a distance perception and calibration module, a temperature perception module, a voice interaction module, a driving control module, an information processing module, an interactive diagnosis and treatment module, etc.

[0067] Specifically, in the bed frame structure, it includes a camera bracket arranged beside the bed frame and an ash tray arranged at the end of the head of the bed; the three-axis mechanical motion platform is arranged on the bed frame, and the moxibustion clamping device is arranged on the three-axis mechanical motion platform. Moreover, the distance perception and calibration module, temperature perception module, voice interaction module, driving control module, and information processing module of the intelligent information module are also arranged on the three-axis mechanical motion platform.

[0068] Preferably, a visible light optical camera is provided on the camera bracket beside the bed frame, and its pointing direction is towards the end of the moxa stick; an infrared thermal imaging scanner is provided on the camera bracket, and its pointing direction is towards the end of the moxa stick; the interactive diagnosis and treatment module is installed on the side of the head of the bed. The infrared thermal imaging scanner is used to obtain the infrared thermal imaging map of the patient. The visible light optical camera is used to take real-time optical images of the whole body of the patient.

[0069] Preferably, the bed frame is used to accurately position the patient in an appropriate position and assist the three-axis mechanical movement platform to move to the target point selected by the doctor. The three-axis movement platform has three independent movement axes to help the doctor or operator accurately position the body part of the patient. The moxa holding device provided on the intelligent treatment instrument is used to fix the moxa stick and perform treatment plans such as moxibustion on the target point according to the treatment plan.

[0070] Preferably, the distance sensing and calibration module is used to detect the distance between the end of the moxa stick and the plane where the patient is located in real time, determine the Z coordinate axis of the end coordinate system of the three-axis mechanical movement platform (moving up and down in the vertical direction), and then obtain the coordinate of the patient's target area according to the information processing module, and compare it with the real-time position of the patient, calculate the deviation of the X axis (left and right distance) and the Y axis (front and back distance) respectively, and then perform automatic calibration according to the deviation.

[0071] Preferably, the temperature sensing module is used to monitor the temperature change on the human body surface in the moxibustion area in real time. When abnormal temperature or other events that need attention are detected, the voice interaction module can remind the user or operator by sending an alarm signal. At the same time, the controller will receive this signal and control the moxa stick to move to the next treatment point according to the preset safety strategy and program; the drive control module is used to control the actuators in the system, such as stepper motors, DC motors, etc., to realize the control and movement of various mechanical components.

[0072] Preferably, the intelligent robot realizes the automatic moxibustion treatment of the patient's meridian acupoints or designated parts through the fusion registration of visible light and infrared images, combined with intelligent electromechanical measurement and control software and hardware components, and has functions such as three-dimensional surface self-positioning of the human body, automatic detection of the human body surface temperature, automatic ash shaking of the moxa burning stick, automatic switching of treatment points and controllable trajectory, and visual interaction throughout the treatment process, and has characteristics such as intelligent moxibustion treatment, autonomous controllability, and customizable design.

[0073] Preferably, the infrared thermal imager scans the whole body or part of the patient in a non-contact manner, generates a high-resolution infrared thermal imaging map through algorithms such as intelligent interpolation, and then uploads the infrared image to the interactive diagnosis and treatment module and the information processing module.

[0074] Preferably, the information processing module constructs the key point distribution maps of the front and back of the patient's body under infrared thermography through key point detection technology, and uses technologies such as semantic segmentation to divide the feature sensitive points to be treated on each part of the human body; the doctor selects the moxibustion points on the patient's thermal imaging image in the interactive diagnosis and treatment module through the analysis of the patient's main complaint and the infrared thermography, combined with the human body temperature distribution and high and low temperature regions.

[0075] Preferably, the patient lies flat on the tabletop of the treatment platform with three-axis mechanical movement. The visible light optical camera acquires the whole-body optical image of the patient and uploads it to the information processing module, and performs real-time key point detection and precise division of moxibustion points on each part.

[0076] Preferably, the information processing module calculates the three-dimensional coordinates of each moxibustion point and transmits them to the drive control module, and controls the three-axis mechanical platform to drive the moxibustion clamping device to accurately reach the treatment point on the patient's body surface.

[0077] Preferably, the temperature sensing module accurately monitors the skin temperature of the treatment point on the patient's body surface. When the skin temperature at the moxibustion area reaches the high temperature warning value, the drive control module automatically drives the moxibustion clamping device to perform the treatment at the next point.

[0078] Preferably, the information processing module is used to process and analyze various data and images obtained from parts such as the thermal imager, visible light optical camera, distance sensing and calibration module, and temperature sensing module to support the doctor's decision-making and treatment. The interactive diagnosis and treatment module can display the human body images of the patient, including infrared thermography images, optical images, etc. The doctor can view information such as the skin surface temperature distribution of the patient and the characteristics of the moxibustion area through the interface to assist in diagnosis and treatment decision-making.

[0079] Preferably, the three-axis mechanical movement platform adopts a three-axis gantry structure, and each axis is equipped with a stepper motor to achieve precise control. In terms of data transmission, serial communication is adopted, and the server transmits the data to the single-chip microcomputer wired through the CH340G module. The communication protocol adopts UART and PWM. After the single-chip microcomputer receives and analyzes the data, it uses the PWM protocol to accurately drive the motor to achieve more precise and stable control. During the control process, the data transmission rate and stability are optimized, further improving the performance and reliability of the system.

[0080] Preferably, to timely warn of and handle the problems of falling off or scalding caused by too long moxa ash, an automatic ash shaking module device is developed to achieve automatic and controllable falling off of moxa ash and automatic position calibration. A ash receiving tray is arranged at the end of the bed frame, and a pressure sensor is equipped at the bottom thereof, and then a mica sheet with high temperature resistance is covered on the pressure sensor. After a period of treatment, the control device moves to the ash receiving tray, pokes the moxa stick onto the mica sheet to make the ash fall. The poking force is detected by the pressure sensor. Specifically, the threshold value for the pressure sensor to take effect is determined through preliminary experiments. This threshold value should not only be able to poke off the ash but also ensure that the moxa stick will not break. The purpose of detecting the poking force is to ensure that the poking force reaches the set threshold value, thereby triggering the operation of the device to stop moving in the Z-axis direction. When the moxa stick pokes onto the mica sheet, the pressure sensor senses the change in force and the device stops. And the length of the moxa stick is calculated by calculating the descending height, so as to adjust the descending height of the device during the treatment process to achieve dynamic adjustment of ash treatment and the length of the moxa stick.

[0081] Embodiment 2

[0082] This embodiment proposes a control method for an intelligent moxibustion thermotherapy robot based on infrared-visible light fusion positioning applicable to the foregoing embodiment, as Figure 4 shown, which includes the following detailed steps:

[0083] S1: The patient is scanned by an infrared thermal imaging scanner to obtain the infrared thermal imaging image of the human body, and then the obtained infrared thermal imaging image is uploaded to the information processing module and the interactive diagnosis and treatment module for the doctor to view. The doctor views the image information according to the interactive diagnosis and treatment module, combines the patient's main complaint, selects the preferably target area, and formulates a preliminary treatment plan.

[0084] S2: The information processing module performs key point detection on the obtained infrared thermal imaging image, and simultaneously generates a human body sparse key point distribution map and a human body dense key point distribution map.

[0085] S3: The visible light optical camera takes a picture of the patient on the intelligent treatment instrument to obtain the optical image of the human body in real time, and then the obtained optical image is uploaded to the information processing module and the interactive diagnosis and treatment module for the doctor to view.

[0086] S4: The information processing module performs semantic segmentation and sparse key point detection on the obtained optical image, and simultaneously generates a human body part segmentation map and a human body sparse key point distribution map.

[0087] S5: The information processing module adjusts the patient's posture according to the comparison of the sparse key point distribution maps in step S2 and step S4, taking the segmentation area as the judgment criterion.

[0088] S6: The visible light optical camera takes pictures of the patient after adjustment, obtains the optical image of the patient in real time, and then uploads the obtained optical image to the information processing module for dense human key point detection to generate a dense human key point distribution map.

[0089] S7: The information processing module calculates the positional relationship between the target area and the key points according to the dense human key point distribution maps in steps S2 and S6, so as to determine the two-dimensional coordinates of the target area. Then, through coordinate mapping, the target area of the infrared thermal imaging map selected by the doctor is mapped onto the optical image to obtain the two-dimensional coordinates of the target area in the optical camera coordinate system. Then, through calculation, it is converted into the coordinates of the end effector of the three-axis mechanical motion platform.

[0090] S8: The intelligent treatment device obtains the coordinates of the end effector of the three-axis mechanical motion platform from the information processing module and controls the moxibustion clamping device to move to the corresponding position.

[0091] The control method of the present invention further includes using a temperature sensing module to monitor the temperature change on the surface of the moxibustion area of the patient in real time to ensure that the temperature during the treatment is controlled within a safe range and avoid discomfort caused by overheating or overcooling to the patient.

[0092] Specifically, the information processing module obtains the data of the temperature sensing module, and then uses the distance sensing and calibration module to automatically adjust the height of the moxibustion clamping device according to the preset treatment parameters and safety range to ensure the safety and effectiveness of moxibustion treatment. After treating for a certain period of time, the information processing module guides the moxibustion device to move to the ashtray, performs the operation of shaking off the ashes, and then returns to the treatment point for height adjustment to continue the treatment. After treating one target point, the information processing module controls the moxibustion stick to move to the next target point to continue the treatment.

[0093] Further, in step S1, it is necessary to preprocess the infrared thermal imaging image obtained by the infrared thermal imaging scanner. It includes steps such as denoising, enhancement, and normalization to ensure image quality and consistency. The infrared thermal imager scans the whole body or part of the patient in a non-contact manner and generates a high-resolution infrared thermal imaging map through algorithms such as intelligent interpolation, and then uploads the infrared image to the interactive diagnosis and treatment module and the information processing module.

[0094] Further, in step S2, the information processing module constructs the key point distribution maps of the front and back of the patient's body under infrared thermal imaging through key point detection technology. The doctor selects the moxibustion points on the patient's thermal imaging in the interactive diagnosis and treatment module by analyzing the patient's main complaint and the infrared thermal imaging map, combining the human body temperature distribution and high and low temperature areas.

[0095] Further, in step S3, the patient lies flat on the treatment platform of the three-axis mechanical movement. The visible light optical camera acquires the whole-body optical image of the patient and uploads it to the information processing module, and performs real-time key point detection and precise division of moxibustion points on each part.

[0096] Further, in step S4, the information processing module uses technologies such as semantic segmentation to divide the feature sensitive points to be treated on each part of the human body.

[0097] Preferably, for the semantic segmentation of the human body area, the key lies in clearly segmenting the human body from the background and each part of the human body. And it should have sufficient details and accuracy to capture the minute features and areas in the human body posture. In addition, due to the influence of lighting conditions, human skin color, and different backgrounds, it is necessary to have robustness. For the segmentation of infrared thermal images, due to the lack of texture details, a network with a larger number of parameters is required for better fitting; for the segmentation of optical images, since it is necessary to calculate the area of the segmentation region in real time and give feedback, a higher speed is required, so a lightweight model needs to be designed to meet the requirements.

[0098] Preferably, as Figure 5 shown, in the semantic segmentation method of the present invention, the human body is divided into fourteen parts: left upper arm aa, left lower arm ab, left hand ac, right upper arm ba, right lower arm bb, right hand bc, left thigh ca, left lower leg cb, left foot cc, right thigh da, right lower leg db, right foot dc, torso e, and head f. The area of each part is represented by S i denoted.

[0099] Further, in step S5, in order to make the optical image better match the infrared image and obtain more accurate key points, when performing optical imaging, the patient needs to be in a posture relatively similar to that recorded in the infrared image. Therefore, a posture adjustment strategy is designed. According to the position and angle relationship of the key points and the main connection lines, the posture is scored, and the score value is used as the criterion for judging the posture accuracy, and the score value threshold is used as the criterion for whether the posture meets the standard. For example, when selecting two key points "left hand" and "left elbow" and the connection line for judging the posture of the patient's left lower arm, based on the positions of the two key points and the line segment angle in the infrared image, the offsets of the two points relative to the reference in the optical image and the angle deviation of the connection line between the two points are calculated respectively, and the position of the lower arm placement is scored according to the preset standard. In addition, adding the comparison of the areas of each limb part of the human body under the same viewing angle can better handle the problems of limb torsion and key point position errors. Using the key point position and the connection line angle as the guiding method for the patient's posture adjustment, after the offset between the measured value and the standard value is less than a certain threshold, the semantic segmentation area coincidence degree is used as the final standard.

[0100] Preferably, the information processing module first needs to compare the infrared human sparse key point distribution map and the optical human sparse key point distribution map simultaneously, then determine the area of the part to which the key point belongs. Next, the information processing module continuously calculates the area coincidence degree of the area of the part to which the key point belongs in the thermal imaging image and the real-time optical image (coincidence degree = intersection area / minimum area, where the intersection area is the area of the overlapping part of the corresponding areas in the infrared thermal imaging image and the optical image, and the minimum area is the area of the smaller of the two). By setting a threshold for the area coincidence degree, the calculated area coincidence degree is compared with the set threshold. If the coincidence degree is higher than the threshold, it is determined that the posture is correct and the adjustment stops; otherwise, it is determined that the posture is incorrect and the adjustment continues. Due to the complexity and deformation of human postures, the information processing module needs to perform real-time calculations and discriminations of human postures and make timely responses.

[0101] Preferably, during the posture adjustment process, as Figure 6 shown, if two key points "left elbow (AA)" and "left wrist (AB)" and their connecting line are selected for the posture judgment of the patient's left forearm, based on the positions and angles of the two key points in the infrared image, the offsets of the two points in the optical image relative to the reference (the Euclidean distance of the coordinate points) and the angle deviation of the connecting line between the two points (the difference between the angle of the connecting line and the vertical direction) are calculated respectively. According to the preset standard, the patient is reminded to adjust the placement position of the left forearm. After reaching the specified threshold, the ratio of the area of the intersection of the two modalities to the smaller area of the two is used as the standard. If it is less than the minimum threshold of this ratio, the posture is considered accurate. The entire process can follow the basic movement laws of the human body as the basic judgment sequence to determine the full-body posture in turn. For example, A, B, C, D, α 1 、α 2 guide the position of the torso, and the area of e is used as the judgment standard for whether the torso position is accurate; A, AA, β 1 guide the position of the right upper arm, and the area of aa is used as the judgment standard for whether the left upper arm position is accurate. And so on, following the adjustment sequence of torso - thigh / upper arm - calf / forearm - foot / hand - head, the patient completes the preliminary posture adjustment. After confirming the positions of the torso and the left upper arm, the position of the left forearm is confirmed. AB is the key point detected in the infrared image, and (AB)' is the key point detected in the real-time optical image. Assuming the coordinates of AB are (x, y) and the coordinates of (AB)' are (x', y'), the distance d is calculated. According to the difference between the angle β 2 ' between (AA)'-(AB)' and the vertical direction and the angle β 2 between AA - AB and the vertical direction, the approximate direction and distance of the adjustment are obtained, and d is used as the criterion for whether the adjustment is appropriate. After reaching the threshold, the area relationship between (AB)' and ab is used as the criterion for whether the final adjustment is correct.

[0102] Further, in step S6, the corresponding optical image of the adjusted patient is obtained again, and the information processing module processes it to obtain the human dense key point distribution map of the optical image.

[0103] Further, in step S7, the selection of the target area first extracts the area related to the patient's body from the infrared image, and identifies the key points marked in the infrared image; establish the association between the two by finding the counterparts of the features in the infrared image in the optical image, and use the matching feature points or area information to perform the spatial mapping from the infrared image to the optical image; then, based on the matching feature points or area information, select the target area in the optical image, so as to further locate the specific points of the target area.

[0104] As Figure 7 shown, the information processing module realizes coordinate conversion by combining the target point positioning algorithm and the coordinate mapping algorithm, which specifically includes the following processes:

[0105] Obtain the human dense key point distribution maps of the infrared thermal imaging and the optical image;

[0106] Select the human key points on the infrared image as the target points, and locate the target area according to the target points;

[0107] Select two key points adjacent to the target point within the target area to form a triangle;

[0108] Calculate the position of the target point on the optical image through the properties of similar triangles,

[0109] Convert the two-dimensional optical coordinates of the target point on the optical image into the coordinates of the end device;

[0110] Move the stepper motor a certain number of steps, record the actual moving distance, and use the calibration relationship of the stepper motor to convert the number of motor steps into the actual three-dimensional space distance;

[0111] Convert the coordinates of the end device corresponding to the target point into the actual three-dimensional space coordinates of the end device according to the relationship between the number of motor steps and the actual three-dimensional space distance.

[0112] Preferably, to map the selected area on the infrared imaging map to the real body coordinate system of the patient, the position of the key points and the mapping relationship between the infrared image and the visible light image are required. Select n key points of the human body, and use the infrared thermal image key point detection model to detect n key points of the human body on the front and back of the body respectively; use the optical image key point detection model to detect n key points of the human body on the front or back of the body in real time. The key points in the infrared image and each key point in the optical image are preferably numbered and correspond one by one (k i , p i ), and the position coordinate information of the key points is represented by (x i , yi ) Two-dimensional coordinate representation. When a doctor selects the area to be treated (target point) on the interactive diagnosis and treatment device, the position information of the target point on the thermal imaging image can be obtained.

[0113] As Figure 8 shown, the relative position relationship between the target point in the infrared thermal imaging map and each human body key point is obtained. This relationship is represented by the Euclidean distance α i between the target point O and each key point k i and the direction angle α i . Map this relationship directly onto the real-time optical image. Starting from p i , find the mapped point A i at a distance of d i in the direction of α i . All the points A i constitute the set A. Using the data fitting algorithm, by setting the loss and reducing the loss to the minimum, the two-dimensional coordinates of the final target point OA can be obtained.

[0114] Furthermore, in step S7, in order to obtain the approximate position of the target point, a target point positioning method is designed. First, 17 key points of the human body are selected to represent specific parts of the human body, and multiple target points that need to be treated are determined. Then, use these key points to locate the approximate area of the human body and obtain the rough position of the target point. For each target point, find two adjacent key points within the approximate area to construct a triangle. Then, use the properties of similar triangles to determine the position of the target point in the infrared image and convert it into the specific position on the optical image. Next, convert the optical coordinates into the coordinates of the three-axis mechanical fish hole platform device, move the stepping motor a certain number of steps, and record the actual moving distance to establish the conversion relationship between the number of steps of the stepping motor and the actual distance. Finally, use the calibration relationship of the stepping motor to convert the number of motor steps into the actual three-dimensional space distance, and through the geometric relationship of the mechanical structure, map the coordinates of the target point to the coordinates of the three-axis mechanical motion platform device. The doctor can then obtain the coordinate information of the target point on the three-axis mechanical motion platform device.

[0115] Preferably, for the key point detection of infrared imaging images in this scenario, due to the lack of texture, it is necessary to grasp the distribution characteristics of the human body contour and the human body as a whole. First, infrared image preprocessing is required: use the human body detection algorithm to preliminarily calibrate the position of the human body, optimize the BoundingBox, determine the final position of the human body and perform cutting and filling operations; perform data enhancement methods such as denoising and interpolation on the image, and normalize the image so that the temperature color range of all infrared images falls within a certain range, which is convenient for subsequent key point detection. The detection of key points uses traditional methods or deep learning human key point detection technology to achieve thermal map-based prediction results for a large number of key points of the human body, and then optimizes the key point positions through a multi-stage optimization strategy.

[0116] Preferably, in order to achieve accurate positioning of the target point, the present invention has created an intelligent target point positioning method, which selects 17 key points of the human body, such as Figure 9 As shown, these key points are carefully selected to represent different specific parts of the human body. The present invention makes full use of the position information of these key points, and through a series of innovative steps, realizes the accurate positioning of the target points on the optical image, thereby providing more accurate and reliable positioning support for the infrared thermal imaging diagnosis and treatment system. First of all, the present invention selects 17 key points, which are distributed in different parts of the human body and can fully reflect the anatomical structure of the human body. By selecting these key points, the present invention can not only represent the specific parts of the human body, but also realize the positioning of the overall area. Secondly, the present invention uses the selected key points to locate the approximate area of ​​the human body. The core idea of ​​this step is to first determine the multiple target points that need to be treated, and then judge the area where the target points may exist through the distribution of the key points, thereby narrowing the search range. For each target point, the present invention further introduces the concept of constructing a triangle with adjacent key points. With Figure 10 Taking the target point a in the optical image as an example, the present invention selects the key points 6 and 12 adjacent to it and constructs a triangle. Through the properties of similar triangles, the present invention can accurately calculate the specific position of the target point a on the optical image. This similarity-based triangle positioning method has a high degree of accuracy and can overcome the complexity and diversity of the human body structure. Next, the present invention converts the optical coordinates into the coordinates of the end device. In order to achieve this step, the present invention introduces a stepper motor and controls it to move a certain number of steps. By recording the actual moving distance, the present invention establishes an accurate conversion relationship between the number of steps of the stepper motor and the actual distance. Finally, through the geometric relationship of the mechanical structure, the present invention successfully maps the coordinates of the target point to the coordinates of the end device.

[0117] Further, in step S8, the three-dimensional mechanical motion platform is controlled according to the calculated coordinates to drive the moxibustion sticks to perform moxibustion on the patient, such as Figure 11 As shown, the specific steps include:

[0118] Obtain the actual three-dimensional space coordinates of the end of the device corresponding to the target point;

[0119] Perform orthogonal direction calculation on the actual three-dimensional space coordinates of the end of the device corresponding to the target point to determine the relative position of the target point on the three-dimensional coordinate axes;

[0120] Plan the movement path of the three-dimensional mechanical motion platform;

[0121] Control the moxibustion clamping device of the three-dimensional mechanical motion platform to reach the actual three-dimensional space coordinates of the end of the device corresponding to the target point and perform the moxibustion operation.

[0122] In specific use, first ensure that the patient is in a comfortable position, clean the skin of the moxibustion area, and prepare the moxibustion apparatus and the required traditional Chinese medicine materials. According to the patient's condition and the doctor's advice, determine the area where moxibustion needs to be performed, and adjust the device parameters according to the requirements to achieve the best treatment effect. Then, use infrared thermal imaging technology to scan the patient's body. The system accurately locates the treatment site and the target point position based on the temperature distribution on the patient's body surface and the infrared thermal imaging image. Load the moxibustion stick onto the moxibustion clamping device and ensure the coordinated movement of the treatment device and the three-axis mechanical motion platform. After starting the moxibustion treatment program, the three-axis mechanical motion platform automatically moves according to the preset path and target point position, positions the moxibustion stick at the target point, and starts the moxibustion treatment. During the moxibustion treatment process, the temperature sensing module continuously monitors the temperature change and infrared thermal imaging image of the moxibustion area to ensure the accuracy and safety of the treatment. Medical staff can observe the treatment process through the interactive diagnosis and treatment module, adjust the parameters in a timely manner or stop the treatment. If moxibustion treatment needs to be performed on different parts, the system can automatically adjust the position and the movement of the moxibustion stick according to the preset treatment path and target point position to achieve continuous moxibustion treatment. When the moxibustion treatment is completed, the system stops the work of the moxibustion apparatus and automatically moves the moxibustion stick to a safe position to end the treatment process.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An intelligent robot for moxibustion and thermal therapy based on infrared-visible light fusion positioning, characterized in that: The robot includes: an infrared thermal imaging scanner, a visible light optical camera, a bed frame, a three-axis mechanical motion platform and an information processing module; The bed frame includes a camera bracket arranged beside the bed frame, the three-axis mechanical motion platform is arranged on the bed frame, and the information processing module is arranged on the three-axis mechanical motion platform; The infrared thermal imaging scanner and the visible light optical camera are installed on the camera bracket, and both the infrared thermal imaging scanner and the visible light optical camera are pointed toward the end of the moxibustion stick; The information processing module performs key point detection on the infrared thermal imaging image acquired by the infrared thermal imaging scanner to obtain a sparse key point distribution map of the human body and a dense key point distribution map of the human body; the information processing module performs semantic segmentation and sparse key point detection on the optical image acquired by the visible light optical camera to obtain a human body part segmentation map and a human body sparse key point distribution map; the information processing module compares the sparse key point distribution maps of the infrared thermal imaging image and the optical image, uses the segmentation area as a judgment criterion, and adjusts the patient's posture according to the posture adjustment strategy; the visible light optical camera acquires the optical image again, and the information processing module performs key point detection to obtain a dense key point distribution map of the human body; the information processing module compares the dense key point distribution maps of the human body in the infrared thermal imaging image and the optical image, determines the two-dimensional coordinates of the target area according to the selected key points, maps the two-dimensional coordinates from the infrared thermal imaging image to the optical image, and then converts them into the coordinates of the end effector of the three-axis mechanical motion platform; The information processing module constructs a key point distribution map of the front and back of the patient's body under infrared thermal imaging through a key point detection algorithm, and then optimizes the key point position through a multi-stage optimization strategy; the information processing module divides the human body parts in the optical image through a semantic segmentation algorithm, and the semantic segmentation algorithm divides the human body into at least the following areas: left upper arm (aa), left forearm (ab), left hand (ac), right upper arm (ba), right forearm (bb), right hand (bc), left thigh (ca), left calf (cb), left foot (cc), right thigh (da), right calf (db), right foot (dc), trunk (e), head (f); the area of ​​each part is represented by S i express; The posture adjustment strategy uses the positions and line angles of several key points on the infrared image as benchmarks, calculates the offsets of several key points on the optical image relative to the benchmarks and the angle deviations of the lines connecting the two points, scores the placement of human body parts according to preset standards, uses the score value as the standard for judging posture accuracy, and uses the score value threshold to judge whether the posture meets the standard. The comparison of the areas of various limbs of the human body under the same viewing angle is added, and the positions of key points and the angles of the lines are used as guides for the patient's posture adjustment. After the offset between the measured value and the standard value is less than a certain threshold, the semantic segmentation area overlap is used as the final standard.

2. The intelligent robot for moxibustion and thermal therapy based on infrared-visible light fusion positioning according to claim 1 is characterized in that: The robot also includes a moxibustion clamping device, which is arranged on a three-axis mechanical motion platform; the robot also includes a distance sensing and calibration module, a temperature sensing module, a voice interaction module, and a drive control module which are also arranged on the three-axis mechanical motion platform; The distance sensing and calibration module detects the distance between the end of the moxibustion stick and the plane where the patient is located in real time, calculates the deviation between the end of the three-axis mechanical motion platform and the target area, and then automatically calibrates according to the deviation; The temperature sensing module monitors the temperature change of the human body surface in the moxibustion area in real time; When the voice interaction module detects an abnormal event, it alerts the user or operator by issuing an alarm signal; The drive control module is used to control the actuator in the three-axis mechanical motion platform; The three-axis mechanical motion platform is a three-axis gantry structure, and each axis is controlled by a stepper motor; the three-axis mechanical motion platform drives the end of the moxibustion stick on the moxibustion clamping device to move and position in three directions of the three-axis mechanical motion platform through the stepper motor; the stepper motor is driven by a controller to realize the motion control of the mechanical structure according to the received instructions; The serial communication part adopts serial communication protocol and realizes connection with the controller through CH340G module. The data transmission adopts UART and PWM protocol. The controller receives and parses the data from the server, and then sends the corresponding instructions to the stepper motor.

3. The intelligent robot for moxibustion and thermal therapy based on infrared-visible light fusion positioning according to claim 2 is characterized in that: The robot also includes an automatic ash shaking module and an ash tray arranged at the end of the bedside. The bottom of the ash tray is equipped with a pressure sensor, and then a mica sheet is covered on the pressure sensor. When shaking the ash, the moxa stick is pressed against the mica sheet to make the ash fall off, and the poking force is detected by the pressure sensor. When the pressure sensor senses the change in force, the moxa clamping device stops descending, and the drive control module calculates the length of the moxa stick by calculating the descending height, thereby adjusting the descending height of the device during the moxibustion process.

4. A control method for an intelligent robot for moxibustion and thermal therapy based on infrared-visible light fusion positioning, characterized in that: The control method comprises the following steps: S1: Obtain infrared thermal imaging and optical images of the human body through an infrared thermal imager and a visible light optical camera respectively, and send the infrared thermal imaging and optical images to the information processing module and the interactive diagnosis and treatment module; S2: Perform key point detection on the acquired infrared thermal imaging through the information processing module to obtain an infrared sparse key point distribution map and an infrared dense key point distribution map; perform semantic segmentation and sparse key point detection on the acquired optical image through the information processing module to obtain a human body part segmentation map and an optical sparse key point distribution map; S3: By comparing the key points in the infrared sparse key point distribution map and the optical sparse key point distribution map, the patient posture adjustment strategy is given based on the segmentation area as the judgment criterion; S4: acquiring an optical image of the patient after posture adjustment through a visible light optical camera, and uploading the acquired optical image to an information processing module for human body dense key point detection to obtain an optical dense key point distribution map; S5: Compare the key points of the infrared dense key point distribution map and the optical dense key point distribution map through the information processing module, calculate the positional relationship between the target area and the key points in the artificially selected infrared thermal imaging image to determine the two-dimensional coordinates of the target area; then map the target area in the infrared thermal imaging image to the optical image through coordinate mapping, obtain the two-dimensional coordinates of the target area in the optical camera coordinate system, and convert it into the coordinates of the end effector of the three-axis mechanical motion platform; S6: The drive control module obtains the coordinates of the end effector of the three-axis mechanical motion platform from the information processing module, and controls the moxibustion clamping device to move to the corresponding position; In step S2, the information processing module constructs a key point distribution map of the front and back of the patient's body under infrared thermal imaging through a key point detection algorithm, and then optimizes the key point position through a multi-stage optimization strategy; the information processing module divides the human body parts in the optical image through a semantic segmentation algorithm, and the semantic segmentation algorithm divides the human body into at least the following areas: left upper arm (aa), left forearm (ab), left hand (ac), right upper arm (ba), right forearm (bb), right hand (bc), left thigh (ca), left calf (cb), left foot (cc), right thigh (da), right calf (db), right foot (dc), trunk (e), head (f); the area of ​​each part is represented by S i express; In step S3, based on the positions and line angles of several key points on the infrared image, the offsets of several key points on the optical image relative to the benchmark and the angle deviations of the lines connecting the two points are calculated respectively, and the positions of human body parts are scored according to preset standards. The score values ​​are used as the standard for judging the accuracy of the posture, and the score value threshold is used to judge whether the posture meets the standard. The areas of various limbs of the human body under the same viewing angle are compared, and the positions of key points and the angles of the lines are used as guidance for adjusting the patient's posture. After the offset between the measured value and the standard value is less than a certain threshold, the semantic segmentation area overlap is used as the final standard.

5. The control method of the moxibustion and thermal therapy intelligent robot based on infrared-visible light fusion positioning according to claim 4 is characterized in that: In step S1, after receiving the infrared thermal image and the optical image, the information processing module first preprocesses the infrared thermal image and the optical image; the infrared image preprocessing includes denoising, enhancement, and normalization; the optical image preprocessing includes correction, alignment, and denoising.

6. The control method of the moxibustion and thermal therapy intelligent robot based on infrared-visible light fusion positioning according to claim 4 is characterized in that: In step S5, the information processing module combines the target point positioning algorithm and the coordinate mapping algorithm to realize the coordinate conversion. The process includes: Obtain dense key point distribution map of human body in infrared thermal imaging and optical images; Select the key points of the human body on the infrared image as the target points, and locate the target area according to the target points; Select two key points adjacent to the target point in the target area to form a triangle; The position of the target point on the optical image is calculated by the properties of similar triangles. Converting the two-dimensional optical coordinates of the target point on the optical image into the coordinates of the end device; Move the stepper motor a certain number of steps, record the actual moving distance, and use the calibration relationship of the stepper motor to convert the motor steps into the actual three-dimensional space distance; According to the relationship between the motor steps and the actual three-dimensional space distance, the terminal device coordinates corresponding to the target point are converted into the actual three-dimensional space coordinates of the terminal device.

7. The control method of the moxibustion and thermal therapy intelligent robot based on infrared-visible light fusion positioning according to claim 6 is characterized in that: In step S5, the selected area on the infrared imaging image is mapped to the patient's real body coordinate system, which requires the use of the position of the key points and the mapping relationship between the infrared image and the visible light image; n key points of the human body are selected, and the infrared thermal image key point detection model is used to detect the n key points of the human body on the front and back of the body respectively; The optical image key point detection model is used to detect n key points of the human body on the front or back of the body in real time; the key points in the infrared image are numbered and correspond one to one with each key point in the optical image (k i ,p i ), the position coordinate information of the key point is given by (x i ,y i ) Two-dimensional coordinate representation; manually select the target point to obtain the position information of the target point on the thermal imaging image; Calculate the relative position relationship between the target point in the infrared thermal imaging image and each key point of the human body. The relationship is calculated by the target point O and each key point k i The Euclidean distance α between i With direction angle α i Representation; Map this relationship directly to the real-time optical image, from p i Set out along α i Direction found i Mapping point A at distance i , all A i Construct a set A; use the data fitting algorithm to reduce the loss to the minimum by setting the loss, so as to obtain the two-dimensional coordinates of the final target point OA.

8. The control method of the moxibustion and thermal therapy intelligent robot based on infrared-visible light fusion positioning according to claim 7 is characterized in that: In step S6, the process of controlling the actuator of the three-dimensional mechanical motion platform to reach the actual three-dimensional space coordinates of the device end corresponding to the target point according to the target point includes the following: Obtain the actual three-dimensional space coordinates of the device end corresponding to the target point; Solve the actual three-dimensional space coordinates of the device end corresponding to the target point in the orthogonal direction to determine the relative position of the target point on the three-dimensional coordinate axis; Plan the motion path of a three-dimensional mechanical motion platform; The moxibustion clamping device of the three-dimensional mechanical motion platform is controlled to reach the actual three-dimensional space coordinates of the end of the device corresponding to the target point.

Images