Robot physiotherapy method and device based on three-dimensional infrared imaging

Abstract

The invention relates to a robot physiotherapy method and device based on three-dimensional infrared imaging, and belongs to the technical field of robots, and the method comprises the steps: constructing a human body three-dimensional graph used for reflecting the body surface temperature of a subject by using a human body infrared image of the subject; when the temperature abnormity does not exist in the human body three-dimensional graph, inputting the human body three-dimensional graph into a pre-trained positioning model used for determining acupuncture point positions in the human body three-dimensional graph, and determining acupuncture point position coordinates contained in the human body three-dimensional graph; or, when the temperature abnormity exists in the human body three-dimensional graph, determining the coordinates of the temperature abnormity position; the coordinates serve as terminal coordinates of the robot, so that the robot plans a target physiotherapy path according to the terminal coordinates, the modern intelligent robot is combined with the traditional Chinese medicine theory, acupoint positioning is carried out through a pre-trained positioning model, and compared with a generally applicable acupoint rule, the method has the advantages that the method is simple and convenient; and the problem of inaccurate regular positioning of common acupuncture points caused by different human body shapes or forms can be flexibly avoided.

Description

Technical Field

[0001] This invention relates to a robotic physiotherapy method and device based on three-dimensional infrared imaging, belonging to the field of robotics technology. Background Technology

[0002] All objects with a temperature above absolute zero (-273.15℃) radiate electromagnetic waves, with wavelengths ranging from 0.75μm to 1000μm known as infrared radiation. Infrared imaging refers to the technology of capturing this invisible infrared radiation and converting it into a visible image. Therefore, in the medical and health fields, infrared imaging can be used to detect abnormal temperatures in body parts, allowing doctors to use these abnormal temperatures for disease diagnosis. For example, the elevated temperature in areas of inflammation such as arthritis and breast tumors can help confirm the location of inflammation. It can also be used for vascular localization, such as displaying the distribution of subcutaneous blood vessels, which can assist in procedures like venipuncture. During the pandemic, it was also widely used for body temperature screening.

[0003] Furthermore, in Traditional Chinese Medicine (TCM) meridian and acupoint theory, acupoints can be located based on the human body's appearance and bone structure. When human tissue experiences inflammation or pain, the temperature of that local tissue will be higher than that of surrounding tissues, and the location of inflammation or pain can be determined by measuring the temperature field. TCM theory states, "Where there is pain, there is obstruction," meaning that pain occurs where blood circulation is blocked, and the temperature of the local tissue at that location will be lower than that of surrounding tissues. Based on the above, human epidermal tissue can be combined with infrared imaging to assist in TCM diagnosis and treatment.

[0004] Chinese invention patent application CN102599895A discloses a method, device, and system for obtaining the temperature of human acupoints. The method involves obtaining the coordinate data of each acupoint in the infrared image of the human body according to a preset acupoint correspondence rule. The preset acupoint correspondence rule is based on the "three courts and five eyes" principle in art and the distribution rules of various organs in the human torso. Although it can locate human acupoints, the "three courts and five eyes" principle and the distribution rules of various organs in the human torso are relatively simple and cannot accurately locate acupoints in the complex and diverse human body structure. Summary of the Invention

[0005] The purpose of this invention is to provide a robotic physiotherapy method and device based on three-dimensional infrared imaging, in order to solve the problem of reduced physiotherapy effect due to the inability to accurately locate acupoints.

[0006] To achieve the above objectives, in one aspect, this invention proposes a robotic physiotherapy method based on three-dimensional infrared imaging, comprising: A three-dimensional human body graphic reflecting the subject's body surface temperature was constructed using infrared images of the subject. When there is no temperature anomaly in the 3D human body graphic, the 3D human body graphic is input into a pre-trained positioning model for determining the location of acupoints in the 3D human body graphic to determine the coordinates of the acupoints contained in the 3D human body graphic; or, when there is a temperature anomaly in the 3D human body graphic, the coordinates of the temperature anomaly location are determined. The coordinates are used as the robot's endpoint coordinates so that the robot can plan the target physiotherapy path according to the endpoint coordinates.

[0007] Furthermore, the localization model is trained using the following method: Collect a set of infrared images for training the localization model; The current human tissue location is determined for each infrared image in the infrared image set, and the acupoint locations contained in the current human tissue location are determined based on the pre-established correspondence between acupoint locations and human tissue locations in traditional Chinese medicine acupoint theory. The infrared image set after determining the acupoint location is input into the model to be trained to obtain the localization model.

[0008] Furthermore, it also includes: selecting a target human tissue part that is the same as the current human tissue part from the correspondence; calculating the proportional relationship between the current human tissue part and the target human tissue part; Based on the aforementioned correspondence and proportional relationship, the locations of acupoints contained in the current human tissue are determined.

[0009] On the other hand, the present invention proposes a robotic physiotherapy device based on three-dimensional infrared imaging, comprising an infrared camera for acquiring infrared images of a subject's human body, a robot for performing physiotherapy operations, an overall frame for fixing the infrared camera and the robot, and a controller, wherein the controller is used to perform the following method: A three-dimensional human body graphic reflecting the subject's body surface temperature was constructed using infrared images of the subject. When there is no temperature anomaly in the 3D human body graphic, the 3D human body graphic is input into a pre-trained positioning model for determining the location of acupoints in the 3D human body graphic to determine the coordinates of the acupoints contained in the 3D human body graphic; or, when there is a temperature anomaly in the 3D human body graphic, the coordinates of the temperature anomaly location are determined. The coordinates are used as the robot's endpoint coordinates so that the robot can plan the target physiotherapy path according to the endpoint coordinates.

[0010] Furthermore, the localization model is trained using the following method: Collect a set of infrared images for training the localization model; The current human tissue location is determined for each infrared image in the infrared image set, and the acupoint locations contained in the current human tissue location are determined based on the pre-established correspondence between acupoint locations and human tissue locations in traditional Chinese medicine acupoint theory. The infrared image set after determining the acupoint location is input into the model to be trained to obtain the localization model.

[0011] Furthermore, it also includes: selecting a target human tissue part that is the same as the current human tissue part from the correspondence; calculating the proportional relationship between the current human tissue part and the target human tissue part; Based on the aforementioned correspondence and proportional relationship, the locations of acupoints contained in the current human tissue are determined.

[0012] Furthermore, the infrared camera includes two infrared cameras, which are arranged side by side.

[0013] Furthermore, the robot switches the terminal device carried by its robotic arm according to the current physiotherapy operation being performed.

[0014] The beneficial effects of this invention are as follows: A three-dimensional human body graphic reflecting the subject's body surface temperature is constructed using infrared images of the subject; when no temperature abnormality is found in the three-dimensional human body graphic, the graphic is input into a pre-trained positioning model to determine the acupoint locations within the graphic, thus determining the coordinates of the acupoint locations contained within the graphic; or, when a temperature abnormality is found in the graphic, the coordinates of the abnormal temperature location are determined; these coordinates are used as the robot's endpoint coordinates, enabling the robot to plan the target physiotherapy path according to these endpoint coordinates. This achieves the combination of the three-dimensional human body graphic constructed from infrared images with acupoint positioning and abnormal location positioning, and determines the endpoint coordinates of the robot's movement. This combines modern intelligent robots with traditional Chinese medicine theory, and uses a pre-trained positioning model for acupoint positioning. Compared to universally applicable acupoint rules, this method flexibly avoids the problem of inaccurate positioning due to different body shapes or forms of ordinary acupoint rules. Attached Figure Description

[0015] Figure 1 This invention presents a flowchart of a robotic physiotherapy method based on three-dimensional infrared imaging. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0017] The inventive concept of this invention lies in combining infrared imaging, three-dimensional graphics construction, and traditional Chinese medicine meridians and acupoints to locate acupoints in a three-dimensional graphic of the human body; at the same time, the human body surface temperature information is integrated into the three-dimensional graphic to facilitate the observation of abnormal body temperature of the subject, which can provide a basis for diagnosis and treatment in traditional Chinese medicine.

[0018] Detailed implementation method 1: like Figure 1 The diagram shows a flowchart of a robotic physiotherapy method based on three-dimensional infrared imaging proposed in this invention, which includes steps S11, S12, and S13, specifically: Step S11: Construct a three-dimensional human body graphic reflecting the subject's body surface temperature using the subject's human infrared image; here, the subject's human infrared image refers to the image acquired by an infrared camera. When constructing the three-dimensional human body graphic, each infrared camera in the infrared camera is imaged and converted into a digital signal. The converted digital signal is processed and modeled using an algorithm to construct the subject's three-dimensional human body graphic and orientation.

[0019] Simultaneously, different colors can be used to represent changes in body surface temperature on a 3D human body graphic, thus forming a body surface temperature field. This allows for the intuitive identification of any temperature anomalies from the 3D human body graphic. Abnormal temperatures refer to abnormally high or low temperatures. An abnormally high temperature occurs when the temperature at a certain location exceeds a preset maximum temperature value; an abnormally low temperature occurs when the temperature at a certain location falls below a preset minimum temperature value.

[0020] Step S12: Determine coordinates based on the presence of temperature anomalies: When no temperature anomalies are present in the 3D human body graphic, the 3D human body graphic is input into a pre-trained positioning model used to determine the locations of acupoints in the 3D human body graphic, and the coordinates of the acupoint locations contained in the 3D human body graphic are determined; or, when temperature anomalies are present in the 3D human body graphic, the coordinates of the temperature anomaly locations are determined. Here, whether it is the acupoint location coordinates or the temperature anomaly coordinates, these coordinates can be directly used with the robot's coordinates, that is, the coordinate system of the 3D human body graphic and the robot's coordinate system are the same world coordinate system, which facilitates the robot's localization of acupoint locations or temperature anomaly locations. In addition, the number of acupoint locations and the number of anomaly locations are based on the actual number appearing in the 3D human body graphic, that is, there can be one, two, or more, which is not limited here.

[0021] Step S13: Use the coordinates as the robot's endpoint coordinates so that the robot can plan the target physiotherapy path according to the endpoint coordinates. Here, after the robot reaches the endpoint coordinates according to the target physiotherapy path, it performs different physiotherapy operations according to different physiotherapy plans. The physiotherapy operations include, but are not limited to, moxibustion, massage, hot compress, cold compress, cupping, etc.

[0022] Through the above steps S11-S13, the three-dimensional human body graphic constructed from infrared images is combined with acupoint positioning and abnormal position positioning, and the coordinates of the robot's endpoint are determined. This realizes the combination of modern intelligent robots and traditional Chinese medicine theory, and acupoint positioning is performed through a pre-trained positioning model. Compared with the universally applicable acupoint rules, this method flexibly avoids the problem of inaccurate positioning of ordinary acupoint rules caused by different human body shapes or forms.

[0023] Additionally, it should be noted that in practical applications, acupoint location and temperature abnormality location can coexist in the same three-dimensional human body graphic. When they coexist, the coordinates must be selected and determined by professionals such as physicians.

[0024] Method Detailed Implementation 2: Following the specific embodiments of the present invention described above, the positioning model described in step S12 is trained by the following method: First, several infrared images are collected for training the positioning model to form an infrared image set; here, when collecting infrared images for training, several infrared images are collected from multiple dimensions such as human body parts, number of volunteers, shooting angle, and number of shots to ensure that the infrared image set has multiple data sources and a wide coverage.

[0025] Next, the current human tissue location in each infrared image in the infrared image set is determined, and the acupoint locations contained in the current human tissue location are determined according to the pre-established correspondence between acupoint locations and human tissue locations in traditional Chinese medicine acupoint theory. It should be noted that the pre-established correspondence between acupoint locations and human tissue locations in traditional Chinese medicine acupoint theory refers to the distribution of each acupoint location on the human body in traditional Chinese medicine acupoint theory. For example, there is a correspondence between the Dantian acupoint and the locations of the navel and groin in traditional Chinese medicine theory.

[0026] When determining the location of acupoints in the current human tissue, specifically, a target human tissue that is the same as the current human tissue is selected from the correspondence; the proportional relationship between the current human tissue and the target human tissue is calculated; and the location of acupoints in the current human tissue is determined according to the correspondence and the proportional relationship. In other words, taking into account the different body "sizes" of different subjects, the problem of inaccurate positioning of ordinary acupoints due to different body shapes or forms is avoided by using the proportional relationship between the current human tissue and the target human tissue.

[0027] Finally, the infrared image set after determining the acupoint location is input into the model to be trained to obtain the localization model.

[0028] Method Detailed Implementation 3: In this specific embodiment, the robotic physiotherapy method based on three-dimensional infrared imaging proposed in this invention will be explained and described in conjunction with actual application scenarios.

[0029] In practical applications, a set of binocular infrared cameras acquires infrared images of the subject's body. These cameras and the health diagnostic robot share a single frame to ensure they operate within a common world coordinate system (also known as an absolute coordinate system). Specifically: A binocular infrared camera is used to capture images of the subject's body. Through algorithmic modeling, a 3D image of the subject's body and its specific location are constructed. Based on this 3D image, a positioning model trained with big data analyzes the body's tissues and, according to Traditional Chinese Medicine's meridian and acupoint theory, outputs the locations of acupoints within the 3D image. The coordinates of these acupoints are then output as data to a health diagnostic robot, enabling intelligent acupoint location and facilitating the robot's planning of targeted therapeutic pathways based on these coordinates.

[0030] By leveraging the temperature differences across different parts of the human body, a three-dimensional human body model is constructed, using different colors to represent these temperature variations, thus forming a body surface temperature field. Based on fundamental theories in Traditional Chinese Medicine (TCM), such as "inflamed tissues have higher temperatures than surrounding tissues," "tissues with poor blood circulation have lower temperatures than surrounding tissues," and "free flow prevents pain, pain indicates obstruction," identifying high or low temperature locations (i.e., abnormal temperature locations) within the body surface temperature field can provide a basis for TCM diagnosis and treatment, enabling differentiated treatment based on syndrome differentiation. Furthermore, determining the coordinates of these abnormal temperature locations allows health diagnostic robots to plan targeted treatment pathways. This technological integration of infrared imaging and health diagnostic robots allows for their application in health diagnosis and non-contact medical procedures, effectively assisting TCM treatment.

[0031] Specific implementation method 1 of the device: On the other hand, the present invention also proposes a robotic physiotherapy device based on three-dimensional infrared imaging. The physiotherapy device will be explained and described below in conjunction with actual application scenarios.

[0032] The robotic medical device includes a set of binocular infrared cameras, a health diagnosis and treatment robot, an overall frame, and a control system (i.e., a controller). The binocular infrared cameras are infrared cameras used to acquire infrared images of the human body of the subject; the health diagnosis and treatment robot is used to perform physiotherapy operations; and the overall frame is used to fix the infrared cameras and the robot.

[0033] Here, the health diagnosis and treatment robot and the binocular infrared camera are placed on the same integrated rack so that the infrared imaging equipment and the diagnosis and treatment robot are in the same world coordinate system. This makes it easy for the coordinates of the target position in the image identified by the infrared imaging equipment to be directly used as the coordinates of the end target position of the diagnosis and treatment robot.

[0034] The controller is used to perform the following steps: (1) Infrared imaging and intelligent acupoint location: The subject's body is photographed using a binocular infrared camera. The binocular infrared camera uses two infrared cameras set side by side and maintaining a fixed distance. Each infrared camera images and converts the images into digital signals. The digital signals acquired by the two cameras are processed and modeled using algorithms to construct a three-dimensional graphic and orientation of the subject's body. The current body tissue is determined in the pre-trained positioning model. The acupoint location is determined based on the proportional relationship between acupoints and human tissues in the theory of acupoints in traditional Chinese medicine (e.g., Dantian acupoint determination: The lower abdomen of the human body is photographed using a binocular infrared camera. The current location is identified as the navel and groin through large model data. The location of the Dantian acupoint is determined based on the correspondence between the Dantian acupoint and the navel and groin in the theory of traditional Chinese medicine. The coordinate value of the Dantian acupoint is determined in the world coordinate system).

[0035] The localization model training method involves using a binocular infrared camera to take multiple photos of different people from different angles, identifying characteristic locations (i.e., body tissues, such as the navel and groin), and determining acupoint locations based on these characteristics. A large training model is then used for model training and learning. Through learning with a certain amount of data, intelligent acupoint recognition can be achieved. When calling the large model, the images taken by the binocular infrared camera are input, and the acupoint locations are output.

[0036] (2) Infrared Imaging and Abnormal Location Determination: Due to the temperature differences in various parts of the human body surface, the constructed three-dimensional human body graphic can use different colors to represent the temperature changes of the human body surface, thus constituting a human body surface temperature field. Based on the theory of "free flow prevents pain, pain indicates obstruction" in traditional Chinese medicine, the human epidermal tissue temperature field is displayed on the screen in a color graphic mode. Physicians and other professionals can determine the abnormal temperature location based on the patient's description and the temperature field display content. After the abnormal location is determined, it is transmitted to the health therapy robot in the form of digital information. The diagnostic robot completes the diagnostic and treatment tasks according to the specified program, achieving the purpose of dialectical treatment.

[0037] (3) Health Treatment Robot: The health treatment robot consists of a collaborative robot, a control system, and a treatment terminal actuator (robotic arm). The health treatment robot can be equipped with one or more treatment methods depending on the needs, such as moxibustion, massage, and hot compresses. Each treatment method is performed according to a fixed program. After receiving location data (i.e., the coordinates of the acupoint or the coordinates of the abnormal temperature location), the health treatment robot completes the treatment work according to the specified program. Following the digital information provided by the software or receiving instructions from physicians or other professionals, the health treatment robot can perform different physiotherapy methods such as moxibustion, massage, hot compresses, cold compresses, and cupping, depending on the terminal device it carries.

[0038] In summary, the present invention proposes a robotic physiotherapy method and device based on three-dimensional infrared imaging, which realizes the use of infrared images of human epidermal tissue to assist in traditional Chinese medicine diagnosis and treatment. During the diagnosis and treatment process, each acupoint or abnormal area is located, which facilitates the precise movement of the diagnostic robot to the acupoint or abnormal area to carry out the physiotherapy operation.

Claims

1. A method of robot physiotherapy based on three-dimensional infrared imaging, characterized in that, The method comprises the following steps: constructing a three-dimensional human body graph for reflecting the surface temperature of the subject by using the infrared image of the subject; when there is no temperature anomaly in the three-dimensional human body graph, inputting the three-dimensional human body graph into a pre-trained positioning model for determining the acupoint position in the three-dimensional human body graph to determine the acupoint position coordinates contained in the three-dimensional human body graph; or when there is a temperature anomaly in the three-dimensional human body graph, determining the temperature anomaly position coordinates; using the coordinates as the end point coordinates of the robot to make the robot plan a target physiotherapy path according to the end point coordinates.

2. The robot physical therapy method based on three-dimensional infrared imaging according to claim 1, characterized in that, The positioning model is obtained by the following method: collecting a plurality of infrared images for training the positioning model to form an infrared image set; determining the current human body tissue part of each infrared image in the infrared image set, and determining the acupoint position contained in the current human body tissue part according to the pre-established corresponding relationship between the acupoint position in the theory of traditional Chinese medicine acupoints and the human body tissue part; inputting the infrared image set after determining the acupoint position into a model to be trained to train the model and obtain the positioning model.

3. The robot physiotherapy method based on three-dimensional infrared imaging according to claim 2, characterized in that, Further comprising: selecting a target human body tissue part same as the current human body tissue part from the corresponding relationship; calculating the proportional relationship between the current human body tissue part and the target human body tissue part; determining the acupoint position contained in the current human body tissue part according to the corresponding relationship and the proportional relationship.

4. A robot physiotherapy device based on three-dimensional infrared imaging, characterized by, The method comprises the following steps: constructing a three-dimensional human body graph for reflecting the surface temperature of the subject by using the infrared image of the subject; when there is no temperature anomaly in the three-dimensional human body graph, inputting the three-dimensional human body graph into a pre-trained positioning model for determining the acupoint position in the three-dimensional human body graph to determine the acupoint position coordinates contained in the three-dimensional human body graph; or when there is a temperature anomaly in the three-dimensional human body graph, determining the temperature anomaly position coordinates; using the coordinates as the end point coordinates of the robot to make the robot plan a target physiotherapy path according to the end point coordinates.

5. The robot physiotherapy device based on three-dimensional infrared imaging according to claim 4, characterized in that, The positioning model is obtained by the following method: collecting a plurality of infrared images for training the positioning model to form an infrared image set; determining the current human body tissue part of each infrared image in the infrared image set, and determining the acupoint position contained in the current human body tissue part according to the pre-established corresponding relationship between the acupoint position in the theory of traditional Chinese medicine acupoints and the human body tissue part; inputting the infrared image set after determining the acupoint position into a model to be trained to train the model and obtain the positioning model.

6. The robot physiotherapy device based on three-dimensional infrared imaging according to claim 5, characterized in that, Further comprising: selecting a target human body tissue part same as the current human body tissue part from the corresponding relationship; calculating the proportional relationship between the current human body tissue part and the target human body tissue part; determining the acupoint position contained in the current human body tissue part according to the corresponding relationship and the proportional relationship.

7. The robot physiotherapy device based on three-dimensional infrared imaging according to claim 4, characterized in that, The infrared camera comprises two infrared cameras arranged side by side.

8. The robot physiotherapy device based on three-dimensional infrared imaging according to claim 4, characterized in that, The robot switches the terminal instrument carried by the mechanical hand according to the current physiotherapy operation.

Images