Physiotherapy service robot control system and method

Abstract

The invention provides a physiotherapy service robot control system and method. The physiotherapy service robot control system comprises a machine vision device, a robot, a central control system, an interaction device and a physiotherapy instrument. The interaction device is connected with the machine vision device and sends a starting signal to the machine vision device, so that the machine vision device starts to collect a human body image, selects a physiotherapy area and plans a physiotherapy track; the machine vision equipment is connected with the central control system and is used for sending physiotherapy area and physiotherapy track data to the central control system; the central control system is connected with the robot and the physiotherapy instrument and is used for sending the physiotherapy area and the physiotherapy track data to the robot and the physiotherapy instrument, so that the robot and the physiotherapy instrument execute physiotherapy services respectively, receiving moving track data and state information sent by the robot and the physiotherapy instrument, and sending the moving track data and the state information to the robot and the physiotherapy instrument; and sending to the interaction device. According to the invention, patient data can be obtained in real time in a remote environment, and treatment parameters are adjusted according to the data, so that more flexible and effective treatment services are provided.

Description

Technical Field

[0001] The present invention relates to the technical field of human physiotherapy, and in particular to a physiotherapy service robot control system and method. Background Art

[0002] As medical needs continue to increase, physical therapy service robots are gradually becoming an important tool in rehabilitation treatment. These robots not only need to have high precision and reliability, but also be able to provide personalized services to meet the needs of different patients. However, existing physical therapy service robots still face many challenges in terms of personalized services, safety, and remote monitoring. For example, in terms of personalization, traditional physical therapy robots mostly use preset programs, which are difficult to dynamically adjust according to patients' real-time feedback, resulting in unsatisfactory treatment effects.

[0003] In addition, in terms of safety, existing robotic systems are slow to respond when dealing with abnormal situations and are unable to avoid potential risks in a timely manner. This deficiency may lead to accidental injuries in actual operations, especially in scenarios involving direct contact with the patient's body.

[0004] In terms of remote monitoring, existing systems generally lack real-time monitoring and feedback mechanisms for patients, resulting in the inability to adjust treatment plans in a timely manner during remote physical therapy. Summary of the invention

[0005] In view of this, the present invention proposes a physical therapy service robot control system and method, aiming to solve the above-mentioned technical problems existing in the prior art.

[0006] In one aspect, the present invention proposes a physical therapy service robot control system, comprising: a machine vision device, a robot, a central control system, an interactive device and a physical therapy instrument; wherein,

[0007] The interactive device is connected to the machine vision device to send a start signal to the machine vision device, so that the machine vision device starts to collect human body images and select a treatment area and plan a treatment trajectory;

[0008] The machine vision device is connected to the central control system to send the therapy area and therapy trajectory data to the central control system;

[0009] The central control system is connected to the robot and the physiotherapy apparatus to send the physiotherapy area and physiotherapy trajectory data to the robot and the physiotherapy apparatus so that the robot and the physiotherapy apparatus respectively perform physiotherapy services, and receive the operation trajectory data and status information sent by the robot and the physiotherapy apparatus, and send the operation trajectory data and status information to the interactive device to realize remote monitoring and data analysis.

[0010] Furthermore, in the above-mentioned physiotherapy service robot control system, the robot is a 6-axis collaborative robot.

[0011] Furthermore, the above-mentioned physiotherapy service robot control system also includes: a mobile online device; wherein,

[0012] The mobile online device is connected to the machine vision device for remotely controlling the machine vision device.

[0013] The physiotherapy service robot control system in the present invention starts the machine vision device through the interactive device. The machine vision device collects human body images, selects the physiotherapy area and plans the physiotherapy trajectory, and then transmits it to the robot and the physiotherapy instrument, so that the robot and the physiotherapy instrument perform physiotherapy services according to the received trajectory data. At the same time, the robot and the physiotherapy instrument transmit the operating status and trajectory data back to the interactive device, which is convenient for remotely viewing the user's physiotherapy data to make adjustments. It can identify and calibrate individual differences, provide personalized physiotherapy services, and avoid errors that may be caused by human operation, thereby improving the safety and effectiveness of physiotherapy.

[0014] On the other hand, the present invention also proposes a control method for a physiotherapy service robot, comprising: step 1, using a pre-trained image detection model to perform real-time detection of a target in a human body image collected by a machine vision device, and using a prediction box drawing function to draw a target detection box in the image;

[0015] Step 2: Send the predicted target position information to the robot, and the robot moves to the specified position according to the received target information.

[0016] Furthermore, in the above-mentioned physiotherapy service robot control method, the pre-training process of the image detection model includes:

[0017] Collect images of the left and right legs of a human body, preprocess them, obtain enhanced image data as training data, and label the enhanced image data to obtain a target data set;

[0018] The target data set is input into the input layer of the YOLOv8 network for training, the feature information of the target is obtained through the feature extraction module, and the extracted feature information is input into the feature fusion module for integration and input into the prediction module of the YOLOv8 network to obtain the prediction result. The binary cross entropy and CIoU are used in the back propagation to calculate the loss value of the target. After multiple iterations until the loss value is minimized, the image detection model is finally obtained.

[0019] Furthermore, in the above-mentioned physiotherapy service robot control method, in the step 1, the preprocessing of the left and right leg images of the human body includes: contrast transformation, rotation, flipping, translation, scaling and adding noise to the large target data.

[0020] Furthermore, in the above-mentioned physiotherapy service robot control method, in the step 1, labeling the enhanced image data includes: marking the two acupuncture points of Dubi and Shangjuxu in the pre-processed images of the left and right legs of the human body.

[0021] Furthermore, in the above-mentioned physiotherapy service robot control method, step 1 comprises:

[0022] Mark and calibrate the depth camera in the machine vision device, create a pipeline and a configuration for the depth camera, set the resolution, format, and frame rate of the depth stream and color stream, start the pipeline to start the data flow, and create an alignment object to align the depth image with the color image;

[0023] Call the get_aligned_images function to get the image frame from the pipeline, align the depth and color images, and convert them into NumPy arrays, while getting the internal parameters of the depth and color images, which are used for the subsequent conversion from depth to camera coordinate system;

[0024] Use the trained image detection model to perform target detection on the color image and obtain the detection results. For each detected target, use the depth frame to obtain the depth information of the target center point, and convert the depth information into three-dimensional coordinates in the camera coordinate system. Draw a circle and text box on the detection result image to display the center point and three-dimensional coordinates of the target.

[0025] Furthermore, in the above-mentioned physiotherapy service robot control method, step 2 comprises:

[0026] Hand-eye calibration for depth cameras in robots and machine vision devices;

[0027] The three-dimensional coordinate information of the target center is converted into the robot arm coordinate system through a formula;

[0028] The converted coordinates are sent to the robot's mechanical arm controller to move the robot's mechanical arm to the target position.

[0029] Furthermore, in the above-mentioned physiotherapy service robot control method, the three-dimensional coordinates of the target center are

[0030] The information is converted into the robot's arm coordinate system using the following formula:

[0031] P arm =R·P camera +t

[0032] Among them, P camera is the coordinate of the target in the camera coordinate system, P armare the transformed coordinates, R is the rotation matrix, and t is the translation vector.

[0033] The physiotherapy service robot control method provided by the present invention, combined with the remote monitoring system of the Internet of Things and cloud computing, enables the physiotherapy service robot to obtain patient data in real time in a remote environment and adjust treatment parameters according to the data, thereby providing more flexible and effective treatment services. The physiotherapy service robot can not only significantly improve the accuracy and safety of treatment, but also realize efficient personalized services in a remote environment, which is of great significance to the intelligent development of the medical field. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0035] Figure 1 A schematic diagram of the overall structure of a physiotherapy service robot control system provided by an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of the operation flow of the physiotherapy service robot control system in an embodiment of the present invention;

[0037] Figure 3 is a structural block diagram of a control system of a physiotherapy service robot in an embodiment of the present invention;

[0038] Figure 4 Schematic diagram of the communication mode of each component in the physiotherapy service robot in an embodiment of the present invention;

[0039] Figure 5 This is a flow chart of a control method for a physiotherapy service robot provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] See also Figure 1 and Figure 3The physiotherapy service robot control system of the embodiment of the present invention includes: a machine vision device 100, a robot 200, a central control system 300, an interactive device 400 and a physiotherapy instrument 500; wherein the interactive device is connected to the machine vision device to send a start signal to the machine vision device, so that the machine vision device starts to collect human body images and select a physiotherapy area and plan a physiotherapy trajectory; the machine vision device is connected to the central control system to send physiotherapy area and physiotherapy trajectory data to the central control system; the central control system is connected to the robot and the physiotherapy instrument to send the physiotherapy area and physiotherapy trajectory data to the robot and the physiotherapy instrument, so that the robot and the physiotherapy instrument respectively perform physiotherapy services, and receive the operation trajectory data and status information sent by the robot and the physiotherapy instrument, and send the operation trajectory data and status information to the interactive device to realize remote monitoring and data analysis.

[0042] Specifically, the interactive device 400 may be a human-machine interface. The machine vision device 100 may be a camera device in the prior art. The robot is preferably a 6-axis collaborative robot, which has high adaptability and flexibility and can adapt to different physical therapy needs and scenarios. The physical therapy device may be set at the center of the robot's tip and kept perpendicular to the end joint to improve the portability of the collaboration between the two.

[0043] The machine vision device uses dual-vision photography technology to collect human body images. It has a built-in depth camera to identify and calibrate the relevant dimensions of each part of the body of the service object, calculate the corresponding execution trajectory, and transmit the physiotherapy path to the physiotherapy service robot. The user inputs relevant information and instructions through the human-machine interface, such as selecting the type of physiotherapy and setting physiotherapy parameters. The machine vision device identifies the area to be treated and collects the corresponding human image data; the central control system comprehensively considers the image data provided by the machine vision device and the information input by the human-machine interface, formulates a physiotherapy plan and path, and issues tasks to the robot and physiotherapy instrument. The robot performs operations such as irradiating light waves and adjusting the height and inclination of the physiotherapy bed; the physiotherapy instrument can perform non-contact physiotherapy tasks, such as light therapy, electrotherapy, and ultrasonic therapy. Through such division of labor and cooperation, the robot and the physiotherapy instrument jointly complete a series of complex physiotherapy tasks from data analysis to actual operation, thereby providing efficient and personalized physiotherapy services.

[0044] At the same time, the human-machine interface is also responsible for starting the physiotherapy device, receiving the robot's stop signal, and collecting the robot's running trajectory. The central control system includes the industrial Internet system to achieve remote monitoring, data collection and equipment linkage control, improve the system's intelligence and flexibility, and improve the convenience and efficiency of services.

[0045] Continue reading Figure 3, this embodiment may also include: a mobile online device 600; wherein the mobile online device 600 is connected to the machine vision device 100 to remotely control the machine vision device.

[0046] Specifically, the mobile online device can be a mobile terminal such as a mobile phone or a tablet, which can receive the running trajectory data and status information sent by the robot and the physiotherapy device in real time to achieve real-time remote monitoring and data analysis. At the same time, the mobile online device can remotely control the physiotherapy device to perform specific physiotherapy tasks.

[0047] See also Figure 4 The physiotherapy service robot control system of the present invention integrates machine vision equipment, robot system, intelligent physiotherapy instrument, and industrial Internet system. On the industrial-grade computer platform, 6-axis collaborative robot programming software, machine vision in-depth analysis and processing software, industrial interconnection configuration and intelligent physiotherapy management software are integrated and installed. The system aggregates key sensors, machine vision, terminal physiotherapy instruments, physiotherapy beds and control data, and accesses them in real time through remote mobile phones. The configuration software communicates with the black box data acquisition module in real time through the Ethernet. The black box communicates with the programmable human-machine interface of the lower-end machine equipment through the Ethernet, and the programmable human-machine interface communicates with the programmable controller (PLC) through the serial port to realize the linkage control of the system's mechanisms. Through the bus control of the computer and intelligent collaborative robots, machine vision, physiotherapy instruments, and physiotherapy beds, the intelligent physiotherapy process is realized.

[0048] Among them, Ethernet is used for data communication between the industrial-grade computer platform and the six-axis collaborative robot programming software, machine vision in-depth analysis and processing software, industrial Internet configuration and smart physiotherapy management software; data aggregation between the central control system and machine vision and terminal physiotherapy equipment; the six-axis robot and smart physiotherapy equipment receive instructions from the central control system; and real-time access between the cloud platform and the system;

[0049] Among them, the Modbus TCP / IP protocol is used for communication between the central control system, industrial-grade computers and the six-axis collaborative robot;

[0050] Among them, the communication between the black box and the programmable human-machine interface of the machine equipment and the communication between the programmable human-machine interface and the programmable controller (PLC) adopt RS-232 and RS-485 respectively;

[0051] Among them, the MQTT protocol is used for remote access / communication between the cloud platform and the MQTT Broker;

[0052] Among them, the central control system aggregates the data of machine vision and terminal physiotherapy instruments, and issues instructions to the six-axis collaborative robot and the intelligent physiotherapy instrument; the six-axis collaborative robot receives the instructions of the central control system and performs the corresponding actions; the intelligent physiotherapy instrument receives the instructions of the central control system and performs the corresponding physiotherapy tasks;

[0053] Among them, the programmable controller (PLC) receives control signals from the programmable human-machine interface to control operations related to the machine equipment; the black box communicates with the programmable human-machine interface of the machine equipment through Ethernet for data collection and monitoring; the programmable human-machine interface (HMI) of the machine equipment communicates with the PLC through the serial port to display and control the status of the machine equipment;

[0054] Among them, the system allows external devices (such as mobile phones) to access and control in real time through the cloud platform, and interact with the system through the MQTT protocol, OPC UA Server and P2P Server to achieve remote control and data access.

[0055] Please combine Figure 2 and Figure 3 As shown, the working process of the physiotherapy service robot control system of this embodiment is: the service object enters the waiting state on the adjustable physiotherapy bed -> the physiotherapy service robot control system is initialized -> the user sets the main parts of the service object and the physiotherapy task content on the interactive device -> the machine vision device performs in-depth human body structure analysis on the service object -> the central control system publishes the physiotherapy path to the robot and publishes the physiotherapy task to the smart physiotherapy instrument -> the robot starts to perform flexible tasks -> the smart physiotherapy instrument executes physiotherapy instructions -> the system collects and summarizes the information of the smart physiotherapy service robot in real time until the task is completed.

[0056] The detailed workflow of the embodiment of the present invention is as follows: First, the service object enters the physiotherapy bed, uses the human-machine interface to open the machine vision device through Ethernet, and the camera collects human body images and transmits them to the central control system through Ethernet; then, the central control system sends the planned physiotherapy trajectory to the six-axis robot through Modbus TCP / IP, and then turns on the physiotherapy instrument. At the same time, mobile online devices can access and control in real time through the cloud platform, interact with the system through the MQTT protocol, OPC UAServer and P2P Server, and realize remote control and data access.

[0057] It can be obviously concluded from the above that the physiotherapy service robot control system provided in this embodiment turns on the machine vision device through the interactive device. The machine vision device collects human body images, selects the physiotherapy area and plans the physiotherapy trajectory, and then transmits it to the robot and the physiotherapy instrument, so that the robot and the physiotherapy instrument perform physiotherapy services according to the received trajectory data. At the same time, the robot and the physiotherapy instrument transmit the operating status and trajectory data back to the interactive device, which is convenient for remote viewing of the user's physiotherapy data to make adjustments. It can identify and calibrate individual differences, provide personalized physiotherapy services, and avoid errors that may be caused by human operation, thereby improving the safety and effectiveness of physiotherapy.

[0058] See also Figure 5 The present invention also provides a control method for a physiotherapy service robot, comprising the following steps:

[0059] Step S1, using a pre-trained image detection model to perform real-time detection on targets in human images collected by a machine vision device, and using a prediction box drawing function to draw a target detection box in the image.

[0060] Specifically, the target is the area of ​​the human leg to be treated.

[0061] Furthermore, the pre-training process of the image detection model includes:

[0062] (1) Collect images of the left and right legs of a human body, and preprocess them to obtain enhanced image data as training data, and label the enhanced image data to obtain a target data set.

[0063] (2) The target data set is input into the input layer of the YOLOv8 network for training, the feature information of the target is obtained through the feature extraction module, and the extracted feature information is input into the feature fusion module for integration and input into the prediction module of the YOLOv8 network to obtain the prediction result. The binary cross entropy and CIoU are used in the back propagation to calculate the loss value of the target. After multiple iterations until the loss value is minimized, the image detection model is finally obtained.

[0064] Specifically, binary cross entropy and CIoU are used in back propagation to calculate the loss value of the large target. After multiple iterations until the loss value is minimized, the target training model Best.pt is finally obtained.

[0065] In this step, the preprocessing of the left and right leg images of the human body includes: contrast transformation, rotation, flipping, translation, scaling and adding noise to the large target data.

[0066] Labeling the enhanced image data includes labeling the two acupoints Dubi and Shangjxu in the preprocessed images of the left and right legs of the human body. More specifically, LabelImg can be used to label the two acupoints as "Dubi" and "Shangjxuu" respectively. By labeling different acupoints, the diversity and quality of the data are ensured, providing high-quality training data for subsequent model training.

[0067] In this embodiment, the YOLOv8s.pt model is used as the training model for the target detection stage. During the model training process, a multi-scale training strategy is adopted to enable the model to better adapt to objects of different scales; in order to further improve the generalization ability of the model, data enhancement technology is also adopted, including random scaling, random cropping and color jittering. At the same time, category balancing technology is also adopted to enable the model to better learn objects of minority categories.

[0068] During the testing process, the standard mAP (mean Average Precision) was used as an evaluation indicator to measure the detection ability of the model. At the same time, in order to further improve the performance of the model, a multi-scale testing strategy was also adopted. According to the evaluation results, the model was adjusted and optimized, the network structure was adjusted, the training strategy was improved, and the hyperparameters were optimized. Closed-loop optimization was performed to further improve the detection ability and robustness of the model.

[0069] Further, step S1 includes:

[0070] Sub-step S11, marks and calibrates the depth camera in the machine vision device, creates a pipeline and a configuration for the depth camera, sets the resolution, format and frame rate of the depth stream and color stream, starts the pipeline to start the data flow, and creates an alignment object for aligning the depth image with the color image.

[0071] Sub-step S12, calling the get_aligned_images function to obtain image frames from the pipeline, align the depth and color images, and convert them into NumPy arrays, while obtaining the internal parameters of the depth and color images, which are used for the subsequent conversion of the depth to the camera coordinate system.

[0072] Sub-step S13, use the trained image detection model to perform target detection on the color image to obtain the detection results. For each detected target, use the depth frame to obtain the depth information of the target center point, and convert the depth information into three-dimensional coordinates in the camera coordinate system. Draw a circle and a text box on the detection result image to display the center point and three-dimensional coordinates of the target.

[0073] Step S2, sending the predicted target position information to the robot, and the robot moves to the designated position according to the received target information.

[0074] Further, step S2 includes:

[0075] Sub-step S21, performing hand-eye calibration on the robot and the depth camera in the machine vision device.

[0076] Specifically, in this embodiment, the camera may be an INTEL RealSense D435i depth camera. The hand-eye calibration method is eye-on-hand (the camera is placed at the end of the robot, so the eye-on-hand calibration method is adopted). The specific calibration method is as follows:

[0077] First, the euler_angles_to_rotation_matrix function is used to accept three Euler angles (rx, ry, rz) as input, representing the rotation around the x-axis, y-axis, and z-axis respectively. These angles are used to construct a rotation matrix that describes the rotation from the world coordinate system to the camera coordinate system. Secondly, the ρose_tohomogeneous_matrix function is used to accept a pose vector, which contains the position (x, y, z) and the rotation (rx, ry, rz). It first calls the euler_angles_to_rotation_matrix function to get the rotation matrix, and then combines the rotation matrix and the translation vector into a 4x4 homogeneous transformation matrix. Then the save_matrices_to_csv function is used to accept a list of matrices and merge them into a large matrix, which is then saved to a CSV file. The columns of each matrix are written to the file consecutively. Then the poses_save_csv function is used to read the pose data from the CSV file, convert it to a homogeneous transformation matrix, and save these matrices to another CSV file. Then use cv2.findChessboardCorners and cv2.cornerSubPix functions to detect corners in the calibration board image. Use cv2.calibrateCamera function to calculate the camera's intrinsic parameters and distortion coefficients based on the detected corners and the physical size of the calibration board. Read the pose data of the robot end effector from the CSV file and convert it into a homogeneous transformation matrix. Finally, use cv2.calibrateHandEye function to calculate the pose relationship from the camera to the robot end effector. The result of the calibration result is the rotation matrix R and translation vector t of the camera coordinate system relative to the robot end coordinate system.

[0078] Sub-step S22, converting the three-dimensional coordinate information of the target center into the robot's mechanical arm coordinate system through a formula.

[0079] Specifically, the three-dimensional coordinate information of the target center is converted into the robot's mechanical arm coordinate system by the following formula:

[0080] P arm =R·P camera +t

[0081] Among them, P camera is the coordinate of the target in the camera coordinate system, P arm are the transformed coordinates, R is the rotation matrix, and t is the translation vector.

[0082] Sub-step S23, sending the converted coordinates to the robot's mechanical arm controller to move the robot's mechanical arm to the target position.

[0083] During the specific implementation, the robot arm's API (application programming interface) and SDK (software development kit) are used to send movement instructions. After the robot arm moves to the target position and waits for 3 seconds, the smart physiotherapy device is turned on.

[0084] The related parts between the method embodiment and the above-mentioned system embodiment can be referred to each other and will not be described in detail here.

[0085] In summary, the present invention combines the remote monitoring system of the Internet of Things and cloud computing, so that the physiotherapy service robot can obtain patient data in real time in a remote environment and adjust treatment parameters according to the data, thereby providing more flexible and effective treatment services. The physiotherapy service robot can not only significantly improve the accuracy and safety of treatment, but also realize efficient personalized services in a remote environment, which is of great significance to the intelligent development of the medical field.

[0086] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A physical therapy service robot control system, characterized in that: include: Machine vision equipment, robots, central control systems, interactive equipment and physiotherapy equipment; among them, The interactive device is connected to the machine vision device to send a start signal to the machine vision device, so that the machine vision device starts to collect human body images and select a treatment area and plan a treatment trajectory; The machine vision device is connected to the central control system to send the therapy area and therapy trajectory data to the central control system; The central control system is connected to the robot and the physiotherapy apparatus to send the physiotherapy area and physiotherapy trajectory data to the robot and the physiotherapy apparatus so that the robot and the physiotherapy apparatus respectively perform physiotherapy services, and receive the operation trajectory data and status information sent by the robot and the physiotherapy apparatus, and send the operation trajectory data and status information to the interactive device to realize remote monitoring and data analysis.

2. The physiotherapy service robot control system according to claim 1, characterized in that: The robot is a 6-axis collaborative robot.

3. The physiotherapy service robot control system according to claim 1, characterized in that: Also includes: Mobile online devices; among them, The mobile online device is connected to the machine vision device for remotely controlling the machine vision device.

4. A control method for a physiotherapy service robot, characterized in that: The following steps are involved: Step 1: Use the pre-trained image detection model to perform real-time detection on the target in the human image collected by the machine vision device, and use the prediction box drawing function to draw the target detection box in the image; Step 2: Send the predicted target position information to the robot, and the robot moves to the specified position according to the received target information.

5. The control method of the physiotherapy service robot according to claim 4, characterized in that: The pre-training process of the image detection model includes: Collect images of the left and right legs of a human body, preprocess them, obtain enhanced image data as training data, and label the enhanced image data to obtain a target data set; The target data set is input into the input layer of the YOLOv8 network for training, the feature information of the target is obtained through the feature extraction module, and the extracted feature information is input into the feature fusion module for integration and input into the prediction module of the YOLOv8 network to obtain the prediction result. The binary cross entropy and CIoU are used in the back propagation to calculate the loss value of the target. After multiple iterations until the loss value is minimized, the image detection model is finally obtained.

6. The control method of the physiotherapy service robot according to claim 4, characterized in that: In the step 1, the preprocessing of the left and right leg images of the human body includes: performing contrast transformation, rotation, flipping, translation, scaling and adding noise to the large target data.

7. The control method of the physiotherapy service robot according to claim 4, characterized in that: In the step 1, labeling the enhanced image data includes: marking the two acupuncture points of Dubi and Shangjuxu in the pre-processed images of the left and right legs of the human body.

8. The control method of the physiotherapy service robot according to claim 4, characterized in that: The step 1 comprises: Mark and calibrate the depth camera in the machine vision device, create a pipeline and a configuration for the depth camera, set the resolution, format, and frame rate of the depth stream and color stream, start the pipeline to start the data flow, and create an alignment object to align the depth image with the color image; Call the get_aligned_images function to get the image frame from the pipeline, align the depth and color images, and convert them into NumPy arrays, while getting the internal parameters of the depth and color images, which are used for the subsequent conversion from depth to camera coordinate system; Use the trained image detection model to perform target detection on the color image and obtain the detection results. For each detected target, use the depth frame to obtain the depth information of the target center point, and convert the depth information into three-dimensional coordinates in the camera coordinate system. Draw a circle and text box on the detection result image to display the center point and three-dimensional coordinates of the target.

9. The control method of the physiotherapy service robot according to claim 8, characterized in that: The step 2 comprises: Hand-eye calibration for depth cameras in robots and machine vision devices; The three-dimensional coordinate information of the target center is converted into the robot arm coordinate system through a formula; The converted coordinates are sent to the robot's mechanical arm controller to move the robot's mechanical arm to the target position.

10. The control method of the physiotherapy service robot according to claim 9, characterized in that: The three-dimensional coordinate information of the target center is converted into the robot's mechanical arm coordinate system by the following formula: P arm =R·P camera +t Among them, P camera is the coordinate of the target in the camera coordinate system, P arm are the transformed coordinates, R is the rotation matrix, and t is the translation vector.

Images