Hair follicle transplantation robot control method and device, computer equipment and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZINGBOT (SHENZHEN) CO LTD
- Filing Date
- 2022-12-19
- Publication Date
- 2026-07-03
Smart Images
Figure CN115813565B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of planning and guidance technology for hair follicle transplantation robots, and in particular to a control method, device, computer equipment, and storage medium for a hair follicle transplantation robot. Background Technology
[0002] Currently, there are two main types of hair follicle harvesting for transplantation: manual harvesting and robotic harvesting. In manual harvesting, the doctor adjusts the surgical instruments based on naked-eye observation, subjectively determining a suitable harvesting angle. This requires a high level of skill and proficiency from the doctor, and maintaining this state for extended periods (usually 6-8 hours) poses a significant challenge to their concentration and physical stamina. Current robotic hair follicle transplantation methods are still far from fully intelligent. When the operator gets close to the patient's head, the robotic arm will abruptly stop due to safety concerns, making it difficult for the doctor to intervene and make corrections or perform manual operations during the robotic hair follicle transplantation procedure. Summary of the Invention
[0003] This application provides a hair follicle transplantation robot control method, device, computer equipment, and storage medium that can introduce human correction or manual operation during the operation of the hair follicle transplantation robot.
[0004] In a first aspect, this application provides a method for controlling a hair follicle transplantation robot. The method includes:
[0005] During the process of the end effector of the hair follicle transplantation robot performing the operation according to the preset planned path, the first motion trajectory parameters of the control target of the operation object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined based on the positioning auxiliary data used to locate the operation object.
[0006] The target control mode of the end effector is determined based on the first motion trajectory parameters and the second motion trajectory parameters.
[0007] Based on the target control mode, the end effector is guided to perform the operation on the target object.
[0008] In one embodiment, determining the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system based on positioning assistance data used for locating the object being manipulated includes:
[0009] Based on the positioning auxiliary data used to locate the manipulated object, determine the real-time three-dimensional position of the manipulated target in the visual coordinate system, as well as the first motion trajectory parameters of the manipulated target in the visual coordinate system.
[0010] Obtain the first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot, and the second transformation relationship between the coordinate system corresponding to the end effector and the base coordinate system of the hair follicle transplantation robot;
[0011] Based on the first and second transformation relationships, the real-time three-dimensional position of the end effector in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined.
[0012] In one embodiment, determining the target control mode of the end effector based on the first motion trajectory parameters and the second motion trajectory parameters includes:
[0013] The first motion trajectory parameters and the second motion trajectory parameters are input into the pre-trained prediction model, and the coordination rate between the operation object and the hair follicle transplantation robot is output.
[0014] Based on the coordination rate range corresponding to each control mode and the target coordination rate range in which the coordination rate is located, the control mode corresponding to the target coordination rate range is taken as the target control mode of the end effector.
[0015] In one embodiment, according to the target control mode, the end effector is guided to perform a task on the target object, including:
[0016] The virtual fixture corresponding to the target control mode is determined. In the scenario of collaborative operation between the operation object and the hair follicle transplantation robot, the virtual fixture provides motion boundaries for defining the motion range of the end effector or control commands for correcting the second motion trajectory.
[0017] Determine the real-time relative positions of the virtual fixture and the end effector when both are in the visual coordinate system;
[0018] Based on the real-time relative position, the end effector is guided to perform operations on the target object within the motion range defined by the virtual fixture.
[0019] In one embodiment, determining the virtual fixture corresponding to the target control mode includes:
[0020] If the target control mode is curve-guided mode, then the virtual fixture corresponding to the target control mode is a virtual curve, which is used to guide the end effector to move along the virtual curve.
[0021] In one embodiment, determining the virtual fixture corresponding to the target control mode includes:
[0022] If the target control mode is the pipeline guidance mode, then the virtual fixture corresponding to the target control mode is a virtual pipeline, which is used to define the motion space of the end effector within the virtual pipeline.
[0023] In one embodiment, determining the virtual fixture corresponding to the target control mode includes:
[0024] If the target control mode is the cone-guided mode, then the virtual fixture corresponding to the target control mode is a virtual cone. The center of the cone surface of the virtual cone is the starting point of the end effector's movement, the vertex of the virtual cone is the ending point of the end effector's movement, and the virtual space defined within the virtual cone is the movement space of the end effector.
[0025] In one embodiment, determining the virtual fixture corresponding to the target control mode includes:
[0026] If the target control mode is the potential field boundary guidance mode, then the virtual fixture corresponding to the target control mode is a virtual region. The virtual region is used to establish a mapping relationship between the distance between the end effector and the virtual region and the resistance of the end effector.
[0027] Secondly, this application also provides a control device for a hair follicle transplantation robot. The device includes:
[0028] The positioning module is used to determine the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system, based on the positioning auxiliary data used to locate the target object, during the operation of the end effector of the hair follicle transplantation robot according to the preset planned path.
[0029] The prediction module is used to determine the target control mode of the end effector based on the first motion trajectory parameters and the second motion trajectory parameters.
[0030] The guidance module is used to guide the end effector to perform operations on the target object according to the target control mode.
[0031] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:
[0032] During the process of the end effector of the hair follicle transplantation robot performing the operation according to the preset planned path, the first motion trajectory parameters of the control target of the operation object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined based on the positioning auxiliary data used to locate the operation object.
[0033] The target control mode of the end effector is determined based on the first motion trajectory parameters and the second motion trajectory parameters.
[0034] Based on the target control mode, the end effector is guided to perform the operation on the target object.
[0035] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:
[0036] During the process of the end effector of the hair follicle transplantation robot performing the operation according to the preset planned path, the first motion trajectory parameters of the control target of the operation object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined based on the positioning auxiliary data used to locate the operation object.
[0037] The target control mode of the end effector is determined based on the first motion trajectory parameters and the second motion trajectory parameters.
[0038] Based on the target control mode, the end effector is guided to perform the operation on the target object.
[0039] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:
[0040] During the process of the end effector of the hair follicle transplantation robot performing the operation according to the preset planned path, the first motion trajectory parameters of the control target of the operation object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined based on the positioning auxiliary data used to locate the operation object.
[0041] The target control mode of the end effector is determined based on the first motion trajectory parameters and the second motion trajectory parameters.
[0042] Based on the target control mode, the end effector is guided to perform the operation on the target object.
[0043] The aforementioned hair follicle transplant robot control method, device, computer equipment, and storage medium, during the process of the hair follicle transplant robot's end effector performing operations according to a preset planned path, determine the target control mode of the end effector by analyzing the first motion trajectory parameters of the target object and the second motion trajectory parameters of the end effector. That is, during the end effector's operation, the control of the target object is introduced to achieve the control purpose of the target object on the hair follicle transplant robot during the end effector's operation. Moreover, according to different control modes, the target object can actively guide and constrain the hair follicle transplant robot to achieve intelligent and automated human-machine collaboration, solving the drawback of existing hair follicle transplant robots that cannot introduce human correction or manual operation during operation. Attached Figure Description
[0044] Figure 1This is an application environment diagram of a hair follicle transplantation robot control method in one embodiment;
[0045] Figure 2 This is a flowchart illustrating the control method for a hair follicle transplantation robot in one embodiment;
[0046] Figure 3 This is the module framework corresponding to the hair follicle transplantation robot control method in another embodiment;
[0047] Figure 4 This is a flowchart of the processing of the sensing device in one embodiment;
[0048] Figure 5 This is a flowchart of the processing of the sensing device in another embodiment;
[0049] Figure 6 This is a processing flowchart of the sensing device in another embodiment;
[0050] Figure 7 This is a processing flowchart of the sensing device in another embodiment;
[0051] Figure 8 This is a flowchart of the processing of the sensing device in another embodiment;
[0052] Figure 9 This is a flowchart illustrating the process of determining the target control mode of the end effector in one embodiment;
[0053] Figure 10 This is a flowchart of the prediction model processing in one embodiment;
[0054] Figure 11 Here is a flowchart of the prediction model processing in another embodiment;
[0055] Figure 12 This is a schematic diagram illustrating the process of guiding the end effector to perform a job on the target object in one embodiment;
[0056] Figure 13 This is a schematic diagram of a virtual fixture being a virtual curve in one embodiment;
[0057] Figure 14 This is a schematic diagram of a virtual clamp being a virtual pipe in one embodiment;
[0058] Figure 15 This is a schematic diagram of a virtual clamp being a virtual cone in one embodiment;
[0059] Figure 16 This is a schematic diagram of a virtual fixture being a virtual area in one embodiment;
[0060] Figure 17 This is a flowchart illustrating the process by which the end effector achieves different effects in one embodiment;
[0061] Figure 18 This is a schematic diagram of the process of a visual servo controller controlling the end effector in one embodiment;
[0062] Figure 19 This is a schematic diagram illustrating the process of a visual servo controller controlling the end effector in another embodiment;
[0063] Figure 20 This is an operation flowchart of a hair follicle transplantation robot control method in one embodiment;
[0064] Figure 21 This is an overall flowchart of the hair follicle transplantation robot control method in one embodiment;
[0065] Figure 22 This is a structural block diagram of the hair follicle transplantation robot control device in one embodiment. Detailed Implementation
[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0067] The hair follicle transplantation robot control method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, during the operation of the hair follicle transplant robot's end effector following a preset planned path, positioning auxiliary data for locating the target object is acquired by a sensing device on a sensing trolley. Based on this positioning auxiliary data, the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the same visual coordinate system are determined. The sensing trolley then determines the target control mode of the end effector based on the first and second motion trajectory parameters and sends this target control mode to the execution trolley. The execution trolley then controls the hair follicle transplant robot to guide the end effector to perform the operation on the target object according to the target control mode. The interactive interface displays the positional relationship between the first and second motion trajectory parameters and other prompts in real time. Figure 3 In the application scenario shown, during the operation of the hair follicle transplant robot's end effector following the preset planned path, the user can intervene at any time to correct the control mode of the end effector or perform manual operation without causing the robotic arm to stop abruptly. Furthermore, depending on the different control modes, the user can actively guide and constrain the hair follicle transplant robot, achieving intelligent and automated human-machine collaboration.
[0068] In one embodiment, such as Figure 2As shown, a control method for a hair follicle transplantation robot is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:
[0069] Step 202: During the process of the end effector of the hair follicle transplantation robot performing the operation according to the preset planned path, the first motion trajectory parameters of the control target of the operation object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined based on the positioning auxiliary data used to locate the operation object.
[0070] In this context, the end effector of the hair follicle transplant robot refers to the load loaded on the output flange of the robotic arm's end joint module. This load includes structures such as the end-effector camera, hair transplant needles, and hair retrieval needles. The pre-planned path refers to a hair transplant plan pre-planned based on parameters such as the density of the extraction area, the location of the implantation holes, and the hair type of the target individual.
[0071] The manipulated object refers to the movable object used to correct the hair follicle transplant robot during the process of the end effector performing operations according to a preset planned path. The movable object can be a person or another robot. The control target refers to the device used to control the current execution state of the hair follicle transplant robot, which can be a handheld target, the gesture of the manipulated object, or a remote control device. The position of the manipulated object and the real-time position information of the control target of the manipulated object are obtained through sensing devices. Among them, the positioning assistance data is obtained through sensing devices, including but not limited to binocular cameras, infrared sensors, magnetic sensors, temperature sensors, etc. The positioning assistance data is the position data in the coordinate system corresponding to the sensing device. It needs to be converted to the visual coordinate system according to the transformation relationship between the coordinate system corresponding to the sensing device and the visual coordinate system. Based on the positioning assistance data in the visual coordinate system, the first motion trajectory parameters of the control target of the manipulated object in the visual coordinate system are determined.
[0072] Optionally, during the process of the end effector of the hair follicle transplantation robot performing operations according to a preset planned path, the computer equipment acquires positioning auxiliary data for locating the operation object through a sensing device. Based on the transformation relationship between the coordinate system corresponding to the sensing device and the visual coordinate system, the computer equipment converts the positioning auxiliary data to the visual coordinate system. Based on the positioning auxiliary data in the visual coordinate system, it determines the first motion trajectory parameters of the operation object's control target in the visual coordinate system. Based on the rotational position information of the torque of the end joint module of the robotic arm in the Cartesian coordinate system, the computer equipment determines the position information of the end effector in the base coordinate system of the hair follicle transplantation robot. Based on the transformation relationship between the base coordinate system and the visual coordinate system of the hair follicle transplantation robot, it converts the position information of the end effector in the base coordinate system to the visual coordinate system, obtaining the second motion trajectory parameters of the end effector in the visual coordinate system.
[0073] Step 204: Determine the target control mode of the end effector based on the first motion trajectory parameters and the second motion trajectory parameters.
[0074] The target control mode can be the same as or different from the control mode corresponding to the current planned path. In this embodiment, the coordination rate between the operating object and the hair follicle transplant robot, mapped by the first and second motion trajectory parameters, determines whether to switch the current control mode. For example, if the operating object enters an automatically avoidable risk area, or if intervention or manual operation is desired, the operating object can change the movement trajectory of the target. The sensing device acquires positioning assistance data and determines the first motion trajectory parameters based on this data. If the coordination rate mapped by the first and second motion trajectory parameters differs from the current control mode, it indicates that the operating object needs to intervene to correct the hair follicle transplant robot or perform manual operation. In this case, the computer device determines the target control mode of the end effector based on the control mode corresponding to the coordination rate and guides the hair follicle transplant robot to operate according to the target control mode. During this process, the hair follicle transplant robot will not stop abruptly due to the approach of the operating object, thus solving the drawback of not being able to introduce human correction or manual operation during the operation of the hair follicle transplant robot.
[0075] Optionally, if the coordination rate between the operation object mapped by the first motion trajectory parameter and the second motion trajectory parameter and the hair follicle transplantation robot indicates that the current control mode needs to be switched, then the computer device determines the target control mode of the end effector according to the control mode corresponding to the coordination rate.
[0076] Step 206: Guide the end effector to perform the operation on the target object according to the target control mode.
[0077] The target object refers to the object to be worked on by the end effector. For example, if the end effector scribing lines on a mold, then the mold is the target object.
[0078] like Figure 3The diagram shows the module framework corresponding to the control method of a hair follicle transplantation robot. The information acquisition unit receives positioning assistance data collected by the sensing device and sends it to the feature extraction unit for feature extraction. The extracted feature information is then sent to the multimodal fusion unit, which fuses the multi-dimensional feature information to obtain fused data. This fused data is then sent to the data processing unit for processing, obtaining the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system. The positioning assistance data, the first motion trajectory parameters, and the second motion trajectory parameters are stored in the data storage unit. Simultaneously, the first and second motion trajectory parameters are sent to the prediction and judgment unit of the computer device to determine whether the current control mode needs to be switched. Alternatively, the user sets control information on the human-machine interface and sends the control information and the second motion trajectory parameters of the end effector in the visual coordinate system to the prediction and judgment unit of the computer device to determine whether the current control mode needs to be switched. If the prediction and judgment unit determines that the current control mode needs to be switched, it sends the prediction result to the control mode switching unit of the execution carriage. The control mode switching unit generates a mode switching command and sends the hair transplant plan, which the user has pre-planned on the human-machine interface based on parameters such as the density of the hair extraction area, the hair type of the implantation hole, etc., as well as the mode switching command, to the motion planning unit of the execution carriage. The motion planning unit plans the path according to the target control mode and sends the planning result to the motion control unit of the execution carriage. The motion control unit controls the end effector to perform the operation on the target object.
[0079] Optionally, if the target control mode of the end effector is the current control mode, the computer device guides the end effector to perform the operation on the target object according to the current control mode; if the target control mode of the end effector is a redefined control mode, the end effector is guided to perform the operation on the target object according to the redefined control mode.
[0080] In the aforementioned hair follicle transplant robot control method, during the process of the hair follicle transplant robot's end effector performing operations according to a preset planned path, the target control mode of the end effector is determined by analyzing the first motion trajectory parameters of the target object and the second motion trajectory parameters of the end effector. That is, during the operation of the end effector, the control of the target object is introduced to achieve the control purpose of the target object on the hair follicle transplant robot during the operation of the end effector. Moreover, according to different control modes, the target object can actively guide and constrain the hair follicle transplant robot to achieve intelligent and automated human-machine collaboration, solving the drawback of existing hair follicle transplant robots that cannot introduce human correction or manual operation during operation.
[0081] In one embodiment, determining the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system based on positioning assistance data used for locating the object being manipulated includes the following steps:
[0082] Step 1: Based on the positioning auxiliary data used to locate the operation object, determine the real-time three-dimensional position of the control target in the visual coordinate system, and the first motion trajectory parameters of the control target of the operation object in the visual coordinate system.
[0083] Optionally, by acquiring positioning auxiliary data for locating the target object through a sensing device, and estimating the depth information of the target object in the visual coordinate system through coordinate system transformation, the real-time three-dimensional position of the target object in the visual coordinate system can be obtained. Based on the change in the real-time three-dimensional position, the first motion trajectory parameters of the target object in the visual coordinate system can be determined.
[0084] Step 2: Obtain the first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot, and the second transformation relationship between the coordinate system corresponding to the end effector and the base coordinate system.
[0085] Optionally, by obtaining the first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot through hand-eye calibration, and combining it with the known second transformation relationship between the camera coordinate system where the end effector's end camera is located and the base coordinate system of the hair follicle transplantation robot, the visual coordinate system and the camera coordinate system where the end effector's end camera is located can be unified under one coordinate system.
[0086] Step 3: Based on the first and second transformation relationships, determine the real-time three-dimensional position of the end effector in the visual coordinate system, and the second motion trajectory parameters of the end effector in the visual coordinate system.
[0087] Since the end-effector camera can obtain the pose information of the target object in real time, and the sensing device can obtain the pose information of the target to be manipulated in real time, the real-time three-dimensional position of the end-effector in the visual coordinate system can be derived by combining the first transformation relationship and the second transformation relationship. Based on the change of the real-time three-dimensional position, the second motion trajectory parameters of the end-effector in the visual coordinate system are determined.
[0088] In one embodiment, positioning auxiliary data for locating the object being operated on is acquired through a sensing device, such as... Figure 4As shown, the sensing device includes a binocular stereo vision camera. During the operation of the end effector according to the preset planned path, the binocular stereo vision camera captures two-dimensional natural images in real time and extracts features to identify information such as the topological structure of the target object. Based on the factory-set internal and external parameters, binocular matching and epipolar correction are performed, and then the depth information of the target object in the visual coordinate system is estimated using the triangulation principle. The first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot is obtained through hand-eye calibration. Combined with the known second transformation relationship between the camera coordinate system of the end effector's end camera and the base coordinate system of the hair follicle transplantation robot, the visual coordinate system and the camera coordinate system of the end effector's end camera can be unified into one coordinate system. Since the end camera can obtain the pose information of the target object in real time, the sensing device can obtain the pose information of the target object in real time. Combining the conclusions in the previous section, the pose relationship between the target object and the target object can be derived. Thus, the constraint relationship of position, velocity, acceleration, and JERK (derivative of acceleration) between the target object and the hair follicle transplantation robot can be obtained based on the real-time trajectory. At the same time, all point cloud data of the human-computer environment can be converted to the world coordinate system to complete the three-dimensional reconstruction and rendering of the human-computer environment, and the reconstructed rendering model can be displayed on the human-computer interaction interface.
[0089] In some embodiments, such as Figure 5 As shown, the sensing device includes an RGB-D depth camera. During the operation of the end effector according to a pre-planned path, the RGB-D depth camera captures two-dimensional natural images while simultaneously emitting infrared light into the detected environment. It can measure pixel distances using infrared structured light or time-of-flight principles, thus completing the pairing of depth with pixels in the RGB image. Features of the manipulated target in the two-dimensional natural image are extracted and identified, thereby achieving depth estimation of the manipulated target in the visual coordinate system, estimating the depth information of the manipulated target in the visual coordinate system. The first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot is obtained through hand-eye calibration. Combined with the known second transformation relationship between the camera coordinate system of the end effector and the base coordinate system of the hair follicle transplantation robot, the pose relationship of the manipulated target relative to the target object can be derived. Therefore, the constraints of position, velocity, acceleration, and JERK (derivative of acceleration) between the manipulated object and the hair follicle transplantation robot are obtained based on the real-time trajectory. Simultaneously, all point cloud data of the human-machine environment can be converted to the world coordinate system to complete the three-dimensional reconstruction and rendering of the human-machine environment, and the reconstructed rendered model is displayed on the human-machine interface.
[0090] In some embodiments, such as Figure 6As shown, the sensing device includes a magnetic sensor. An electromagnet magnetic source generator is installed in the control room. The control target and the end effector are each equipped with a soft magnet. During the operation of the end effector according to the preset planned path, the magnetic sensor monitors the changes in the magnetic field under the constant electromagnet magnetic source generator in real time. The magnetic sensor calculates the spatial coordinates of the soft magnet by measuring the changes in the magnetic field. Through the anisotropic magnetization mechanism and magnetic-potential coupling relationship of the soft magnet, the pose coordinates of the control target and the end effector in the visual coordinate system are estimated by a nonlinear observer while keeping the system energy consumption constant. The first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot is obtained through hand-eye calibration. Combined with the known second transformation relationship between the camera coordinate system of the end effector and the base coordinate system of the hair follicle transplantation robot, the pose relationship between the control target and the target object can be derived. Thus, the constraint relationship of position, velocity, acceleration, and JERK (derivative of acceleration) between the control object and the hair follicle transplantation robot can be obtained based on the real-time trajectory. At the same time, the point cloud data of the human-computer environment can be converted to the world coordinate system for 3D reconstruction and rendering, and the reconstructed rendering model can be displayed on the human-computer interaction interface.
[0091] In some embodiments, such as Figure 7 As shown, the sensing device includes a temperature-sensing camera and a 3D ultrasonic locator. During the pre-operation preparation phase, the extrinsic parameters of the 3D ultrasonic probe and the temperature-sensing camera in the 3D ultrasonic locator are calibrated. During the operation of the end effector following the preset planned path, the 3D ultrasonic locator acquires environmental information, while the temperature-sensing camera further locks the relative relationship between the control target and the hair follicle transplantation robot, thus completing multimodal fusion of the images from the 3D ultrasonic locator and the temperature-sensing camera, and further estimating the pose coordinates of the control target and the end effector in the visual coordinate system. By obtaining the first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot through hand-eye calibration, and combining this with the known second transformation relationship between the camera coordinate system of the end effector and the base coordinate system of the hair follicle transplantation robot, the pose relationship of the control target relative to the target object can be derived. Therefore, based on the real-time trajectory, the constraints of position, velocity, acceleration, and JERK (derivative of acceleration) between the manipulated object and the hair follicle transplantation robot can be obtained. At the same time, the point cloud data of the human-computer environment can be converted to the world coordinate system for 3D reconstruction and rendering, and the reconstructed rendering model can be displayed on the human-computer interaction interface.
[0092] In some embodiments, such as Figure 8As shown, the sensing device includes a binocular stereo vision camera, an RGB-D depth camera, a magnetic sensor, a 3D ultrasonic locator, and a temperature sensor. It performs multimodal fusion of the pose relationship between the manipulation target and the target object obtained in the above embodiments to complete 3D reconstruction. Furthermore, it allows selection of the modality to be superimposed onto the mixed reality device worn by the user, i.e., selecting the 3D image of the manipulation target relative to the target object obtained in the above embodiments to be superimposed and displayed. It identifies the user's gestures and the relative coordinate relationship between the user and the end effector, thereby establishing constraints on the position, velocity, acceleration, and JERK (derivative of acceleration) between the user and the hair follicle transplant robot. Simultaneously, the point cloud data of the human-machine environment can be converted to the world coordinate system for 3D reconstruction and rendering, and the reconstructed rendered model is displayed on the human-machine interface. The user completes the fusion of the 3D image of the manipulation target relative to the target object obtained in the above embodiments and selects and switches between these 3D images through gestures, further enhancing the intelligence and safety between the user and the hair follicle transplant robot.
[0093] In this embodiment, by introducing multiple sensing schemes, the movement trajectory of the target being manipulated or the gestures of the object being manipulated are identified. Combined with real-time trajectory tracking, visual active constraint obstacle avoidance is achieved, which serves as a guided virtual wall, thereby achieving intelligent and automated human-machine collaboration.
[0094] In one embodiment, such as Figure 9 As shown, based on the first motion trajectory parameters and the second motion trajectory parameters, the target control mode of the end effector is determined, including:
[0095] Step 902: Input the first motion trajectory parameters and the second motion trajectory parameters into the pre-trained prediction model, and output the coordination rate between the operation object and the hair follicle transplantation robot.
[0096] The collaboration rate refers to the probability of cooperation between the object being operated on and the hair follicle transplantation robot for the same task. In this embodiment, a pre-trained prediction model determines whether to switch the current control mode based on the first motion trajectory parameters and the second motion trajectory parameters.
[0097] In some embodiments, such as Figure 10As shown, the prediction model is mainly based on multimodal learning. It processes the positioning assistance data, first motion trajectory parameters, second motion trajectory parameters, and constraint relationships through an embedding layer, and then concatenates these four types of data as input to the prediction model. The prediction model finally outputs the coordination rate. In this embodiment, the positioning assistance data is unprocessed raw data, such as the raw data directly acquired by the sensing device in the previous embodiment, including image information, temperature information, and ultrasound information. The constraint relationships in this embodiment are the position, velocity, acceleration, and JERK (derivative of acceleration) constraints between the operating object and the hair follicle transplantation robot in the previous embodiment.
[0098] In this embodiment, the prediction model uses CNN (Convolutional Neural Network), DNN (Deep Neural Network), and FC (Fully Connected Layer). The network that outputs the coherence rate can be replaced by other artificial intelligence methods, such as LSTM (Long Short-Term Memory Artificial Neural Network). CNN can also be replaced by other neural networks such as Multilayer Perceptrons (MLPs).
[0099] In some embodiments, such as Figure 11 As shown, the prediction model is mainly based on multimodal learning. In this embodiment, the pose data of the manipulated target, the second motion trajectory parameters, and the constraint relationships are processed through an embedding layer, and then the three types of data are concatenated as input to the prediction model. The prediction model finally outputs the coordination rate. Among them, the pose data of the manipulated target is the real-time three-dimensional position of the manipulated target in the visual coordinate system determined based on the positioning assistance data; the constraint relationships in this embodiment are the position, velocity, acceleration, and JERK (derivative of acceleration) constraint relationships between the manipulated object and the hair follicle transplantation robot in the above embodiment.
[0100] In this embodiment, the prediction model uses LSTM (Long Short-Term Memory) artificial neural network, DNN (Deep Neural Network), and FC (Fully Connected) layers. The network that outputs the coherence ratio can be replaced by other artificial intelligence methods, such as CNN (Convolutional Neural Network) or Multilayer Perceptrons (MLPs).
[0101] Step 904: Based on the coordination rate range corresponding to each control mode and the target coordination rate range in which the coordination rate is located, the control mode corresponding to the target coordination rate range is taken as the target control mode of the end effector.
[0102] Different control modes correspond to different coordination rate ranges. Determining whether the coordination rate falls within the threshold range of each control mode's trigger condition determines whether to continue prediction or switch control modes. For example, if the control mode corresponding to the target coordination rate range is the current control mode, prediction needs to continue and there is no need to switch control modes; if the control mode corresponding to the target coordination rate range is not the current control mode, it is determined that the current control mode needs to be switched to the target control mode.
[0103] Optionally, the computer device acquires the coordination rate output by the prediction model, compares the coordination rate with the coordination rate range corresponding to different control modes, determines the target coordination rate range in which the coordination rate is located, and uses the control mode corresponding to the target coordination rate range as the target control mode of the end effector.
[0104] In this embodiment, the first and second motion trajectory parameters are analyzed using a prediction model, and the coordination rate between the manipulated object and the hair follicle transplantation robot is output. Different control modes are set to correspond to different coordination rate ranges. By determining the target coordination rate range, the control mode corresponding to the target coordination rate range is used as the target control mode for the end effector. In the above method, based on the coordination rate between the manipulated object and the hair follicle transplantation robot, it is possible to quickly and accurately determine whether to switch the current control mode.
[0105] In one embodiment, such as Figure 12 As shown, according to the target control mode, the end effector is guided to perform a task on the target object, including:
[0106] Step 1202: Determine the virtual fixture corresponding to the target control mode. In the scenario where the operation object and the hair follicle transplantation robot work together, the virtual fixture provides motion boundaries for defining the motion range of the end effector or control commands for correcting the second motion trajectory.
[0107] Virtual fixtures are an application of virtual reality technology, designed to assist in the operation of objects. In scenarios where objects collaborate with hair follicle transplantation robots, they provide control commands to define the motion boundaries of the end effector's range of motion or to correct the end effector's trajectory parameters. Different control modes correspond to different virtual fixtures. In this embodiment, the control modes include curve guidance mode, pipe guidance mode, cone guidance mode, and potential field boundary guidance mode.
[0108] In some embodiments, such as Figure 13 As shown, if the target control mode is the curve-guided mode, then the virtual fixture corresponding to the target control mode is a virtual curve, which is used to guide the end effector to move along the virtual curve.
[0109] Where f(s) represents the curved trajectory, the solid line represents the actual tracking trajectory, and P tool S represents the position point of the end effector of the hair follicle transplant robot. tool τ represents the point on curve f(s) that is closest to the actual tracking trajectory. f (S tool ) represents the curve f(s) in S tool The tangent direction at point e tool P represents tool With S tool The error between [variables]. In curve guidance mode, high positional accuracy can be achieved, but it lacks some autonomous obstacle avoidance capability. Therefore, it can be applied to sliding guidance that requires high positional accuracy.
[0110] In some embodiments, such as Figure 14 As shown, if the target control mode is the pipeline guidance mode, then the virtual fixture corresponding to the target control mode is the virtual pipeline, which is used to define the motion space of the end effector within the virtual pipeline.
[0111] In this context, the virtual pipeline C is a pipeline with curve f(s) as its central axis and radius r. Pipeline guidance can be understood as an extension of curve guidance. Pipeline guidance provides both operational space for human-computer collaboration and ensures a certain level of security.
[0112] In some embodiments, such as Figure 15 As shown, if the target control mode is the cone-guided mode, then the virtual fixture corresponding to the target control mode is a virtual cone. The center of the cone surface of the virtual cone is the starting point of the end effector's motion, the vertex of the virtual cone is the ending point of the end effector's motion, and the virtual space defined within the virtual cone is the motion space of the end effector.
[0113] In this design, cone D is a cone with an axis starting point Q, an ending point P, and an opening angle α. Because the cone-shaped virtual clamp has customizable boundaries and can guide the hair follicle transplant robot to precisely locate the target point, both accurate positioning and safety are well guaranteed. Compared to the pipe-shaped guide mode, the cone-shaped guide mode offers more operational space near the starting position, making it suitable for guiding movements ranging from a large area to precise positioning.
[0114] In some embodiments, such as Figure 16 As shown, if the target control mode is the potential field boundary guidance mode, then the virtual fixture corresponding to the target control mode is a virtual region. The virtual region is used to establish a mapping relationship between the distance between the end effector and the virtual region and the resistance of the end effector.
[0115] Wherein, ρ(P) tool) represents the distance between the hair follicle transplantation robot and the prohibited region, where region F is the prohibited region. When ρ(P tool As the area F decreases, the resistance force encountered by the hair follicle transplant robot increases, preventing it from crossing region F. This human-robot collaboration obstacle can be expressed mathematically using an artificial potential field model. When ρ(P) tool When the potential field approaches zero, the force F acting on the hair follicle transplant robot in the potential field is... r This will approach infinity, so to avoid generating excessive force during actual operation, a safety distance ρ is set outside the prohibited area. min When ||ρ(P) tool )||<ρ min , let ||F r ||=F max In the potential field boundary guidance mode, the highest safety priority is ensured, which also serves as a safety guarantee for the human-machine collaboration system throughout the entire surgical process.
[0116] Step 1204: With both the virtual fixture and the end effector in the visual coordinate system, determine the real-time relative position of the virtual fixture and the end effector.
[0117] Optionally, such as Figure 17 As shown, the target control mode is input to the execution carriage. The execution carriage performs spatial geometric description of the virtual fixture and the end effector in the visual coordinate system, thereby placing both the virtual fixture and the end effector in the visual coordinate system. The relative positions of the virtual fixture and the end effector are continuously monitored in real time, and the operation defined by the target control mode is executed. During the execution of the target control mode, different processing methods will exhibit different implementation effects. For example, the implementation effect can be any one of the following: curve guidance, pipe guidance, cone guidance, and artificial potential field boundary.
[0118] Step 1206: Based on the real-time relative position, guide the end effector to perform the operation on the target object within the motion range defined by the virtual fixture.
[0119] In scenarios where the object being manipulated and the hair follicle transplant robot work collaboratively, the object moves in real time. Therefore, to achieve movement in such scenarios, the hair follicle transplant robot needs to plan its path and control its motion in real time. Thus, this embodiment provides the following... Figure 18The path planning for the end effector based on the visual servo controller is shown. Taking a robotic arm with 6 joints as an example, the 6 joints are represented by J1-J6. First, the motion characteristics of the end effector are extracted, and the actual position of the end effector is estimated based on the motion characteristics. The deviation between the actual position and the theoretical position of the end effector is input to the visual servo controller. The visual servo controller outputs the control amount required by each joint controller of the hair follicle transplant robot to make the end effector reach the theoretical position. According to the control amount, the rotation of each joint module of the robotic arm is controlled. The rotation amount of each joint module is fed back in real time until the end effector reaches the theoretical position, and then the control of each joint module is stopped.
[0120] In this embodiment, a visual servo controller based on image recognition can also be used to achieve path planning for the end effector, such as... Figure 19 As shown, taking a robotic arm with 6 joints as an example, the 6 joints are represented by J1-J6 respectively. First, the motion features of the end effector are extracted, and the deviation between the motion features and the image features is input to the visual servo controller. The image features refer to the position features extracted from the image containing the ideal pose of the end effector. The visual servo controller outputs the control amount required by each joint controller of the hair follicle transplant robot to make the end effector reach the theoretical position. Based on the control amount, the rotation of each joint module of the robotic arm is controlled. The rotation amount of each joint module is fed back in real time until the end effector reaches the theoretical position, and then the control of each joint module is stopped.
[0121] like Figure 20 The diagram illustrates the overall flow of the hair follicle transplantation robot control method. The computer imports a pre-planned path into the hair follicle transplantation robot, which then automatically runs along this path. A sensing device collects and processes environmental data in real time to obtain positioning assistance data. Based on this data, the robot determines the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system. Based on these parameters, it determines whether to switch the current control mode. If so, it determines the target control mode for the end effector and the corresponding virtual fixture. The robot then performs motion planning based on the motion boundaries defined by the virtual fixture, implementing the target control mode according to the planned scheme. Finally, the operation ends when the current operation is complete.
[0122] Optionally, the computer device executes different processing methods and displays different implementation effects based on the motion boundaries defined by the virtual fixture corresponding to the target control mode. The implementation effects can be curve guidance, pipe guidance, cone guidance, and artificial potential field boundary guidance, etc.
[0123] In this embodiment, by determining the virtual fixture corresponding to the target control mode, the virtual fixture provides motion boundaries for defining the range of motion of the end effector or control commands for correcting the second motion trajectory in scenarios where the operation object and the hair follicle transplantation robot work together. According to the motion boundaries or control commands defined by the virtual fixture, the end effector is guided to perform operations on the target object within the range of motion defined by the virtual fixture. Different implementation effects such as attraction guidance, sliding deviation, and elastic boundaries can be achieved, thereby enabling the hair follicle transplantation robot to automatically avoid collisions with the user, follow the user for guidance, or perform prescribed tasks such as elastic virtual walls, improving the level of intelligence and automation of the entire system and ensuring the safety and robustness of the surgical process.
[0124] In one embodiment, such as Figure 21 As shown, a method for controlling a hair follicle transplantation robot is provided, which specifically includes the following steps:
[0125] Step 2202: During the process of the end effector of the hair follicle transplantation robot performing the operation according to the preset planned path, the first motion trajectory parameters of the control target of the operation object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined based on the positioning auxiliary data used to locate the operation object.
[0126] Step 2204: Input the first motion trajectory parameters and the second motion trajectory parameters into the pre-trained prediction model, and output the coordination rate between the operation object and the hair follicle transplantation robot.
[0127] Step 2206: Based on the coordination rate range corresponding to each control mode and the target coordination rate range in which the coordination rate is located, the control mode corresponding to the target coordination rate range is taken as the target control mode of the end effector.
[0128] Step 2208: Determine the virtual fixture corresponding to the target control mode. In the scenario where the operation object and the hair follicle transplantation robot work together, the virtual fixture provides motion boundaries for defining the motion range of the end effector or control commands for correcting the second motion trajectory. If the target control mode is a curve guidance mode, then execute step 2210; if the target control mode is a pipe guidance mode, then execute step 2212; if the target control mode is a cone guidance mode, then execute step 2214; if the target control mode is a potential field boundary guidance mode, then execute step 2216.
[0129] Step 2210: The virtual fixture corresponding to the target control mode is a virtual curve. The virtual curve is used to guide the end effector to move along the virtual curve. Then, proceed to step 2218.
[0130] Step 2212: The virtual fixture corresponding to the target control mode is a virtual pipeline. The virtual pipeline is used to define the motion space of the end effector within the virtual pipeline. Then, proceed to step 2218.
[0131] Step 2214: The virtual fixture corresponding to the target control mode is a virtual cone. The center of the cone surface of the virtual cone is the starting point of the end effector's motion, the vertex of the virtual cone is the ending point of the end effector's motion, and the virtual space defined within the virtual cone is the motion space of the end effector. Then, proceed to step 2218.
[0132] Step 2216: The virtual fixture corresponding to the target control mode is a virtual region. The virtual region is used to establish a mapping relationship between the distance between the end effector and the virtual region and the resistance of the end effector. Then, proceed to step 2218.
[0133] Step 2218: With both the virtual fixture and the end effector in the visual coordinate system, determine the real-time relative position of the virtual fixture and the end effector.
[0134] Step 2220: Based on the real-time relative position, guide the end effector to perform operations on the target object within the motion range defined by the virtual fixture.
[0135] In this embodiment, visual servo control is used to realize real-time planning during the operation of the hair follicle transplantation robot. Combined with the active constraint guided virtual wall method, visual obstacle avoidance control of the hair follicle transplantation robot is completed. Thus, the end effector of the hair follicle transplantation robot can cooperate with the target object when performing operations on the target object.
[0136] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0137] Based on the same inventive concept, this application also provides a hair follicle transplantation robot control device for implementing the hair follicle transplantation robot control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the hair follicle transplantation robot control device provided below can be found in the limitations of the hair follicle transplantation robot control method described above, and will not be repeated here.
[0138] In one embodiment, such as Figure 22 As shown, a hair follicle transplantation robot control device is provided, including: a positioning module 100, a prediction module 200, and a guidance module 300, wherein:
[0139] The positioning module 100 is used to determine the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system, based on the positioning auxiliary data used to locate the target object, during the operation of the end effector of the hair follicle transplantation robot according to the preset planned path.
[0140] The prediction module 200 is used to determine the target control mode of the end effector based on the first motion trajectory parameters and the second motion trajectory parameters.
[0141] The guidance module 300 is used to guide the end effector to perform operations on the target object according to the target control mode.
[0142] In one embodiment, the positioning module 100 is further configured to: determine the real-time three-dimensional position of the control target in the visual coordinate system and the first motion trajectory parameters of the control target of the operation object in the visual coordinate system based on the positioning auxiliary data used for positioning the operation object.
[0143] Obtain the first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot, and the second transformation relationship between the coordinate system corresponding to the end effector and the base coordinate system of the hair follicle transplantation robot;
[0144] Based on the first and second transformation relationships, the real-time three-dimensional position of the end effector in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined.
[0145] In one embodiment, the prediction module 200 is further configured to: input the first motion trajectory parameters and the second motion trajectory parameters into the pre-trained prediction model, and output the coordination rate between the operation object and the hair follicle transplantation robot;
[0146] Based on the coordination rate range corresponding to each control mode and the target coordination rate range in which the coordination rate is located, the control mode corresponding to the target coordination rate range is taken as the target control mode of the end effector.
[0147] In one embodiment, the guidance module 300 is further configured to: determine the virtual fixture corresponding to the target control mode, wherein the virtual fixture provides motion boundaries for defining the range of motion of the end effector or control instructions for correcting the second motion trajectory in a scenario where the operation object and the hair follicle transplantation robot work together.
[0148] Determine the real-time relative positions of the virtual fixture and the end effector when both are in the visual coordinate system;
[0149] Based on the real-time relative position, the end effector is guided to perform operations on the target object within the motion range defined by the virtual fixture.
[0150] In one embodiment, the guidance module 300 is further configured to: if the target control mode is a curve guidance mode, then the virtual fixture corresponding to the target control mode is a virtual curve, and the virtual curve is used to guide the end effector to move along the virtual curve.
[0151] In one embodiment, the guidance module 300 is further configured to: if the target control mode is a pipeline guidance mode, then the virtual fixture corresponding to the target control mode is a virtual pipeline, and the virtual pipeline is used to define the motion space of the end effector within the virtual pipeline.
[0152] In one embodiment, the guidance module 300 is further configured to: if the target control mode is a cone guidance mode, then the virtual fixture corresponding to the target control mode is a virtual cone, the center of the cone surface of the virtual cone is the starting point of the motion of the end effector, the vertex of the virtual cone is the ending point of the motion of the end effector, and the virtual space defined within the virtual cone is the motion space of the end effector.
[0153] In one embodiment, the guidance module 300 is further configured to: if the target control mode is a potential field boundary guidance mode, then the virtual fixture corresponding to the target control mode is a virtual region, and the virtual region is used to establish a mapping relationship inversely proportional to the resistance of the end effector and the virtual region.
[0154] Each module in the aforementioned hair follicle transplantation robot control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0155] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.
[0157] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A computer-readable storage medium for controlling a hair follicle transplantation robot, wherein a computer program is stored thereon, characterized in that, When the computer program is executed by the processor, it performs the following steps: During the process of the end effector of the hair follicle transplantation robot performing the operation according to the preset planned path, the first motion trajectory parameters of the control target of the operation object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined based on the positioning auxiliary data used to locate the operation object. The target control mode of the end effector is determined based on the first motion trajectory parameters and the second motion trajectory parameters. According to the target control mode, the end effector is guided to perform a task on the target object, including: The virtual fixture corresponding to the target control mode is determined. In the scenario where the operation object and the hair follicle transplantation robot work together, the virtual fixture provides motion boundaries for defining the motion range of the end effector, or control commands for correcting the second motion trajectory. When both the virtual fixture and the end effector are in the visual coordinate system, determine the real-time relative position of the virtual fixture and the end effector; Based on the real-time relative position, the end effector is guided to perform operations on the target object within the motion range defined by the virtual fixture; Determining the target control mode of the end effector based on the first motion trajectory parameters and the second motion trajectory parameters includes: The first motion trajectory parameters and the second motion trajectory parameters are input into the pre-trained prediction model, and the coordination rate between the operation object and the hair follicle transplantation robot is output. Based on the coordination rate range corresponding to each control mode and the target coordination rate range in which the coordination rate is located, the control mode corresponding to the target coordination rate range is taken as the target control mode of the end effector.
2. The computer-readable storage medium according to claim 1, characterized in that, The positioning assistance data is acquired through a sensing device; the sensing device uses a binocular stereo vision measurement method and / or a spatial position sensor measurement method to perceive the relative positional relationship between the control target, the target object, and the end effector in real space.
3. The computer-readable storage medium according to claim 1, characterized in that, The step of determining the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system based on the positioning auxiliary data used for locating the target object includes: Based on the positioning auxiliary data used to locate the operation object, the real-time three-dimensional position of the control target in the visual coordinate system and the first motion trajectory parameters of the control target of the operation object in the visual coordinate system are determined. Obtain the first transformation relationship between the visual coordinate system and the base coordinate system of the hair follicle transplantation robot, and the second transformation relationship between the coordinate system corresponding to the end effector and the base coordinate system of the hair follicle transplantation robot; Based on the first transformation relationship and the second transformation relationship, the real-time three-dimensional position of the end effector in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system are determined.
4. The computer-readable storage medium according to claim 1, characterized in that, The step of determining the virtual fixture corresponding to the target control mode includes: If the target control mode is a curve-guided mode, then the virtual fixture corresponding to the target control mode is a virtual curve, and the virtual curve is used to guide the end effector to move along the virtual curve.
5. The computer-readable storage medium according to claim 1, characterized in that, The step of determining the virtual fixture corresponding to the target control mode includes: If the target control mode is a pipeline guidance mode, then the virtual fixture corresponding to the target control mode is a virtual pipeline, and the virtual pipeline is used to define the motion space of the end effector within the virtual pipeline.
6. The computer-readable storage medium according to claim 1, characterized in that, The step of determining the virtual fixture corresponding to the target control mode includes: If the target control mode is a cone-guided mode, then the virtual fixture corresponding to the target control mode is a virtual cone, the center of the cone surface of the virtual cone is the starting point of the movement of the end effector, the vertex of the virtual cone is the ending point of the movement of the end effector, and the virtual space defined within the virtual cone is the movement space of the end effector.
7. The computer-readable storage medium according to claim 1, characterized in that, The step of determining the virtual fixture corresponding to the target control mode includes: If the target control mode is a potential field boundary guidance mode, then the virtual fixture corresponding to the target control mode is a virtual region. The virtual region is used to establish a mapping relationship between the distance between the end effector and the virtual region and the resistance of the end effector.
8. A control device for a hair follicle transplantation robot, characterized in that, The device includes: The positioning module determines the first motion trajectory parameters of the target object in the visual coordinate system and the second motion trajectory parameters of the end effector in the visual coordinate system based on the positioning auxiliary data used to locate the target object during the operation of the hair follicle transplantation robot's end effector in the preset planned path. The prediction module determines the target control mode of the end effector based on the first motion trajectory parameters and the second motion trajectory parameters. The guidance module guides the end effector to perform a task on the target object according to the target control mode. The guidance module is also used to: determine the virtual fixture corresponding to the target control mode; in scenarios where the virtual fixture works collaboratively with the hair follicle transplantation robot on the target object, provide motion boundaries for defining the range of motion of the end effector, or control commands for correcting the second motion trajectory; determine the real-time relative position of the virtual fixture and the end effector when both are in the visual coordinate system; and guide the end effector to perform operations on the target object within the range of motion defined by the virtual fixture based on the real-time relative position. The prediction module is further configured to input the first motion trajectory parameters and the second motion trajectory parameters into a pre-trained prediction model, and output the coordination rate between the operation object and the hair follicle transplantation robot; it is also configured to, based on the coordination rate range corresponding to each control mode and the target coordination rate range in which the coordination rate is located, use the control mode corresponding to the target coordination rate range as the target control mode of the end effector.