TMS robot navigation control method based on visual positioning

Abstract

The invention discloses a TMS robot navigation control method based on visual localization. The method comprises the following steps: acquiring visual detection data, magnetic resonance image data and localization tracking data of the head of a diagnosis and treatment object; establishing a head three-dimensional model of the diagnosis and treatment object according to the visual detection data and the magnetic resonance image data; performing pose correction on the head three-dimensional model according to the positioning and tracking data; and generating a motion track of the diagnosis and treatment coil according to the head three-dimensional model after pose correction and the target stimulation target. The head of the TMS diagnosis and treatment object is positioned and tracked through the visual positioning technology and the optical tracking technology, the motion path of the diagnosis and treatment coil is adaptively planned based on the real-time three-dimensional model of the head, the coil positioning precision and positioning efficiency can be improved, and the treatment effect, safety and patient comfort are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent medical treatment, and in particular to a TMS robot navigation control method based on visual positioning. BACKGROUND

[0002] Transcranial magnetic stimulation (TMS) is a kind of nerve regulation therapy for non-invasive stimulation of brain nerves by using alternating magnetic field. In the process of implementing TMS, the specific part (stimulation area) of the patient's head is stimulated by a stimulating coil, and based on the principle of electromagnetic induction, the pulse magnetic field generated by the stimulating coil generates induced current in the cerebral cortex, thereby stimulating the cerebral cortex nerves. It is usually used for the treatment of mental diseases and nerve rehabilitation.

[0003] Traditional TMS treatment is usually operated by a doctor holding a magnetic stimulation coil to align the specific part of the patient's head, and the positioning efficiency and positioning accuracy are low. In the prior art, coil positioning assistance is usually achieved by using transcranial magnetic stimulation navigation control technology, which imports the magnetic resonance imaging data of the patient into the transcranial magnetic stimulation navigation software to realize path planning for a specific treatment area. However, the existing transcranial magnetic stimulation navigation control technology has the following problems: the existing transcranial magnetic stimulation navigation control technology cannot monitor the position of the patient, nor can it automatically adjust the position of the coil according to the movement of the patient's position, which is easy to cause the treatment area to deviate, affecting the treatment accuracy and safety. SUMMARY

[0004] In view of the above problems, the present application provides a TMS robot navigation control method based on visual positioning, which automatically plans the motion path of the diagnosis and treatment coil by positioning and tracking the movement of the patient's head, solving the problem that the existing transcranial magnetic stimulation navigation control technology cannot automatically adjust the position of the coil according to the movement of the patient's position, resulting in deviation of the treatment area.

[0005] To solve the above problems, the present application provides a TMS robot navigation control method based on visual positioning, the TMS robot comprising a mechanical arm and a diagnosis and treatment coil, the diagnosis and treatment coil being fixed to the mechanical arm, the method comprising: acquiring visual detection data, magnetic resonance image data and positioning tracking data of the head of a diagnosis and treatment object; establishing a three-dimensional model of the head of the diagnosis and treatment object according to the visual detection data and the magnetic resonance image data; correcting the pose of the three-dimensional model of the head according to the positioning tracking data; generating the motion trajectory of the diagnosis and treatment coil according to the three-dimensional model of the head after pose correction and the target stimulation target point.

[0006] Optionally, the establishing the head three-dimensional model of the subject based on the visual detection data and the magnetic resonance image data comprises: establishing a head three-dimensional point cloud of the subject based on the visual detection data; establishing a brain three-dimensional model based on the magnetic resonance image data, wherein the brain three-dimensional model marks a target stimulation target point; and registering the head three-dimensional point cloud and the brain three-dimensional model to establish the head three-dimensional model.

[0007] Optionally, the establishing the head three-dimensional point cloud of the subject based on the visual detection data comprises: performing face detection based on images of different perspectives and obtaining face rectangular frame coordinates in the images; calculating parallax values between different images according to the face rectangular frame coordinates; converting two-dimensional image coordinates into three-dimensional space coordinates according to the parallax values and camera parameters to establish an initial head point cloud; and performing point cloud preprocessing on the initial head point cloud to generate the head three-dimensional point cloud.

[0008] Optionally, the registering the head three-dimensional point cloud and the brain three-dimensional model to establish the head three-dimensional model comprises: obtaining a plurality of anatomical feature points based on the head three-dimensional point cloud; obtaining model feature points corresponding to the anatomical feature points based on the head three-dimensional model; establishing a coarse registration transformation according to a spatial relationship between the anatomical feature points and the corresponding model feature points to coarsely register the head three-dimensional point cloud and the brain three-dimensional model; and performing fine registration on the head three-dimensional point cloud and the brain three-dimensional model based on an iterative closest point algorithm.

[0009] Optionally, the generating the motion trajectory of the treatment coil according to the head three-dimensional model after the pose correction and the target stimulation target point comprises: obtaining a theoretical coordinate of the target stimulation target point in a model coordinate system of the head three-dimensional model, and converting the theoretical coordinate in the model coordinate system into a robot reference coordinate system to establish a treatment point coordinate; establishing a target pose of the treatment coil in the robot reference coordinate system according to a stimulation direction requirement of the target stimulation target point; and generating the motion trajectory of the treatment coil according to the treatment point coordinate and the target pose.

[0010] Optionally, the generating the motion trajectory of the treatment coil according to the head three-dimensional model after the pose correction and the target stimulation target point further comprises: obtaining a model pose transformation amount at an adjacent sampling time; and updating the treatment point coordinate when the model pose transformation amount exceeds a preset tolerance threshold.

[0011] Optionally, the pose correction of the head three-dimensional model according to the positioning tracking data comprises: determining a real-time pose of the head of the diagnosis and treatment object according to the positioning tracking data; converting the real-time pose of the head from a tool coordinate system to a model coordinate system to obtain a model real-time pose; and correcting the pose of the head three-dimensional model according to the model real-time pose.

[0012] Optionally, the TMS robot navigation control method based on visual positioning further comprises: obtaining a kinematics model and a dynamics model of the mechanical arm, and optimizing the motion trajectory according to the kinematics model and the dynamics model.

[0013] Optionally, the TMS robot navigation control method based on visual positioning further comprises: obtaining diagnosis and treatment feedback data in a diagnosis and treatment process, and adjusting diagnosis and treatment parameters according to the diagnosis and treatment feedback data.

[0014] Optionally, before the visual detection data, the magnetic resonance image data and the positioning tracking data of the head of the diagnosis and treatment object are obtained, the method further comprises: calibrating a binocular camera to obtain an intrinsic matrix, a distortion coefficient of each camera and an extrinsic matrix between the two cameras; and calibrating a head tracking tool fixed to the head of the diagnosis and treatment object to obtain a spatial relationship between a head model and the head tracking tool.

[0015] The present application has the beneficial effects compared with the prior art: a head three-dimensional model is established through the visual detection data and the magnetic resonance image data of the head of the diagnosis and treatment object, and a motion path of a diagnosis and treatment coil is automatically planned through positioning and tracking of the movement of the head of the patient, which solves the problem that the existing transcranial magnetic stimulation navigation control technology cannot automatically adjust the position of the coil according to the position of the patient, resulting in deviation of the treatment area, can improve the positioning accuracy and efficiency of the coil, improve the treatment effect, safety and comfort of the patient, realize high-precision and non-invasive brain treatment, reduce the operation complexity of TMS treatment and improve the work efficiency of medical personnel.

[0016] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0018] Figure 1 A flowchart of a TMS robot navigation control method based on visual positioning provided by an embodiment of the present application is shown in FIG. 1.

[0019] Figure 2 A structural diagram of a TMS robot and its auxiliary equipment provided by an embodiment of the present application is shown in FIG. 2.

[0020] Figure 3 A flowchart of a head three-dimensional model construction method based on visual positioning provided by an embodiment of the present application is shown in FIG. 3.

[0021] Figure 4 A flowchart of a path planning method of a diagnosis and treatment coil provided by an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0024] Figure 1 A flowchart of a TMS robot navigation control method based on visual positioning provided by an embodiment of the present application is shown in FIG. 1. Figure 2 A structural diagram of a TMS robot and its auxiliary equipment provided by an embodiment of the present application is shown in FIG. 2. The present embodiment can be applied to an automatic TMS application scenario. The TMS robot determines a stimulation area and automatically positions and tracks the head of a diagnosis and treatment object. The method can be executed by the TMS robot, which can be realized in the form of hardware and / or software.

[0025] As Figure 2As shown, the TMS robot 1 includes a mechanical arm 101 and a therapy coil 102 fixed to the end of the mechanical arm 101, and the mechanical arm 101 is arranged on a movable trolley. In this embodiment, the mechanical arm 101 can be a mechanical arm with six degrees of freedom. The auxiliary equipment of the TMS robot includes but is not limited to: a binocular camera 2, an optical tracker 3, a head tracking tool 4, a coil tracking tool 5, a reclining chair 6 and an operating table 7. Among them, the binocular camera and the optical tracker are arranged at a position that can capture the robot working area and the patient's head at the same time; the tracking tool is an object that can be tracked by an infrared tracking camera, for example, a rigid body with reflective marker points (Marker), and the head tracking tool can be fixed on the patient's head and move with the head; the coil tracking tool can be fixed on the therapy coil and move with the coil.

[0026] Referring to Figure 1 and Figure 2 As shown, the TMS robot navigation control method based on visual positioning of the present application includes the following steps:

[0027] S1: acquiring visual detection data, magnetic resonance image data and positioning tracking data of the head of the subject.

[0028] The visual detection data can be understood as a data set obtained by the TMS robot through a visual system (such as a camera, an infrared camera, a depth camera, a laser radar or a 3D scanner, etc.) for understanding the structure and position of the patient's head in three-dimensional space. In this embodiment, a binocular camera can be used to collect visual detection data. Specifically, the binocular camera is triggered synchronously to obtain face images at the same instant. The original images collected by the binocular camera are corrected according to the calibration parameters.

[0029] The magnetic resonance image data can be understood as a data set representing the anatomical information and spatial coordinates of the brain of the subject. Specifically, the magnetic resonance image data of the brain of the subject can be obtained by using magnetic resonance imaging (MRI) technology. In the pixel coordinate system, the magnetic resonance image data is a three-dimensional voxel grid, and each voxel has its unique index (i, j, k) and corresponding three-dimensional physical coordinates (x, y, z).

[0030] The positioning tracking data can be understood as a data set representing the position and pose of a key object (such as a patient's head, the end of a mechanical arm, a therapeutic coil) in a three-dimensional space. In this embodiment, an optical tracker (such as NDI Polaris Vega) can be placed in a position that can simultaneously see the robot working area (at least capable of covering the area where the end of the mechanical arm and the therapeutic coil are located) and the patient's head. Specifically, a coil tracking tool can be fixed on the therapeutic coil, and a head tracking tool can be fixed on the head of the subject, and the optical tracker can measure the position and pose of the therapeutic coil in the three-dimensional space by real-time monitoring of the coil tracking tool, and measure the position and pose of the subject's head in the three-dimensional space by real-time monitoring of the head tracking tool.

[0031] S2: Establishing a three-dimensional model of the subject's head according to the visual detection data and the magnetic resonance image data.

[0032] The visual detection data is used to provide high-precision, real-texture head surface (such as face and scalp) geometric information, and the magnetic resonance image data is used to provide head internal (such as brain cortex, skull) anatomical structure.

[0033] The three-dimensional model of the head is a three-dimensional model of the head established based on the head surface (such as face and scalp) geometric information and the head internal (such as brain cortex, skull) anatomical structure.

[0034] Specifically, a head surface model with facial feature points (such as nose tip, left inner corner of eye, right inner corner of eye, chin tip) is constructed according to the visual detection data; a head internal model with a target stimulation target point is constructed according to the magnetic resonance image data; the head surface model and the head internal model are fused by surface registration to establish the three-dimensional model of the subject's head.

[0035] S3: Pose correction of the three-dimensional model of the head according to the positioning tracking data.

[0036] In some optional embodiments, the pose correction of the three-dimensional model of the head according to the positioning tracking data includes: determining a real-time pose of the subject's head according to the positioning tracking data; converting the real-time pose of the head from a tool coordinate system to a model coordinate system to obtain a real-time pose of the model; and performing pose correction of the three-dimensional model of the head according to the real-time pose of the model.

[0037] Exemplarily, the real-time pose of the head can be represented as , the real-time pose of the model can be represented as , the transformation matrix of the tool coordinate system to the model coordinate system can be represented as , the real-time pose of the head , the real-time pose of the model , and the transformation matrix satisfy: .

[0038] Specifically, in the diagnosis and treatment process, the positioning tracking data obtains a new model real-time pose by tracking head movement , and the position and pose of the head three-dimensional model are adjusted according to the model real-time pose , and the target stimulation target point in the head three-dimensional model is updated to a new position following the model pose correction.

[0039] S4: generating the motion trajectory of the diagnosis and treatment coil according to the head three-dimensional model after pose correction and the target stimulation target point.

[0040] In the embodiment, the motion trajectory is established based on the coil path constraint (obstacles such as the head model need to be avoided), the mechanical arm self-collision constraint (satisfying that the mechanical arm connecting rods cannot collide), the boundary constraint (established based on the initial position of the coil and the position of the stimulation target point), and the optimization objective of the shortest motion time. It should be noted that the specific strategy of adjusting the constraint condition and the optimization objective can be adjusted by the person skilled in the art based on the actual motion planning scene, and the specific limitation is not made.

[0041] Specifically, the target stimulation target point in the head three-dimensional model is updated to a new position following the model pose correction, the motion trajectory of the diagnosis and treatment coil is generated according to the updated position, and the planned motion trajectory is sent to the robot controller. The robot controller performs kinematics solving and interpolation to drive the mechanical arm joint motor to smoothly track the motion trajectory.

[0042] Therefore, the technical scheme of the present application establishes a head three-dimensional model through visual detection data and magnetic resonance image data of the head of the diagnosis and treatment object, and automatically plans the motion path of the diagnosis and treatment coil through positioning and tracking the movement of the patient's head, solves the problem that the existing transcranial magnetic stimulation navigation control technology cannot automatically adjust the coil position according to the patient position movement, leading to the deviation of the treatment area, improves the coil positioning accuracy and positioning efficiency, improves the treatment effect, safety and patient comfort, realizes high-precision and non-invasive brain treatment, reduces the operation complexity of TMS treatment, and improves the work efficiency of medical personnel.

[0043] Figure 3 A flowchart of a head three-dimensional model construction method based on visual positioning provided by the embodiment of the present application is shown in FIG. 2. Figure 3 As shown in the above step S2, the head three-dimensional model of the diagnosis and treatment object is established according to the visual detection data and the magnetic resonance image data, which includes:

[0044] S201: establishing a head three-dimensional point cloud of the diagnosis and treatment object based on visual detection data.

[0045] In some optional embodiments, the step S201 specifically comprises: performing face detection based on images of different perspectives, and obtaining face rectangular frame coordinates in the images; calculating disparity values between different images according to the face rectangular frame coordinates; converting two-dimensional image coordinates into three-dimensional space coordinates according to the disparity values and camera parameters, and establishing an initial head point cloud; and performing point cloud preprocessing on the initial head point cloud to generate a head three-dimensional point cloud. Optionally, the point cloud preprocessing includes but is not limited to: denoising, downsampling, and standardization processing. The denoising refers to removing discrete noise points caused by environmental interference or scanning errors. Typically, a statistical outlier removal filter can be used to perform denoising. The downsampling refers to a preprocessing step of reducing the amount of point cloud data, reducing the computational complexity, and retaining key geometric features. Typically, voxel grid downsampling can be used, that is, small cubes (i.e., voxels) are divided in space, and only one representative point is retained in each voxel.

[0046] Specifically, before performing the head three-dimensional point cloud construction, the binocular camera is calibrated to obtain an intrinsic matrix, distortion coefficients of each camera, and an extrinsic matrix between the two cameras, i.e., a rotation matrix and a translation vector of the image. When performing the head three-dimensional point cloud construction, the left and right cameras of the binocular camera are triggered synchronously to obtain face images at the same instant, and the original images collected by the binocular camera are subjected to epipolar rectification according to the calibration parameters. Further, the disparity map is converted into a three-dimensional point cloud in combination with the intrinsic matrix of each camera and the extrinsic matrix between the two cameras. In order to speed up the point cloud reconstruction, face detection is performed in the left and right images to obtain face region rectangular frame coordinates. In the face matrix frame, each pixel point in the left image is found in the right image to obtain the best matching point, and the horizontal pixel distance between them, i.e., the disparity, is calculated. According to the principle of triangulation, the two-dimensional image coordinates are converted into three-dimensional point cloud coordinates according to the disparity values and the camera parameters. Further, the original point cloud of the face region is subjected to denoising, downsampling, and standardization processing to optimize the point cloud quality, eliminate the influence of noise, posture, position, and size changes on subsequent processing.

[0047] S202: establishing a brain three-dimensional model based on magnetic resonance image data.

[0048] The brain three-dimensional model is marked with a target stimulation target point.

[0049] S203: registering the head three-dimensional point cloud and the brain three-dimensional model to establish a head three-dimensional model.

[0050] In some optional embodiments, step S203 specifically includes: acquiring multiple anatomical feature points based on the head 3D point cloud; acquiring model feature points corresponding to the anatomical feature points based on the head 3D model; establishing a coarse registration transformation based on the spatial relationship between the anatomical feature points and the corresponding model feature points, and performing coarse registration between the head 3D point cloud and the brain 3D model; and performing fine registration between the head 3D point cloud and the brain 3D model based on the iterative nearest point algorithm. Specifically, coarse registration between the head 3D point cloud and the brain 3D model refers to initially aligning the head 3D point cloud and the brain 3D model and registering the head 3D point cloud to the brain 3D model. Fine registration between the head 3D point cloud and the brain 3D model refers to finely aligning the head 3D point cloud and the brain 3D model and registering the head 3D point cloud to the brain 3D model.

[0051] Specifically, for each anatomical feature point p in the 3D point cloud of the head... i (e.g., tip of the nose, left inner corner of the eye, right inner corner of the eye, tip of the chin), search for the point with the closest Euclidean distance on the triangular mesh surface of the 3D brain model, and use it as the model feature point m corresponding to the anatomical feature point. i Each group corresponds to the anatomical feature point p i and model feature points m i Form a set of feature point pairs In the process of establishing feature point pairs, distance thresholds and normal constraints are set. By eliminating point pairs whose distance exceeds the distance threshold, mismatches caused by incorrect correspondences between different anatomical locations can be avoided. By eliminating point pairs whose normal vector directions differ too much (e.g., the tilt angle is greater than 10°), matching scalp points to areas with mismatched curvature, such as cheeks, can be avoided. After eliminating mismatched feature point pairs, the remaining feature point pairs are used as the basis for further analysis. Calculate a registration transform that minimizes the sum of squared distances between current feature point pairs. For example, a registration transform can be defined as... Registration transformation Applied to the 3D point cloud of the head, the following transformation equation is established: ,in, This represents the transformed point cloud coordinates, which are the coordinates in the model coordinate system. Represents the point cloud coordinates before the transformation; Represents the rotation component of the registration transformation; This represents the translation component of the registration transformation. Further, the average error change of the point cloud before and after the transformation is calculated. If the error change is less than a preset change threshold (e.g., ), then... Alternatively, if the maximum number of iterations is reached, the iteration stops, and the final registration transformation matrix is ​​established based on the data from the last iteration to register the head 3D point cloud to the brain 3D model.

[0052] Figure 4 A flowchart of a path planning method of a treatment coil is provided for an embodiment of the present application. Referring to FIG. 4, the path planning method of the treatment coil comprises the following steps. Figure 4 In step S4, a movement trajectory of the treatment coil is generated according to the head three-dimensional model after the pose correction and the target stimulation target point, and specifically comprises:

[0053] S401: Obtain the theoretical coordinates of the target stimulation target point in the model coordinate system of the head three-dimensional model, and convert the theoretical coordinates in the model coordinate system to the robot reference coordinate system to establish the treatment point coordinates.

[0054] In some optional embodiments, step S401 specifically comprises: constructing the model coordinate system, such as selecting at least 3 facial anatomical feature points in the head three-dimensional model to construct the model coordinate system, determining the origin and X, Y, Z axis directions of the coordinate system, including but not limited to: selecting the tip of the nose as the origin of the model coordinate system, selecting a vector pointing from the left inner corner of the eye to the right inner corner of the eye as the X axis of the model coordinate system, selecting a vector pointing from the midpoint of the two inner corners of the eye to the tip of the nose as the Y axis of the model coordinate system, and calculating the Z axis of the model coordinate system through the cross product of the X axis and the Y axis to form a right-handed coordinate system. A transformation matrix from the camera coordinate system to the model coordinate system is established according to the facial anatomical feature point coordinates in the camera coordinate system and the facial anatomical feature point coordinates in the model coordinate system. Further, the transformation matrix from the model coordinate system to the robot reference coordinate system is calculated according to the transformation matrix from the model coordinate system to the camera coordinate system and the transformation matrix from the camera coordinate system to the robot reference coordinate system; the coordinates of the target stimulation target point in the robot reference coordinate system, i.e. the treatment point coordinates in the robot reference coordinate system, are calculated according to the theoretical coordinates of the target stimulation target point in the model coordinate system and the transformation relationship from the model coordinate system to the robot reference coordinate system.

[0055] S402: Establish the target pose of the treatment coil in the robot reference coordinate system according to the stimulation direction requirement of the target stimulation target point.

[0056] In some optional embodiments, step S402 specifically comprises: finding the scalp surface point corresponding to the target stimulation target point on the head three-dimensional model, and calculating the surface normal vector at the point The plane where the treatment coil is located can be perpendicular to the normal vector , i.e. the central axis of the treatment coil is parallel to the normal vector . Since rotating the treatment coil around the normal vector may affect the stimulation effect of the coil, a tangent vector is introduced, and a complete coil rotation matrix is constructed according to the normal vector and the tangent vector . If the coil rotation matrix is defined as , the treatment point coordinates are The target pose of the diagnosis and treatment coil in the robot reference coordinate system can be expressed as: .

[0057] S403: generating a motion trajectory of the diagnosis and treatment coil according to the treatment point coordinates and the target pose.

[0058] The motion trajectory of the diagnosis and treatment coil is a series of time point sequences in joint space or Cartesian space.

[0059] In this embodiment, the design principle of the path planning algorithm at least meets the factors of obstacle avoidance, reduction of treatment time, reduction of patient discomfort, etc.

[0060] Specifically, the head three-dimensional model can be added as an obstacle to the coil path planning scene, and the treatment point coordinates and the target pose are added to the coil path planning scene, and a collision-free and smooth motion trajectory is calculated by combining path planning and collision detection technology.

[0061] In some optional embodiments, the motion trajectory of the diagnosis and treatment coil is generated according to the head three-dimensional model after pose correction and the target stimulation target point, and further includes: obtaining a model pose transformation amount at an adjacent sampling time; and updating the treatment point coordinates when the model pose transformation amount exceeds a preset tolerance threshold. Specifically, when the model pose transformation amount exceeds the preset tolerance threshold, the transformation matrix of the model coordinate system to the camera coordinate system is updated by the pose of the head three-dimensional model at the current sampling time, and the coordinates of the target stimulation target point in the robot reference coordinate system, i.e., the treatment point coordinates in the robot reference coordinate system, are recalculated according to the updated transformation matrix.

[0062] Therefore, the technical scheme of the present application introduces dynamic monitoring and path planning algorithm, so that the TMS robot can autonomously complete the TMS diagnosis and treatment task without human intervention.

[0063] In some optional embodiments, the TMS robot navigation control method based on visual positioning further includes: obtaining a kinematics model and a dynamics model of the mechanical arm, and optimizing the motion trajectory according to the kinematics model and the dynamics model. The kinematics model is used to describe the geometric relationship between the joint space of the mechanical arm and the Cartesian space of the diagnosis and treatment coil. Typically, the kinematics model of the mechanical arm includes but is not limited to at least one of the following: inverse kinematics model and end position adjustment model. The dynamics model is used to describe the relationship between the motion of the mechanical arm and the force or torque required to produce the motion. Typically, the dynamics model can use the Lagrange dynamics equation. Specifically, during the TMS diagnosis and treatment process, the target pose T desired of the diagnosis and treatment coil is established by combining the end position adjustment model, the Lagrange dynamics equation is established by combining the position, velocity and acceleration vectors of the joint, and the motion trajectory of the diagnosis and treatment coil is optimized according to the target pose T desiredSolve joint angles q satisfying the equation to realize the closed-loop control of the robot arm.

[0064] In some optional embodiments, the TMS robot navigation control method based on visual positioning further comprises: acquiring diagnosis and treatment feedback data in the diagnosis and treatment process, and adjusting diagnosis and treatment parameters according to the diagnosis and treatment feedback data. Wherein, the diagnosis and treatment feedback data can be understood as a data set of closed-loop feedback information for evaluating TMS efficacy, ensuring safety and optimizing treatment parameters. Typically, the diagnosis and treatment feedback data includes but is not limited to: neurophysiological feedback data, clinical efficacy feedback data and operation and safety feedback data in the TMS treatment process. Diagnosis and treatment parameters can be understood as parameters representing the state of TMS treatment. Typically, the diagnosis and treatment parameters include but are not limited to: the position, posture and stimulation intensity of the diagnosis and treatment coil. Specifically, the TMS robot of the present application can provide an intuitive human-computer interaction interface for the doctor to operate the TMS robot and its auxiliary equipment, which can display diagnosis and treatment parameters (such as the position, posture and stimulation intensity of the diagnosis and treatment coil) and diagnosis and treatment feedback data (such as patient response and treatment effect evaluation) in real time through the human-computer interaction, and adaptively adjust the treatment parameters according to the diagnosis and treatment feedback data, or allow the operator to manually adjust the treatment parameters, which can improve the treatment effect.

[0065] In some optional embodiments, before acquiring the visual detection data, magnetic resonance image data and positioning tracking data of the head of the diagnosis and treatment object, the TMS robot navigation control method based on visual positioning further comprises: calibrating the binocular camera to acquire the intrinsic matrix, distortion coefficient of each camera and the extrinsic matrix between the two cameras; and calibrating the head tracking tool fixed to the head of the diagnosis and treatment object to acquire the spatial relationship between the head model and the head tracking tool. Exemplarily, the spatial relationship between the head model and the head tracking tool can be represented as . Specifically, before the treatment starts, the spatial relationship of the TMS robot and its auxiliary equipment needs to be established, the pose of the head tracking tool is measured based on the optical tracker, and the transformation matrix of the head tracking tool in the tool coordinate system is obtained, denoted as . Further, the spatial relationship between the head model and the head tracking tool is established by point cloud registration or probe point taking method, denoted as , according to which the spatial relationship between the head model and the head tracking tool is obtained . Thus, through the calibration work before the diagnosis and treatment, the transformation matrix is established, which provides data basis for coordinate system transformation in the visual positioning process, and can provide positioning efficiency.

[0066] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps recited in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0067] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A TMS robot navigation control method based on visual positioning, the TMS robot comprising a mechanical arm and a therapy coil, the therapy coil being fixed to the mechanical arm, characterized in that, The method comprises: acquiring visual detection data, magnetic resonance image data and positioning tracking data of the head of the subject; establishing a three-dimensional model of the head of the subject according to the visual detection data and the magnetic resonance image data; correcting the pose of the three-dimensional model of the head according to the positioning tracking data; generating a motion trajectory of the treatment coil according to the three-dimensional model of the head after pose correction and a target stimulation target point.

2. The visual positioning based TMS robot navigation control method of claim 1, wherein, The three-dimensional model of the head of the subject is established according to the visual detection data and the magnetic resonance image data, comprising: establishing a three-dimensional point cloud of the head of the subject based on the visual detection data; establishing a three-dimensional model of the brain based on the magnetic resonance image data, wherein the three-dimensional model of the brain marks the target stimulation target point; registering the three-dimensional point cloud of the head and the three-dimensional model of the brain to establish the three-dimensional model of the head.

3. The visual positioning based TMS robot navigation control method of claim 2, wherein, The three-dimensional point cloud of the head of the subject is established based on the visual detection data, comprising: performing face detection based on images of different perspectives and acquiring face rectangular frame coordinates in the images; calculating parallax values between different images according to the face rectangular frame coordinates; converting two-dimensional image coordinates to three-dimensional space coordinates according to the parallax values and camera parameters to establish an initial point cloud of the head; performing point cloud preprocessing on the initial point cloud of the head to generate the three-dimensional point cloud of the head.

4. The visual positioning based TMS robot navigation control method of claim 2, wherein, The three-dimensional model of the head is established by registering the three-dimensional point cloud of the head and the three-dimensional model of the brain, comprising: obtaining a plurality of anatomical feature points based on the three-dimensional point cloud of the head; obtaining model feature points corresponding to the anatomical feature points based on the three-dimensional model of the head; establishing a coarse registration transformation according to the spatial relationship between the anatomical feature points and the corresponding model feature points to coarsely register the three-dimensional point cloud of the head and the three-dimensional model of the brain; precisely registering the three-dimensional point cloud of the head and the three-dimensional model of the brain based on an iterative closest point algorithm.

5. The visual positioning based TMS robot navigation control method of claim 1, wherein, The motion trajectory of the treatment coil is generated according to the three-dimensional model of the head after pose correction and the target stimulation target point, comprising: obtaining theoretical coordinates of the target stimulation target point in a model coordinate system of the three-dimensional model of the head, and converting the theoretical coordinates in the model coordinate system to a robot reference coordinate system to establish a treatment point coordinate; establishing a target pose of the treatment coil in the robot reference coordinate system according to stimulation direction requirements of the target stimulation target point; generating a motion trajectory of the treatment coil according to the treatment point coordinate and the target pose.

6. The visual positioning based TMS robot navigation control method of claim 5, wherein, The motion trajectory of the treatment coil is generated according to the three-dimensional model of the head after pose correction and the target stimulation target point, further comprising: obtaining a model pose transformation amount at an adjacent sampling time; updating the treatment point coordinate when the model pose transformation amount exceeds a preset tolerance threshold.

7. The visual positioning based TMS robot navigation control method of claim 1, wherein, The pose of the three-dimensional model of the head is corrected according to the positioning tracking data, comprising: determining a real-time pose of the head of the subject according to the positioning tracking data; converting the real-time pose of the head from a tool coordinate system to a model coordinate system to obtain a model real-time pose; According to the model, the head three-dimensional model is corrected in real time.

8. The visual positioning based TMS robot navigation control method according to any one of claims 1-7, wherein, Further comprising: Obtaining a kinematics model and a dynamics model of the mechanical arm, and optimizing the motion trajectory according to the kinematics model and the dynamics model.

9. The visual positioning based TMS robot navigation control method according to any one of claims 1-7, wherein, Further comprising: Obtaining diagnosis and treatment feedback data in the diagnosis and treatment process, and adjusting diagnosis and treatment parameters according to the diagnosis and treatment feedback data.

10. The visual positioning based TMS robot navigation control method according to any one of claims 1-7, wherein, Before the visual detection data, the magnetic resonance image data and the positioning tracking data of the head of the diagnosis and treatment object are obtained, the method further comprises: Calibrating the binocular camera to obtain the intrinsic matrix, the distortion coefficient of each camera and the extrinsic matrix between the two cameras; and, Calibrating the head tracking tool fixed on the head of the diagnosis and treatment object to obtain the spatial relationship between the head model and the head tracking tool.

Images