Control device, system and device for transcranial magnetic stimulation device

Abstract

The present invention discloses a control device, system and equipment for a transcranial magnetic stimulation device, relating to the field of medical devices, including a memory and a processor, wherein the processor first obtains a depth image and an RGB image of a user's head through a first depth camera and a second depth camera, and the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head, which can ensure the comprehensiveness of the images captured by the depth camera, and then obtains a target point cloud map of the target mark located on the user's head based on the depth image and the RGB image. Finally, based on the target point cloud map, the treatment head on the transcranial magnetic stimulation device is controlled to move to the position of the target mark. No additional preparation operations are required and the method of use is simple.

Description

Technical Field

[0001] The present invention relates to the field of medical devices, and in particular to a control device, system and equipment for transcranial magnetic stimulation equipment. Background Art

[0002] Transcranial magnetic stimulation (TMS) is a technology that uses pulsed magnetic fields to induce currents in localized areas of the cerebral cortex, temporarily stimulating or inhibiting brain function. Existing TMS treatment devices require reconstructing a three-dimensional image of the user's head from a CT (Computed Tomography) scan. This 3D image is then used for target location. However, this method requires the user to undergo an MRI before 3D reconstruction can be performed based on the MRI image, resulting in lengthy preparation and cumbersome operation. Summary of the Invention

[0003] The purpose of the present invention is to provide a control device, system and device for a transcranial magnetic stimulation device, which does not require additional preparation operations before using the transcranial magnetic stimulation device and is simple to use.

[0004] To solve the above technical problems, the present invention provides a control device for a transcranial magnetic stimulation device, comprising:

[0005] memory for storing computer programs;

[0006] The processor is configured to implement the following steps when executing the computer program:

[0007] Acquire a depth image and an RGB image captured by a first depth camera and a second depth camera, wherein the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head;

[0008] Obtaining a target point cloud image of a target mark located on the user's head based on the depth image and the RGB image;

[0009] Based on the target point cloud map, the treatment head on the transcranial magnetic stimulation device is controlled to move to the position of the target mark.

[0010] Preferably, before obtaining a target point cloud map of a target mark located on the user's head based on the depth image and the RGB image, the method further includes:

[0011] Determining whether the RGB image contains an image corresponding to the user's head;

[0012] If so, the step of obtaining a target point cloud map of the target mark located on the user's head based on the depth image and the RGB image is entered.

[0013] Preferably, obtaining a target point cloud map of a target mark located on the user's head based on the depth image and the RGB image includes:

[0014] Performing three-dimensional reconstruction on the depth image and the RGB image to obtain a head point cloud image corresponding to the user's head;

[0015] A target point cloud image corresponding to the target mark in the head point cloud image is extracted.

[0016] Preferably, before extracting the target point cloud image corresponding to the target mark in the head point cloud image, the method further includes:

[0017] Filtering the head point cloud image, and performing image binarization processing on the filtered head point cloud image;

[0018] Performing edge detection and morphological image processing on the current head point cloud image to obtain the original target point cloud image corresponding to the target mark;

[0019] When the original target point cloud image is deformed, the original target point cloud image is corrected.

[0020] Preferably, controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud image includes:

[0021] Obtaining a first target point coordinate of the target point mark in a depth camera coordinate system based on the target point cloud image;

[0022] Performing coordinate transformation on the first target point coordinates to obtain second target point coordinates of the target point mark in the robotic arm coordinate system of the transcranial magnetic stimulation device;

[0023] Determining the spatial posture of the target mark based on the target point cloud image;

[0024] The treatment head on the transcranial magnetic stimulation device is controlled to move to the second target coordinate, and the spatial posture of the treatment head is consistent with the spatial posture of the target mark.

[0025] Preferably, after controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes:

[0026] When the treatment head is started, obtaining the actual treatment time of the treatment head;

[0027] When the actual treatment time reaches a preset treatment time threshold, the treatment head is controlled to stop working.

[0028] Preferably, after controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes:

[0029] obtaining the contact force between the treatment head and the user's head through a force sensor provided on the treatment head;

[0030] When the contact force is greater than a preset contact force threshold, the mechanical arm that moves the treatment head is controlled to stop moving.

[0031] Preferably, after controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes:

[0032] Determining whether the user's head is close to or far from the treatment head;

[0033] If the user's head is close to the treatment head, controlling the treatment head to move away from the user's head;

[0034] If the user's head is away from the treatment head, the step of acquiring the depth image and the RGB image captured by the first depth camera and the second depth camera is entered, so that the treatment head is moved back to the user's head.

[0035] In order to solve the above technical problems, the present invention also provides a control system for a transcranial magnetic stimulation device, comprising:

[0036] An acquisition unit, configured to acquire a depth image and an RGB image captured by a first depth camera and a second depth camera, wherein the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head;

[0037] a target point cloud map determining unit, configured to obtain a target point cloud map of a target mark located on the user's head based on the depth image and the RGB image;

[0038] A moving unit is used to control the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map.

[0039] In order to solve the above technical problems, the present invention also provides a transcranial magnetic stimulation device, including the control device of the above transcranial magnetic stimulation device, and also including a first depth camera and a second depth camera.

[0040] The present invention provides a control device, system and equipment for a transcranial magnetic stimulation device, including a memory and a processor, wherein the processor first obtains a depth image and an RGB image of a user's head through a first depth camera and a second depth camera, and the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head, which can ensure the comprehensiveness of the images captured by the depth camera, and then obtains a target point cloud map of the target mark located on the user's head based on the depth image and the RGB image. Finally, based on the target point cloud map, the treatment head on the transcranial magnetic stimulation device is controlled to move to the position of the target mark. No additional preparation operations are required and the method of use is simple. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the prior art and the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A schematic structural diagram of a control device for a transcranial magnetic stimulation device provided by the present invention;

[0043] Figure 2 A control flow chart of a control device for a transcranial magnetic stimulation device provided by the present invention;

[0044] Figure 3 A target point cloud image processing flow chart of a control device for a transcranial magnetic stimulation device provided by the present invention;

[0045] Figure 4 A checkerboard model diagram for calibrating the intrinsic parameters of a depth camera in a control device of a transcranial magnetic stimulation device provided by the present invention;

[0046] Figure 5 A framework diagram of a mechanical arm coordinate system in a control device of a transcranial magnetic stimulation device provided by the present invention;

[0047] Figure 6 A control block diagram of a tracking controller in a control device of a transcranial magnetic stimulation device provided by the present invention;

[0048] Figure 7 A control block diagram of an end position controller in a control device of a transcranial magnetic stimulation device provided by the present invention;

[0049] Figure 8 This is a structural schematic diagram of a control system of a transcranial magnetic stimulation device provided by the present invention. DETAILED DESCRIPTION

[0050] The purpose of the present invention is to provide a control method and related components of a transcranial magnetic stimulation device, which does not require additional preparation operations before using the transcranial magnetic stimulation device and is simple to use.

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0052] Please refer to Figure 1 , Figure 1 This is a schematic structural diagram of a control device for a transcranial magnetic stimulation device provided by the present invention, wherein the control device for the transcranial magnetic stimulation device comprises:

[0053] Memory 11, for storing computer programs;

[0054] The processor 12 is configured to implement the following steps when executing the computer program:

[0055] S1. Acquire a depth image and an RGB image captured by a first depth camera and a second depth camera, wherein the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head;

[0056] S2. Obtaining a target point cloud map of the target mark located on the user's head based on the depth image and the RGB image;

[0057] S3. Control the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map.

[0058] Existing image-guided transcranial magnetic stimulation devices require the user to undergo a CT scan and obtain an MRI (Magnetic Resonance Imaging) image to perform a three-dimensional reconstruction of the head shape. This is then combined with facial recognition to locate the target marker. However, this method is cumbersome and requires the user to undergo an MRI scan in advance, which takes a long time to prepare. Furthermore, the head shape obtained through 3D MRI reconstruction deviates from the user's actual head shape, making it difficult to accurately position the treatment head.

[0059] In this application, a depth camera is first used to capture an image of an area including the user's head. The depth camera is a camera sensor that can simulate human vision to obtain a depth image and an RGB image of the captured scene. The depth image contains the distance data of each point in the depth camera's capture area relative to the camera origin. In addition, the present application is provided with a first depth camera and a second depth camera on the transcranial magnetic stimulation device, and the first depth camera and the second depth camera are arranged on the left and right sides above the user's head. Therefore, the first depth camera and the second depth camera can capture a picture including the shape of the user's head and ensure the comprehensiveness of the captured picture. The first depth camera and the second depth camera can also be arranged symmetrically above the patient's head, with the camera viewing angle being 45° diagonally downward.

[0060] In addition, before starting to obtain the depth image and RGB image captured by the first depth camera and the second depth camera, the first depth camera and the second depth camera can be calibrated with internal parameters and hand-eye calibration. The camera internal parameter calibration is used to calibrate the depth camera that captures the image, obtain the internal parameters of the depth camera, and ensure the accuracy of the image. The camera internal parameter calibration is mainly based on the chessboard calibration method used by OpenCV. First, fix the camera and prepare a chessboard template. Please refer to Figure 4 , Figure 4 The present invention provides a checkerboard model diagram for the calibration of the depth camera intrinsic parameters in the control method of the transcranial magnetic stimulation device. The checkerboard calibration plate is moved and transformed into different positions, and then photographed to obtain a series of two-dimensional images and three-dimensional images. During calibration, it is necessary to obtain the three-dimensional points used for calibration in the image and the corresponding two-dimensional point pairs on the image. On the black and white checkerboard, the two-dimensional image points can be easily found through corner detection. The checkerboard corner points are obtained through the corner detection algorithm cv2.findChessboardCorners in OpenCV, and then cv2.cornerSubPix is ​​used to obtain more accurate corner pixel coordinates. After obtaining the three-dimensional points used for calibration and the corresponding two-dimensional point pairs, cv2.calibrateCamera is used for calibration to obtain the calibration results, the camera's intrinsic parameter matrix, distortion coefficients, rotation matrix, and translation vector.

[0061] After obtaining the coordinates of the target point in the depth camera coordinate system through the visual algorithm, they need to be converted into coordinates that can be used by the robotic arm. Therefore, hand-eye calibration is required to obtain the transformation matrix. In this application, the depth camera is fixed next to the robotic arm used to move the treatment head and does not move with the robotic arm. It is an "eye outside the hand" situation. By obtaining the coordinates of several fixed points in the depth camera coordinate system and the robotic arm coordinate system, the transformation matrix is ​​obtained according to the coordinate system conversion formula.

[0062] In addition, in the present application, after obtaining the depth image and RGB image captured by the first depth camera and the second depth camera, the image captured by the first depth camera can be integrated with the image captured by the second depth camera for subsequent processing, and the present application does not specifically limit this.

[0063] Before performing transcranial magnetic stimulation on the user, the user can first wear a treatment cap. The treatment cap is made of rubber elastic material, which can effectively reduce the impact of hair while wrapping the head. Then, a target mark is posted on the treatment cap at the position corresponding to the target point on the user's head. The target mark can be a QR code or a colored sequin. When there are multiple targets on the user's head that need to be subjected to transcranial magnetic stimulation, a QR code can be selected as the target mark. Each QR code corresponds to a number. When performing transcranial magnetic treatment on the user, the corresponding number can be obtained by scanning the QR code information, and then the treatment head can be controlled to perform transcranial magnetic stimulation on the positions of each target mark in the preset number order.

[0064] The target mark can be a QR code made of cardboard or PCB material, with a size of 10mmx10mm, which meets the accuracy requirements of the depth camera without affecting the penetration of the magnetic field.

[0065] After acquiring the depth image and RGB image captured by the depth camera, a target point cloud map of the target mark located on the user's head is obtained based on the depth image and RGB image. Specifically, after acquiring the full-view image of the depth image and RGB image captured by the depth camera, a full-view point cloud map can be first obtained based on the full-view image, and then the target point cloud map can be extracted from the full-view point cloud map. The specific method for extracting the target point cloud map is not particularly limited in this application.

[0066] Finally, based on the target point cloud, the treatment head on the transcranial magnetic stimulation device is controlled to move to the target marker's location, so that the treatment head generates pulse stimulation that acts on the cerebral cortex corresponding to the target marker, temporarily stimulating or inhibiting the cerebral cortex. Since the user's head is three-dimensional, the treatment head is also three-dimensional. To ensure the effectiveness of transcranial magnetic stimulation, the treatment head's spatial posture after moving to the target marker can be controlled to be consistent with the spatial posture of the target marker to ensure the treatment head's therapeutic effect.

[0067] Furthermore, the treatment head can be moved to the target mark using a robotic arm. The robotic arm can be a six-axis or other type of robotic arm. Before the robotic arm moves, it can calculate the optimal motion trajectory based on the target mark to avoid collision with the bracket or the user's head. Initially, the robotic arm grips the treatment head and rapidly moves from its starting position to a safe point in the direction of the normal to the section plane from the target mark. It then decelerates and slowly moves to the target mark, ultimately achieving the goal of gripping the treatment head to the target mark.

[0068] In summary, the present invention provides a control device for a transcranial magnetic stimulation device. First, a depth image and an RGB image of the user's head are obtained through a first depth camera and a second depth camera, and the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head, which can ensure the comprehensiveness of the image captured by the depth camera. Then, a target point cloud map of the target mark located on the user's head is obtained based on the depth image and the RGB image. Finally, based on the target point cloud map, the treatment head on the transcranial magnetic stimulation device is controlled to move to the position of the target mark. No additional preparation operations are required and the method of use is simple. In addition, the present application also improves the accuracy and repeatability of target positioning for transcranial magnetic stimulation treatment, reduces the pressure on medical staff, and eliminates operational errors caused by handheld coils or fixed treatment coils during transcranial magnetic stimulation treatment.

[0069] Based on the above embodiment:

[0070] As a preferred embodiment, before obtaining a target point cloud image of a target mark located on the user's head based on the depth image and the RGB image, the method further includes:

[0071] S4, determining whether the RGB image contains an image corresponding to the user's head;

[0072] If so, go to S2.

[0073] Considering that the user's head may not be in the shooting area of ​​the depth camera or only part of the user's head may be in the shooting area of ​​the depth camera during actual use, it may be impossible to obtain the target point cloud map of the target mark in the above case.

[0074] Please refer to Figure 2 , Figure 2 This is a control flow chart for a control device of a transcranial magnetic stimulation device provided by the present invention. In this embodiment, before obtaining a target point cloud map of target markers located on the user's head based on the depth image and RGB image, the system first determines whether the RGB image captured by the depth camera contains an image corresponding to the user's head. If so, this proves that the depth camera can fully capture the image of the user's head, and the system proceeds to the subsequent step of obtaining a target point cloud map of target markers located on the user's head based on the depth image and RGB image. Therefore, these steps avoid delays in the treatment process caused by situations where the user's head is not within the depth camera's capturing area, or only partially within the depth camera's capturing area, thereby improving treatment efficiency.

[0075] As a preferred embodiment, a target point cloud image of a target mark located on the user's head is obtained based on the depth image and the RGB image, including:

[0076] Perform 3D reconstruction on the depth image and RGB image to obtain a head point cloud image corresponding to the user's head;

[0077] Extract the target point cloud corresponding to the target mark in the head point cloud.

[0078] In this embodiment, since the depth image contains the distance data of each point in the depth camera shooting area relative to the camera origin, in order to obtain the target point cloud map of the target mark, the depth image and the RGB image can be first three-dimensionally reconstructed. The three-dimensional reconstructed image contains the stereoscopic graphics of all objects in the full viewing angle range captured by the depth camera, that is, the head point cloud map corresponding to the user's head can be obtained, and then the target point cloud map corresponding to the target mark located on the user's head is extracted based on the head point cloud map, so as to determine the actual position of the target mark and then control the treatment head to move to the position of the target mark.

[0079] As a preferred embodiment, before extracting the target point cloud image corresponding to the target mark in the head point cloud image, the method further includes:

[0080] Filter the head point cloud image and perform image binarization processing on the filtered head point cloud image;

[0081] Perform edge detection and morphological image processing on the current head point cloud image to obtain the original target point cloud image corresponding to the target mark;

[0082] When the original target point cloud image is deformed, the original target point cloud image is corrected.

[0083] Considering that there may be factors in the depth image and RGB image taken by the depth camera that interfere with the positioning of the target marker, such as noise in the image or image deformation.

[0084] Please refer to Figure 3 , Figure 3 This is a target point cloud image processing flow chart of a control device of a transcranial magnetic stimulation device provided by the present invention.

[0085] To address the above technical issues, in this embodiment, after obtaining the head point cloud image, the head point cloud image is first filtered and then binarized to prevent noise from affecting the target point cloud image. The filtering can be implemented by selecting a suitable sliding window that slides pixel by pixel along the rows of the head point cloud image. During each slide, all pixels in the window are sorted by grayscale value, and the median value of the data is used as the output to replace the grayscale value of the original center pixel of the window.

[0086] Image binarization facilitates further image processing, simplifies the image, reduces the amount of data, and highlights the outline of the target area. When implementing image binarization, the maximum inter-class variance value can be adaptively selected as the threshold for image binarization, which is not specifically limited in this application.

[0087] After filtering and binarizing the head point cloud image, edge detection and morphological image processing are performed on the current head point cloud image to initially locate the target marker and obtain the original target point cloud image corresponding to the target marker. Specifically, if the target marker is in the form of a QR code, since the edge of the QR code is a collection of pixels with a sudden change in image grayscale value, the first-order and second-order derivatives of the image grayscale value at the edge will show significant changes. Therefore, the edge position of the image can be determined by using this change. Edge detection is then performed using the Laplacian of Gaussian operator to achieve preliminary positioning of the target marker.

[0088] Morphological image processing of the head point cloud can further eliminate the influence of interference factors on the extraction of the target point cloud. For example, when the target mark is a QR code, the morphological algorithm can eliminate the interference points inside the QR code, identify the connected areas in the QR code image, and then achieve preliminary positioning of the target mark in the form of a QR code.

[0089] In addition, considering that the target marker is placed on the user's head and does not necessarily face the shooting angle of the first depth camera and the second depth camera, deformation may occur during the above-mentioned edge detection and morphological image processing, which may affect the subsequent extraction of the target point cloud image. Therefore, it is necessary to perform deformation detection and correction on the original target point cloud image. Specifically, the original target point cloud image can be processed using a line detection module and a correction module. Taking the case where the target marker is a QR code as an example, the line detection module uses Hough transform to detect the lines of the QR code, and the correction module is used to correct the rotation of the QR code. Suppose the equation of any straight line in the rectangular coordinate system is: Suppose the equation of a straight line l in the rectangular coordinate system is: y = kx + b, then the parametric equation of the straight line l can be expressed as: ρ = xcosθ + ysinθ, where ρ represents the distance from the origin to the straight line, and θ represents the angle between the straight line and the x-axis. Count the number of curves passing through each intersection in the polar coordinate system, and find the point with the largest number, which corresponds to a certain straight line; assuming that the pixel coordinates of the QR code before rotation are (x0, y0), expressed in polar coordinates as (rcosa, rsina), and the pixel coordinates after rotation by angle θ are (x1, y1), and the polar coordinates after rotation are expressed as (rcos(a-θ), rsin(a-θ)), then:

[0090]

[0091]

[0092] The rotation correction of the QR code can be achieved through the above method.

[0093] To sum up, in this embodiment, by filtering the head point cloud image, image binarization, edge detection, morphological image processing, and correcting the original target point cloud image when the original target point cloud image is deformed, the accuracy of extracting the target point cloud image and the accuracy of subsequent control of the treatment head to move to the target mark are further guaranteed, thereby ensuring the accuracy of transcranial magnetic therapy.

[0094] As a preferred embodiment, controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud image includes:

[0095] Obtain the first target coordinates of the target mark in the depth camera coordinate system based on the target point cloud image;

[0096] Performing coordinate conversion on the first target point coordinates to obtain second target point coordinates of the target point marker in the robotic arm coordinate system of the transcranial magnetic stimulation device;

[0097] Determine the spatial posture of the target marker based on the target point cloud map;

[0098] The treatment head on the transcranial magnetic stimulation device is controlled to move to the second target coordinate, and the spatial posture of the treatment head is consistent with the spatial posture of the target mark.

[0099] In this embodiment, in order to control the movement of the treatment head to the position of the target mark, the first target coordinates of the target mark in the depth camera coordinate system are first obtained based on the target point cloud map. The depth image taken by the depth camera contains the distance data of each point in the camera field of view relative to the camera origin, but to control the movement of the robotic arm to the distance between the position of the target mark, it is necessary to know the distance of the target mark relative to the robotic arm origin. Therefore, after obtaining the first target coordinates, the first target coordinates need to be converted into a coordinate system to obtain the second target coordinates of the target mark in the robotic arm coordinate system of the transcranial magnetic stimulation device, that is, the actual position of the target mark in space is obtained.

[0100] Furthermore, considering that the target marker is a three-dimensional shape when placed on the user's head, and the treatment head is also three-dimensional, to ensure treatment effectiveness, the spatial posture of the treatment head at the target marker must be consistent with the spatial posture of the target marker itself. Therefore, in this application, the spatial posture of the target marker is also obtained based on the target point cloud image.

[0101] In addition, the spatial posture of the target marker can be calculated by utilizing the similar properties of triangles. Taking the target marker as a QR code marker as an example, after obtaining the square QR code pattern, the position and depth data of these three pixel points in the corresponding depth image are obtained based on the three vertices of the QR code, and the 3D coordinates of the projection point in the depth camera coordinate system are solved, and the coordinates of the center point of the QR code are obtained to provide target positioning information for the collaborative robot arm.

[0102] At the same time, after obtaining the square QR code pattern, the target point cloud map is obtained according to the regional position of the pattern in the image. Then, the ICP (Iterative Closest Point) algorithm in the PCL (Point Cloud Library) is used to estimate the pose of the target point cloud map and the template QR code point cloud to obtain the pose transformation matrix, thereby determining the spatial pose of the treatment head at that position.

[0103] In summary, in this embodiment, the second target coordinates of the target marker in the robotic arm coordinate system and the spatial posture of the target marker are obtained, and then the treatment head is controlled to move to the second target coordinates, and the spatial posture of the treatment head at the target marker is consistent with the spatial posture of the target marker, further ensuring the accuracy of transcranial magnetic stimulation.

[0104] As a preferred embodiment, after controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes:

[0105] S5. When the treatment head is started, obtaining the actual treatment time of the treatment head;

[0106] When the actual treatment time reaches the preset treatment time threshold, the treatment head is controlled to stop working.

[0107] Please refer to Figure 2 , Figure 2 This is a control flow chart of a control device for a transcranial magnetic stimulation device provided by the present invention. In this embodiment, considering that each transcranial magnetic stimulation session generally lasts about 30 minutes, it is cumbersome for other personnel to patrol to see if the treatment duration has been reached. Therefore, in this application, after treatment is started, the actual treatment duration of the treatment head is obtained, and when the actual treatment duration reaches a preset treatment duration threshold, the treatment head is controlled to stop working, thereby achieving automatic control of the transcranial magnetic stimulation treatment duration, eliminating the need for manual patrols and achieving accurate control.

[0108] In addition, the actual treatment time of the treatment head can be obtained in real time or periodically, which is not specifically limited in this application. The preset treatment time threshold can be set according to actual conditions, which is not specifically limited in this application.

[0109] In addition, if the user has multiple treatment targets on his head, when the actual treatment time corresponding to one of the treatment targets reaches the preset treatment time threshold, transcranial magnetic stimulation can be performed on other treatment targets in a preset order.

[0110] As a preferred embodiment, after controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes:

[0111] S6. Obtaining the contact force between the treatment head and the user's head through a force sensor provided on the treatment head;

[0112] When the contact force is greater than a preset contact force threshold, the mechanical arm of the mobile treatment head is controlled to stop moving.

[0113] Please refer to Figure 2 , Figure 2 This is a control flow chart for the control device of a transcranial magnetic stimulation device provided by the present invention. Considering that the robotic arm or control device controlling the treatment head may malfunction, potentially causing harm to the user, in this embodiment, a force sensor provided on the treatment head detects the contact force between the treatment head and the user's head. When the contact force exceeds a preset contact force threshold, the robotic arm that moves the treatment head stops moving, thereby ensuring user safety.

[0114] In addition, when controlling the robotic arm to move the treatment head to the target mark position, it is necessary to obtain the conversion relationship between the robotic arm workspace and the joint space at the speed and displacement levels, and on this basis, perform gravity compensation on the end tool, and design the tracking controller and end position controller for the head tracking control mode and contact force adjustment mode respectively.

[0115] The kinematic analysis of the robot arm includes forward kinematics analysis, inverse kinematics analysis and Jacobian matrix analysis. Forward kinematics is to solve the position and posture of the end of the robot arm after the joint variables are known, that is, to determine the relationship between the robot arm mapping from the joint space to the task space. Forward kinematics is solved by the DH parameter method, please refer to Figure 5 , Figure 5 This is a framework diagram of the mechanical arm coordinate system in the control device of a transcranial magnetic stimulation device provided by the present invention. Figure 5 In the figure, (X0, Y0, Z0) through (X6, Y6, Z6) represent the six coordinate systems corresponding to the robotic arm. d1, a2, a3, d4, and d5 represent the distances between the origins of the coordinate systems where each axis joint resides, d represents the distance between the Z and Y axes, and a represents the distance between the X axes. Inverse kinematics solves for each joint angle given the known end-point pose. The relationship between the robot's workspace velocity and the joint-space velocity is determined by the Jacobian matrix. This matrix is ​​solved using the vector product method, establishing a transformation relationship between the robotic arm's end and each driven joint at the velocity level.

[0116] During treatment, the end of the robotic arm is connected to the treatment head. To accurately measure the contact force between the treatment head and the user's head, gravity compensation is required for the end tool to eliminate the influence of tool gravity on the force sensor measurements. The end tool primarily consists of an aluminum alloy fixture, the treatment head, and a cooling tube connected to the treatment head. A base coordinate system {0}, a sensor coordinate system {Sensor}, and a tool coordinate system {Tool} are established. The sensor coordinate system's X, Y, and Z axes are aligned with the end tool coordinate system, with the Z axis coinciding. The tool coordinate system's X, Y, and Z axes are aligned with the sensor coordinate system.

[0117] During the treatment process, the tracking controller uses the target posture of the end of the manipulator as input, and finally obtains the end velocity to achieve real-time tracking of the patient's head. Considering the safety of human-computer interaction during the tracking process, the manipulator will be controlled at the velocity level. In order to make the end of the manipulator move in the target direction, in each detection cycle, the current posture Xcurrent of the end of the manipulator and the target posture Xtarget of the end of the manipulator are subtracted, expressed in the form of axis angles, and the relationship between the PD parameter and the end velocity X is established. Please refer to Figure 6 , Figure 6 This is a control block diagram of a tracking controller in a control device of a transcranial magnetic stimulation device provided by the present invention. Figure 6 Here, Xtarget is the target position of the end of the manipulator calculated by the algorithm, and kp and kd are parameters in the motor control. kp is the proportional control parameter. Proportional control is the simplest control method. The output of its controller is proportional to the input error signal. When there is a proportional control system, there is a steady-state error in the output. Kd is the differential control parameter. In differential control, the output of the controller is proportional to the differential of the input error signal (i.e., the rate of change of the error). A controller with proportional plus differential can make the control action that suppresses the error equal to zero or even negative in advance, thereby avoiding serious overshoot of the controlled quantity. d / dt is the integral, and q is the terminal velocity.

[0118] In the contact force adjustment mode, safe contact force values ​​are set in the X, Y, and Z axes of the force sensor coordinate system so that the treatment coil can fit the head tightly and the coil can move in the opposite direction when the contact force is too large. The end position controller is designed for this mode. The contact force between the treatment coil and the head is detected in real time by the force sensor installed at the end of the robotic arm. The impedance control strategy is used to equate the interaction between the patient's head and the robotic arm to a spring-damper-mass model. Please refer to Figure 7 , Figure 7 This is a control block diagram of an end position controller in a control device of a transcranial magnetic stimulation device provided by the present invention. Figure 7Where Xcurrent is the real-time pose, which can be solved as the angle of each axis of the robot arm through the angle solver. q is the end motion speed, Fmeasure is the measured real-time contact force, Fcontact is the axial contact force after gravity compensation of the actual contact force, Ftarget is the contact force threshold set in the system, ΔF is the difference between the contact force and the threshold, ΔX is the pose difference after the force difference is solved by the reverse pose, and the real-time pose Xcurrent is compensated.

[0119] In addition, the preset contact force threshold can be set according to actual conditions, and the contact force between the treatment head and the user's head can be obtained periodically or in real time, which is not particularly limited in this application.

[0120] In addition, since each transcranial magnetic stimulation treatment lasts for more than 30 minutes, the weight of the treatment coil exceeds 3 kilograms. In this case, the existing technology of manually holding and using a static positioning system to fix the treatment coil is not able to continuously track the movement trajectory of the target point on the user's head, nor is it possible to compensate for the ineffective stimulation caused by "off-target" caused by involuntary movement of the user's head during use. Therefore, in order to ensure the effect of transcranial magnetic stimulation, it is necessary to control the movement of the treatment head accordingly when the user's head moves. For example, when the user's head is away from the treatment head, the treatment head is controlled to move back to the position of the new target mark, and when the user's head is too close to the treatment head, the treatment head is controlled to move away from the user's head.

[0121] As a preferred embodiment, after controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes:

[0122] Determine whether the user's head is close to or far from the treatment head;

[0123] If the user's head is close to the treatment head, control the treatment head to move away from the user's head;

[0124] If the user's head is away from the treatment head, the process proceeds to the step of acquiring the depth image and the RGB image captured by the first depth camera and the second depth camera, so that the treatment head can be moved back to the user's head.

[0125] In this embodiment, considering that each TMS session lasts for over 30 minutes, the treatment head weighs over 3 kg. In this case, the existing method of manually holding the treatment coil and using a static positioning system to fix the treatment coil cannot continuously track the motion trajectory of the user's head target point, nor can it compensate for ineffective stimulation caused by "off-target" movements caused by involuntary head movements during stimulation.

[0126] Therefore, in this embodiment, in order to ensure the effect of transcranial magnetic stimulation, it is necessary to control the movement of the treatment head in an adaptive manner when the user's head moves. Specifically, it is first determined whether the user's head is close to or far away from the treatment head. For example, considering that there may not be obvious displacement when the user's head is close to the treatment head, the torque applied to the treatment head can be detected by a torque sensor, and when the torque applied to the treatment head is greater than a preset threshold, it is determined that the user's head is close to the treatment head. For example, when the user's head is far away from the treatment head, it is possible to determine whether the user's head has moved based on the images captured by the first depth camera and the second depth camera, for example, using an inter-frame difference algorithm. Then, when the user's head is far away from the treatment head, the treatment head is controlled to move back to the position of the new target mark. When the user's head is too close to the treatment head, the treatment head is controlled to move away from the user's head, so as to achieve the purpose of the treatment head tracking the movement of the user's head in real time and ensure the therapeutic effect of the transcranial magnetic stimulation device.

[0127] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the structure of a control system of a transcranial magnetic stimulation device provided by the present invention, the control system comprising:

[0128] An acquisition unit 21 is configured to acquire a depth image and an RGB image captured by a first depth camera and a second depth camera, wherein the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head;

[0129] a target point cloud map determining unit 22, configured to obtain a target point cloud map of a target mark located on the user's head based on the depth image and the RGB image;

[0130] The moving unit 23 is used to control the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map.

[0131] For the relevant introduction of the control system of the transcranial magnetic stimulation device provided by the present invention, please refer to the embodiment of the control method of the above-mentioned transcranial magnetic stimulation device, which will not be repeated here.

[0132] The present invention also provides a transcranial magnetic stimulation device, comprising the control device of the above-mentioned transcranial magnetic stimulation device, and also comprising a first depth camera and a second depth camera.

[0133] For the relevant introduction of the transcranial magnetic stimulation device provided by the present invention, please refer to the embodiment of the control method of the above-mentioned transcranial magnetic stimulation device, which will not be repeated here.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0135] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A control device for a transcranial magnetic stimulation device, characterized in that: include: memory for storing computer programs; The processor is configured to implement the following steps when executing the computer program: Acquire a depth image and an RGB image captured by a first depth camera and a second depth camera, wherein the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head; Obtaining a target point cloud image of a target mark located on the user's head based on the depth image and the RGB image; Controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map; Obtaining a target point cloud map of a target mark located on the user's head based on the depth image and the RGB image, including: Performing three-dimensional reconstruction on the depth image and the RGB image to obtain a head point cloud image corresponding to the user's head; Extracting a target point cloud image corresponding to the target mark in the head point cloud image; Before extracting the target point cloud image corresponding to the target mark in the head point cloud image, the method further includes: Filtering the head point cloud image, and performing image binarization processing on the filtered head point cloud image; Performing edge detection and morphological image processing on the current head point cloud image to obtain the original target point cloud image corresponding to the target mark; When the original target point cloud image is deformed, correcting the original target point cloud image; Controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map includes: Obtaining a first target point coordinate of the target point mark in a depth camera coordinate system based on the target point cloud image; Performing coordinate transformation on the first target point coordinates to obtain second target point coordinates of the target point mark in the robotic arm coordinate system of the transcranial magnetic stimulation device; Determining the spatial posture of the target mark based on the target point cloud image; The treatment head on the transcranial magnetic stimulation device is controlled to move to the second target coordinate, and the spatial posture of the treatment head is consistent with the spatial posture of the target mark.

2. The control device of the transcranial magnetic stimulation device according to claim 1, characterized in that Before obtaining a target point cloud map of a target mark located on the user's head based on the depth image and the RGB image, the method further includes: Determining whether the RGB image contains an image corresponding to the user's head; If so, the step of obtaining a target point cloud map of the target mark located on the user's head based on the depth image and the RGB image is entered.

3. The control device of the transcranial magnetic stimulation device according to claim 1, wherein: After controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes: When the treatment head is started, obtaining the actual treatment time of the treatment head; When the actual treatment time reaches a preset treatment time threshold, the treatment head is controlled to stop working.

4. The control device of the transcranial magnetic stimulation device according to any one of claims 1 to 3, characterized in that: After controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes: obtaining the contact force between the treatment head and the user's head through a force sensor provided on the treatment head; When the contact force is greater than a preset contact force threshold, the mechanical arm that moves the treatment head is controlled to stop moving.

5. The control device of the transcranial magnetic stimulation device according to any one of claims 1 to 3, characterized in that: After controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map, the method further includes: Determining whether the user's head is close to or far from the treatment head; If the user's head is close to the treatment head, controlling the treatment head to move away from the user's head; If the user's head is away from the treatment head, the step of acquiring the depth image and the RGB image captured by the first depth camera and the second depth camera is entered, so that the treatment head is moved back to the user's head.

6. A control system for a transcranial magnetic stimulation device, characterized in that: include: An acquisition unit, configured to acquire a depth image and an RGB image captured by a first depth camera and a second depth camera, wherein the first depth camera and the second depth camera are respectively arranged on the left and right sides above the user's head; a target point cloud map determining unit, configured to obtain a target point cloud map of a target mark located on the user's head based on the depth image and the RGB image; a moving unit, configured to control the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map; Obtaining a target point cloud map of a target mark located on the user's head based on the depth image and the RGB image, including: Performing three-dimensional reconstruction on the depth image and the RGB image to obtain a head point cloud image corresponding to the user's head; Extracting a target point cloud image corresponding to the target mark in the head point cloud image; Before extracting the target point cloud image corresponding to the target mark in the head point cloud image, the method further includes: Filtering the head point cloud image, and performing image binarization processing on the filtered head point cloud image; Performing edge detection and morphological image processing on the current head point cloud image to obtain the original target point cloud image corresponding to the target mark; When the original target point cloud image is deformed, correcting the original target point cloud image; Controlling the treatment head on the transcranial magnetic stimulation device to move to the position of the target mark based on the target point cloud map includes: Obtaining a first target point coordinate of the target point mark in a depth camera coordinate system based on the target point cloud image; Performing coordinate transformation on the first target point coordinates to obtain second target point coordinates of the target point mark in the robotic arm coordinate system of the transcranial magnetic stimulation device; Determining the spatial posture of the target mark based on the target point cloud image; The treatment head on the transcranial magnetic stimulation device is controlled to move to the second target coordinate, and the spatial posture of the treatment head is consistent with the spatial posture of the target mark.

7. A transcranial magnetic stimulation device, characterized in that A control device comprising the transcranial magnetic stimulation device according to any one of claims 1 to 5, further comprising a first depth camera and a second depth camera.

Images