Overall process transcranial magnetic stimulation data recording and collecting system and method based on anonymity
The transcranial magnetic stimulation data recording system, which uses real-time acquisition and anonymization, solves the problems of inaccurate stimulation location and inefficient data management, and achieves precise visual monitoring and efficient data sharing, supporting large-sample studies.
Patent Information
- Application Number
- CN202411105757.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-13
AI Technical Summary
Existing transcranial magnetic stimulation (TMS) techniques suffer from problems such as inaccurate stimulation location, unclear stimulation effects, lack of full-process monitoring, and low data management efficiency, making it difficult to achieve efficient data sharing and large-sample studies.
An anonymous transcranial magnetic stimulation (TMS) data recording and acquisition system is adopted. RGB-D image data is acquired in real time through facial and coil depth cameras. Combined with magnetic resonance data, three-dimensional modeling and anonymization processing are performed to achieve precise visualization monitoring of stimulation points and secure data sharing.
It enables real-time monitoring of the transcranial magnetic stimulation process, precise presentation of stimulation points, and full-process retrospective analysis, protecting the privacy of subjects while improving data sharing efficiency and supporting large-sample data analysis and stimulation effect evaluation.
Smart Images

Figure CN118919010A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of transcranial magnetic stimulation medical imaging, and in particular relates to a system and method for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity. Background Art
[0002] Transcranial Magnetic Stimulation (TMS) is a non-invasive neuroscience research technology that can change the membrane potential of the cerebral cortex nerve cells in a specific brain area by generating a magnetic field, causing an induced current to affect brain metabolism and neural electrical activity, thereby causing a series of physiological and biochemical reactions. TMS is painless, non-destructive, easy to operate, safe and reliable, and has broad significance in clinical medicine and neuroscience research.
[0003] However, in the clinical application and research of TMS, there are problems such as inaccurate stimulation position and unclear stimulation effect during the stimulation process. The lack of full monitoring of the stimulation process is also a major challenge. Traditional TMS equipment and data acquisition methods have the following shortcomings: First, users cannot obtain the precise position information of the stimulation point during each pulse stimulation during the stimulation process; second, users cannot have an intuitive understanding and quantitative research on the changes in the stimulation point position caused by the subject's head movement or the change in the coil position; third, users cannot trace back the experimental process afterwards.
[0004] On the other hand, the large amount of data generated by TMS needs to be effectively managed, analyzed and protected. As a result, it is impossible to further study and summarize the effects of TMS stimulation. At present, the target selection and stimulation effects are also studied due to the lack of large sample data. The current common data processing methods are difficult to achieve effective protection of data while meeting the needs of efficient sharing.
[0005] In addition, with the rapid development of big data and artificial intelligence technology in recent years, how to apply these advanced technologies to TMS research, improve the efficiency of data utilization and promote retrospective analysis of TMS research to improve the therapeutic effect of TMS has also become an urgent problem to be solved in this field. Therefore, there is an urgent need for a transcranial magnetic stimulation data acquisition, analysis and sharing system that can perform precise and visual stimulation while achieving efficient and secure data processing, analysis and sharing. Summary of the invention
[0006] In order to solve the problems existing in the background technology, the purpose of the present invention is to provide an anonymous full-process transcranial magnetic stimulation data recording and acquisition system and method, which can provide accurate, visual, and full-process monitoring for transcranial magnetic stimulation users, and conduct quantitative research on stimulation differences. In addition, the system can solve the problem of unclear individual stimulation effects of transcranial magnetic stimulation based on big data and sharing system.
[0007] The technical solution adopted by the present invention is as follows:
[0008] Before the stimulation occurs, the user enters the subject and the test information, which mainly includes the following:
[0009] Project name;
[0010] Information about the data source organization;
[0011] Subject information: subject number, age, gender, education level, and symptoms;
[0012] Subject information: brand and model of transcranial magnetic device, test time, stimulation location, stimulation intensity and stimulation mode;
[0013] Step S1, before stimulation occurs, the magnetic resonance data of the subject is input into the data processing terminal, and the data processing terminal establishes a complete three-dimensional structural model of the brain according to the magnetic resonance data, and establishes a complete three-dimensional model of the stimulation coil by measuring the actual physical size of the stimulation coil and setting the target point at the same time. The target point is the position point inside the brain of the subject where the stimulation coil is focused, and then the stimulation coil is placed close to the head surface of the subject;
[0014] In step S1, the data processing terminal imports the magnetic resonance data and reconstructs the point cloud and mesh model of the subject's head skin surface, the gray matter point cloud and mesh model inside the brain; before the stimulation begins, the operator can set and save the stimulation target position and coil posture; the operator can also select the built-in settings of the system of the present invention or import existing settings. After the stimulation parameters are input or selected, the system visualizes the target position and coil posture.
[0015] The coil position and angle can be determined in two ways:
[0016] Method 1: The operator sets the coordinates of the target point, and the system obtains the optimal coil position and angle based on the target point coordinates, and then uses the optimal coil position and angle to set the stimulation coil;
[0017] Method 2: The operator selects any position to freely set the position and angle of the stimulation coil.
[0018] Step S2: When stimulation occurs, the following work is performed in real time: the transcranial magnetic stimulator sends a pulse trigger signal to the data processing terminal, and the data processing terminal performs anonymization processing based on the received original RGB-D image data to obtain anonymous RGB-D image data:
[0019] Step S2.1, the transcranial magnetic stimulator sends a pulse excitation signal to the stimulation coil when a stimulation pulse occurs, so that the stimulation coil applies a pulse magnetic field to the subject's brain, and at the same time sends a pulse trigger signal with the same frequency as the pulse excitation signal to the data processing terminal;
[0020] Step S2.2: Next, the facial depth camera and the coil depth camera are used to collect the original RGB-D image data in real time, and the original RGB-D image data (i.e., RGB-depth image data) is transmitted to the data processing terminal;
[0021] The RGB-D image data includes facial image data of the subject and coil image data of the stimulation coil, the facial image data of the subject includes a depth image and an RGB image of the subject's face, and the coil image data of the stimulation coil includes a depth image and an RGB image of the stimulation coil; the original facial image data and coil image data are respectively obtained by shooting at a camera frame rate by a facial depth camera and a coil depth camera.
[0022] Step S2.3: The data processing terminal anonymizes the received original RGB-D image data to obtain and save anonymous RGB-D image data.
[0023] Step S3: After the stimulation occurs, the anonymous RGB-D image data and magnetic resonance data are analyzed, processed and modeled to obtain the stimulation error of the target point for visualization and data security sharing.
[0024] Step S3.1, the data processing terminal extracts key frame image data from the anonymous RGB-D image data and saves it;
[0025] Step S3.2, the data processing terminal completes the three-dimensional modeling of the internal structure of the subject's brain and the stimulation coil according to the magnetic resonance data and the anonymous RGB-D image data to obtain the three-dimensional modeling data, and saves the three-dimensional modeling data;
[0026] Step S3.3, the data processing terminal simulates the focus of the magnetic field effect according to the three-dimensional modeling data, obtains the stimulation error between the target target point and the actual target point in the key frame image data, and visualizes the stimulation error;
[0027] Step S3.4: The data processing terminal classifies, stores and shares the original RGB-D image data, the anonymous RGB-D image data and the key frame image data.
[0028] The specific method of anonymizing the original RGB-D image data in step S2.3 is:
[0029] First, the facial features of the subject are extracted from the facial image data through the existing facial feature deep learning algorithm (such as the face recognition algorithm based on Dlib, etc.), and the RGB pixels of the subject's eye sockets and the RGB pixels below the lips are identified; then the eye socket data and the facial data below the lips in the facial image data are deleted through the corresponding point cloud relationship between the depth image and the RGB image of the subject's face, and the anonymous facial image data of the subject is obtained and saved.
[0030] The anonymous RGB-D image data includes the original coil image data and the anonymous facial image data, and the anonymous facial image data includes the anonymous depth image and RGB image of the subject's face. In subsequent data sharing, the user of the anonymous facial image data cannot identify the subject's identity through this part of the data, so the subject's privacy can be protected.
[0031] In step S3.1, the specific method of extracting key frame image data from anonymous RGB-D image data is as follows: extracting three frames of RGB-D images before and after the stimulation occurs from each frame of anonymous RGB-D image data as key frame image data; the key frame image data includes the depth image and RGB image of the stimulation coil, and the anonymous depth image and RGB image of the subject's face.
[0032] If the stimulation pulse frequency is greater than the camera frame rate, two frames of RGB-D images before and after the stimulation are extracted and saved as key frame image data.
[0033] The step S3.2 is specifically as follows:
[0034] Step S3.2.1, the data processing terminal detects the facial features of the subject according to the facial RGB image in the anonymous RGB-D image data, establishes the facial mask point cloud data of the subject through the depth data corresponding to the facial features in the facial depth image, deletes the facial feature data in the magnetic resonance data according to the facial mask point cloud data, and then uses the facial mask point cloud data to align, match and integrate with the magnetic resonance data with the facial feature data deleted to obtain the subject's internal brain structure model in the facial depth camera coordinate system, and the internal brain structure model does not display the subject's eye socket data and the facial data below the lips, so as to protect the subject's privacy;
[0035] Step S3.2.2, the data processing terminal detects the coil image area based on the RGB image of the stimulation coil, establishes a local three-dimensional model of the stimulation coil through the depth data corresponding to the coil image area in the depth image of the stimulation coil, and aligns, matches and integrates the local three-dimensional model of the stimulation coil with the complete three-dimensional model of the stimulation coil to obtain the spatial position information of the entire stimulation coil.
[0036] The step S3.3 is specifically as follows: the data processing terminal simulates the focus of the magnetic field effect based on the complete three-dimensional model and spatial position information of the stimulation coil, obtains the position of the actual target in the internal structure model of the brain as the position of the actual target inside the brain, and then uses the position error between the actual target and the target target in each key frame image as the stimulation error, and visualizes the stimulation error in the data processing terminal, and determines the stimulation effectiveness at the current moment of stimulation based on the real-time displayed stimulation error:
[0037] If the stimulation error is less than the preset error threshold, it means that the actual target position meets the requirements, and the stimulation at the moment of stimulation is determined to be effective stimulation;
[0038] Otherwise, it indicates that the actual target position does not meet the requirements, and the stimulation at the moment of occurrence is judged as invalid stimulation.
[0039] In step S3.4, the data processing terminal classifies, stores and shares various types of image data in the following manner:
[0040] First, the data processing terminal encrypts all stored image data and sets sharing permissions, wherein the image data includes original RGB-D image data, anonymous RGB-D image data, key frame image data, and the entire process screen after 3D modeling; the source user of the original RGB-D image data has the permission to view and download the original RGB-D image data, anonymous RGB-D image data, key frame RGB-D image data, the entire process screen of 3D modeling, and all stimulation errors of the stimulation process, and the other shared users have the permission to view and download anonymous RGB-D image data, key frame image data, the 3D modeling process screen, and the stimulation errors of the entire stimulation process.
[0041] During the acquisition process, the facial depth camera acquires and stores the depth data and RGB data of the subject's face in real time at the camera frame rate; during the acquisition process, the coil depth camera acquires and stores the depth data and RGB data of the stimulation coil in real time at the camera frame rate; the depth data / RGB data both contain the following data information: timestamp, stimulation pulse trigger signal and number of pulses sent.
[0042] During the test, the data processing terminal uses the first stimulation pulse trigger signal as the starting time, collects the depth data and RGB data of the face and coil in real time at the camera frame rate, and stores them according to the data specifications.
[0043] The system includes a transcranial magnetic stimulator, a facial depth camera, a coil depth camera, a data processing terminal and a stimulation coil; the transcranial magnetic stimulator is connected to the stimulation coil, and the transcranial magnetic stimulator generates a magnetic field for regulating brain activity through the stimulation coil, and simultaneously sends a pulse trigger signal to the data processing terminal; the facial depth camera faces the face of the subject and takes pictures, and is used to collect the facial image data of the subject in real time, and the facial image data includes the depth image data of the subject's face and the facial RGB image data; the coil depth camera faces the stimulation coil and takes pictures of the stimulation coil, and is used to collect the coil image data of the stimulation coil in real time, and the coil image data includes the depth image data and the RGB image data of the stimulation coil; the transcranial magnetic stimulator, the facial depth camera, and the coil depth camera are all connected to the data processing terminal, and the data processing terminal receives the magnetic resonance data of the subject, and the data processing terminal completes the stimulation data processing and sharing of the transcranial magnetic stimulation process based on the received magnetic resonance data, the pulse trigger signal, the facial image data and the coil image data;
[0044] The data processing terminal includes a playback module, which stores original RGB-D image data, anonymous RGB-D image data, key frame image data and three-dimensional modeling process screen data. The playback module is used to play back various types of stored image data at a preset speed.
[0045] The data processing terminal anonymizes the collected raw data to protect the privacy of the subjects and facilitate data sharing and further research. The anonymization method is to extract facial features through the RGB image using a deep learning algorithm, delete the eye socket data that identifies the subject, and the RGB pixels below the lips from the facial image data of the subject, and delete the point cloud facial data corresponding to the above facial features through the correspondence between the depth map and the RGB map, and obtain the anonymized facial image data of the subject and save it.
[0046] The data processing terminal completes the registration of magnetic resonance data with the anonymized facial feature point cloud data and performs 3D modeling of the brain and coil, providing the function of replaying the original data, anonymized data and 3D modeling at different speeds, and also provides the playback of key frames of different data. The playback function is set at different levels according to user permissions. The data source user can view the playback of the original data, and other users can view the playback of the anonymized data.
[0047] The data processing terminal completes the calculation and statistics of stimulation deviation: it simulates the magnetic field effect according to the coil structure, calculates the error between the actual stimulation point (actual target) and the target target in the key frame, and displays it visually.
[0048] The data processing terminal stores and shares the original data and processed data in accordance with the specifications. The data structure stored after each stimulation is:
[0049] The first level is index information, where users can query data based on key information, including subject symptoms, stimulation location, and data source agency;
[0050] The second level is the specific information of the subjects under the index information, including: subject number, gender, and age;
[0051] The third level is the stimulation information of each test, including: test time, original data (timestamp, RGB data, depth data, stimulation pulse trigger signal), anonymized data of the whole process (timestamp, RGB data, depth data, stimulation pulse trigger signal), 3D modeling whole process data (timestamp, point cloud data with RGB, stimulation pulse trigger signal), stimulation key frames, and stimulation pulse target error data of the whole process.
[0052] In terms of data sharing, the data processing terminal encrypts all stored data and sets permissions to provide data. It provides basic information of the data source user, original data, anonymized data, 3D modeling data and full-process stimulation errors. It also provides anonymized data, 3D modeling data and full-process stimulation errors to other shared users.
[0053] The system of the present invention realizes real-time monitoring during the stimulation process, accurate presentation of stimulation points, retrospective tracing of the entire process, and visual modeling of stimulation points inside the brain. It also provides the difference between actual stimulation and target stimulation and effectiveness evaluation, and shares the original data safely and effectively, which can provide assistance for subsequent stimulation effect research and optimization of stimulation schemes.
[0054] The system of the present invention supports the high-precision position information collection of the subject and the stimulation coil during the whole process of transcranial magnetic stimulation; and time-aligns these position data with the pulse data of transcranial magnetic stimulation; provides online visualization tools and offline retrospective analysis to optimize the stimulation effect; and proposes a data storage specification and anonymization sharing method to improve the efficiency and data security of large sample data storage during the whole process of transcranial magnetic stimulation. The present invention solves the problems of the existing transcranial magnetic stimulation in scientific research and clinical practice, such as the non-intuitive stimulation process, unclear stimulation effect, and lack of large sample data integration methods. It can accurately evaluate the effects of transcranial magnetic stimulation in different positions and postures, and realize the rapid and safe sharing of data at the same time, which has significant value for clinical and scientific research in the field of cognitive neuroscience.
[0055] Compared with the prior art, the present invention has the following beneficial effects:
[0056] 1. Combination of depth information and RGB information: This system collects depth information and RGB information of the face and coil surface at the same time, which can obtain more accurate and richer information than collecting only one type of information. For example, depth information can reflect the shape and position of an object, while RGB information can provide the color and texture of the object. The combination of the two can more accurately describe the state of the face and coil.
[0057] 2. Recommendation of coil position and angle for optimal stimulation effect: This system recommends the coil position and angle for optimal stimulation effect to the intended stimulation target.
[0058] 3. Data anonymization: This system has also made sufficient preparations to protect the privacy of subjects. The anonymization process is added in the data processing stage, which protects the privacy of subjects even in the subsequent data sharing and research process, and increases the reliability of the system.
[0059] 4. Key frame extraction technology: This system captures important information through key frames, effectively reducing the time and cost of data processing and research and analysis, while ensuring the validity of the data.
[0060] 5. Playback function: This system provides a variety of data playback functions to facilitate users to conduct post-analysis in different dimensions.
[0061] 6. Quantitative study of target points and actual targets: The effect of transcranial magnetic stimulation is related to the relative position of the coil head. This system can quantitatively study the relationship between position and stimulation effect.
[0062] 7. Data sharing: The system realizes data sharing under the premise of fully considering the privacy protection of subjects and data security protection, which is beneficial to the optimization of stimulus plans and research on stimulus effects within the industry. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] Figure 1 A schematic diagram of system composition provided for one embodiment of the present invention;
[0064] Figure 2 The present invention is a specific method flow chart of one embodiment of the present invention.
[0065] In the figure: 1- transcranial magnetic stimulator; 2- facial depth camera; 3- coil depth camera; 4- data processing terminal; 5- magnetic resonance data. DETAILED DESCRIPTION
[0066] Hereinafter, the technical scheme in the real-time example of the present invention will be clearly and completely described in conjunction with the example diagrams in the examples of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0067] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.
[0068] Take the example of a X institution using a transcranial magnetic stimulator Z to perform magnetic stimulation on the left DLPFC of a 30-year-old male subject Y with depression. The specific system implemented is as follows: Figure 1 As shown, Figure 1 As shown, the system includes a transcranial magnetic stimulator 1, a facial depth camera 2, a coil depth camera 3, a data processing terminal 4 and a stimulation coil; the transcranial magnetic stimulator 1 is connected to the stimulation coil, and the transcranial magnetic stimulator 1 generates a magnetic field for regulating brain activity through the stimulation coil, and simultaneously sends a pulse trigger signal to the data processing terminal 4; the facial depth camera 2 faces the face of the subject and takes pictures, and is used to collect the facial image data of the subject in real time, and the facial image data includes the depth image data and the facial RGB image data of the subject's face; the coil depth camera 3 faces the stimulation coil and takes pictures of the stimulation coil, and is used to collect the coil image data of the stimulation coil in real time, and the coil image data includes the depth image data and the RGB image data of the stimulation coil; the transcranial magnetic stimulator 1, the facial depth camera 2, and the coil depth camera 3 are all connected to the data processing terminal 4, and the data processing terminal 4 receives the magnetic resonance data 5 of the subject, and the data processing terminal 4 completes the stimulation data processing and sharing of the transcranial magnetic stimulation process based on the received magnetic resonance data 5, the pulse trigger signal, the facial image data and the coil image data.
[0069] The data processing terminal 4 includes a playback module, which stores original RGB-D image data, anonymous RGB-D image data, key frame image data and 3D modeling process screen data. The playback module is used to play back various types of stored image data at a preset speed.
[0070] The stimulation coil is controlled by the transcranial magnetic stimulator 1 and generates a magnetic field after being powered on. The magnetic field penetrates the skull to the human brain and stimulates the designated part to be stimulated. The data processing terminal 4 completes the data processing processes such as target visualization modeling, acquisition process time synchronization, face data anonymization processing, test process three-dimensional reconstruction, stimulation process playback, stimulation effect evaluation, data storage and data sharing based on the magnetic resonance data 5.
[0071] The magnetic resonance data 5 contains the spatial position coordinates of the entire brain. The facial depth camera 2 obtains the facial point cloud information of the subject, matches it with the facial information obtained from the magnetic resonance data 5, and can detect the displacement of the subject's head in real time. The facial depth camera 2 remains in a constant position during use and is always facing the subject's face. The coil depth camera 3 completes the physical model establishment of the stimulation coil, and also detects the position information of the stimulation coil in real time. The coil depth camera 3 remains in a constant position during use and is always facing the stimulated position of the subject. The transcranial magnetic stimulator 1 stimulates a specific area of the subject's head through a stimulation coil. The stimulation coil of the transcranial magnetic stimulator 1 has a variety of shapes depending on the purpose of use.
[0072] The data processing terminal 4 collects the following data based on unified specifications: full name of the institution, project name, subject information number, age, gender, education level, symptoms, etc., transcranial magnetic stimulator equipment information brand, model and stimulation setting stimulation position, stimulation intensity, and stimulation mode.
[0073] The facial depth camera 2 and the coil depth camera 3 are high-precision depth cameras that can simultaneously collect the depth data and RGB data of the face and the coil. During the test, the facial depth camera 2 is facing the subject's face, and the coil depth camera 3 is facing the coil. The data processing terminal 4 includes a playback module, which stores the original RGB-D image data, anonymous RGB-D image data, key frame image data, and three-dimensional modeling process screen data; the playback module is used to play back the stored various types of image data at a preset speed.
[0074] Different levels are set in the playback module according to user permissions. The original RGB-D image data source user can view the original RGB-D image data playback, and other users can view the anonymized data playback.
[0075] like Figure 2 As shown, the embodiments of the present invention and the implementation method thereof include:
[0076] S1. Before the experiment begins, the operator of institution X is requested to input basic information into the data processing terminal 4, including the name of institution X, the name of the project Z, the subject number Y, the subject's gender male, the subject's age 30 years old, the subject's education level undergraduate, the transcranial magnetic stimulator model Z, the stimulation location dorsolateral prefrontal cortex DLPFC, the stimulation intensity 80MT%, the stimulation pulse frequency 5Hz, the number of pulses per train 10, the number of trains 20, the time interval 8s, the total stimulation amount 600, the stimulation time A, and the stimulation plan is 5 times a week for 2 weeks.
[0077] S2. The operator inputs the subject's magnetic resonance data 5 into the data processing terminal 4. The data processing terminal 4 establishes a complete three-dimensional structural model of the brain according to the magnetic resonance data 5. The operator selects point F of DLPFC as the target point, obtains the position of the target point in the complete three-dimensional structural model of the brain as the position of the target point in the brain, and then places the stimulation coil according to the coil position and angle recommended by the system, and establishes a complete three-dimensional model of the stimulation coil by measuring the actual physical size of the stimulation coil;
[0078] S3. Before stimulation occurs, the work starts. The subject sits on the chair with his face facing the facial depth camera 2; the coil depth camera 3 is aimed at the stimulation coil and the test part, the camera frame rate is 20fps, and the two depth cameras 2 and 3 complete the coordinate unification at the data processing terminal 4.
[0079] After the stimulation starts, the transcranial magnetic stimulator 1 sends a trigger signal for each stimulation pulse to the data processing terminal 4. There are 600 trigger signals in total; with the first pulse trigger signal as the time starting point, the facial depth camera 2 and the coil depth camera 3 collect the original RGB-D image data in real time at the camera frame rate, and with the last trigger signal as the end point, the original RGB-D image data is transmitted to the data processing terminal 4 and stored;
[0080] The RGB-D image data includes facial image data of the subject and coil image data of the stimulation coil, the facial image data of the subject includes a depth image and an RGB image of the subject's face, and the coil image data of the stimulation coil includes a depth image and an RGB image of the stimulation coil.
[0081] Step S4: the data processing terminal 4 performs anonymization processing on the received original RGB-D image data to obtain and save anonymous RGB-D image data. The specific method of anonymization processing is as follows:
[0082] The facial features of the subject are extracted based on the deep learning algorithm, and the eye socket data and facial data below the lips of subject Y are deleted from the facial image data of the subject to obtain and save the anonymous facial image data of the subject.
[0083] Step S5: The data processing terminal 4 extracts key frame image data from the anonymous RGB-D image data and saves it. The specific method of extracting the key frame is as follows:
[0084] From each frame of anonymous RGB-D image data, three frames of RGB-D images before and after the stimulation are extracted as key frame image data; in this implementation case, the stimulation signal frequency is 5 Hz, and the system extracts 600 trigger signal moments and three frames of data before and after.
[0085] Step S6, the data processing terminal 4 completes the three-dimensional modeling of the internal structure of the subject's brain and the stimulation coil according to the magnetic resonance data 5 and the anonymous RGB-D image data, and saves the three-dimensional modeling picture;
[0086] Step S6 is specifically as follows:
[0087] Step S6.1, the data processing terminal 4 detects the facial features of the subject according to the facial RGB image in the anonymous RGB-D image data, establishes the facial mask point cloud data of the subject through the depth data corresponding to the facial features in the facial depth image, deletes the facial feature data in the magnetic resonance data 5 according to the facial mask point cloud data, and then uses the facial mask point cloud data to align, match and integrate with the magnetic resonance data 5 from which the facial feature data is deleted to obtain the internal brain structure model of the subject in the facial depth camera coordinate system, wherein the brain structure model does not display the orbital data of the subject and the facial data below the lips, thereby protecting the privacy of the subject;
[0088] Step S6.2, the data processing terminal 4 detects the coil image area based on the RGB image of the stimulation coil, establishes a local three-dimensional model of the stimulation coil through the depth data corresponding to the coil image area in the depth image of the stimulation coil, and aligns, matches and integrates the local three-dimensional model of the stimulation coil with the complete three-dimensional model to obtain the spatial position information of the entire stimulation coil.
[0089] The operator can view the original video, anonymized video, 3D modeling screen and key frame playback saved by himself and the previous times at different magnifications in the system. The operator can also view the anonymized video and key frame screen of other operators in the shared system according to different keywords.
[0090] Step S7, the data processing terminal 4 simulates the focus of the magnetic field effect based on the three-dimensional modeling and spatial position information of the stimulation coil, obtains the position of the actual target in the internal structure model of the brain (inside the brain), calculates the position error between the target target and the actual target in each key frame, and uses it as the stimulation error. The stimulation error is visualized in the data processing terminal 4, and the stimulation effect of this process is quantitatively evaluated according to the stimulation error:
[0091] If the stimulation error is less than the preset error threshold, it means that the actual target position meets the requirements, and the stimulation effect at the moment of stimulation is judged to be effective;
[0092] Otherwise, it indicates that the actual target position does not meet the requirements, and the stimulation effect at the moment of stimulation is judged to be invalid.
[0093] Step S8: The data processing terminal 4 classifies, stores and shares the original RGB-D image data, anonymous RGB-D image data, key frame image data, etc.
[0094] The data structure stored after each stimulation is:
[0095] The first level is index information, where users can query data based on key information, including subject symptoms, stimulation location, data source agency, and project name;
[0096] The second level is the specific information of the subjects under the index information, including: subject number, gender, age, and education level;
[0097] The third level is the stimulation information of each test, including: test time, original RGB-D image data (timestamp, RGB data, depth data, stimulation pulse trigger signal), anonymous RGB-D image data of the whole process (timestamp, RGB data, depth data, stimulation pulse trigger signal), 3D modeling whole process data (timestamp, point cloud data with RGB, stimulation pulse trigger signal), stimulation key frame image data, and stimulation pulse target error data of the whole process;
[0098] In terms of data sharing, all stored data are encrypted and set up with different permissions to provide data. Provide the user with basic information of this stimulation, original RGB-D image data, anonymous RGB-D image data, 3D modeling data and full-process stimulation errors; and provide the user with anonymous RGB-D image data, 3D modeling data and full-process stimulation errors of other researchers.
[0099] The embodiments of the present invention do not impose any form of limitation on this aspect. Any technician familiar with this profession can use the technical content disclosed above to make some more or modify them into equivalent embodiments with equivalent changes without departing from the scope of the technical solution of this aspect. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of this aspect without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of this aspect.
Claims
1. A system for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity. Features: The invention comprises a transcranial magnetic stimulator (1), a facial depth camera (2), a coil depth camera (3), a data processing terminal (4) and a stimulation coil; the transcranial magnetic stimulator (1) is connected to the stimulation coil, the transcranial magnetic stimulator (1) generates a magnetic field for regulating brain activity through the stimulation coil, and simultaneously sends a pulse trigger signal to the data processing terminal (4); the facial depth camera (2) faces the face of a subject and is used to collect facial image data of the subject in real time; the coil depth camera (3) faces the stimulation coil and is used to collect coil image data of the stimulation coil in real time; the transcranial magnetic stimulator (1), the facial depth camera (2) and the coil depth camera (3) are all connected to the data processing terminal (4), and the data processing terminal (4) receives magnetic resonance data (5) of the subject, and the data processing terminal (4) completes stimulation data processing and sharing of the transcranial magnetic stimulation process based on the received magnetic resonance data (5), the pulse trigger signal, the facial image data and the coil image data.
2. A system for recording and collecting data of transcranial magnetic stimulation based on anonymity throughout the entire process according to claim 1, Features: The data processing terminal (4) includes a playback module, in which original RGB-D image data, anonymous RGB-D image data, key frame image data and three-dimensional modeling process screen data are stored. The playback module is used to play back various types of stored image data at a preset speed.
3. A method for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity applied to the system described in any one of claims 1-2, It is characterized in that The following steps are involved: Step S1, before stimulation occurs, the subject's magnetic resonance data (5) is input into the data processing terminal (4), the data processing terminal (4) establishes a complete three-dimensional structural model of the brain according to the magnetic resonance data (5), measures the actual size of the stimulation coil, and establishes a complete three-dimensional model of the stimulation coil according to the measured actual size of the stimulation coil; at the same time, a target point is set, and then the stimulation coil is placed close to the head surface of the subject; Step S2: When stimulation occurs, the following operations are performed in real time: the transcranial magnetic stimulator (1) sends a pulse trigger signal to the data processing terminal (4), and the data processing terminal (4) performs anonymization processing based on the received original RGB-D image data to obtain anonymous RGB-D image data; Step S3: After the stimulation occurs, the anonymous RGB-D image data and magnetic resonance data (5) are combined for analysis, processing and modeling to obtain the stimulation error of the target point for visualization and data security sharing.
4. The method for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity according to claim 3, Features: The step S2 is specifically as follows: Step S2.1, the transcranial magnetic stimulator (1) sends a pulse excitation signal to the stimulation coil when a stimulation pulse occurs, and simultaneously sends a pulse trigger signal corresponding to the pulse excitation signal to the data processing terminal (4); Step S2.2: Next, the facial depth camera (2) and the coil depth camera (3) are used to collect original RGB-D image data in real time, and the original RGB-D image data is transmitted to the data processing terminal (4); the RGB-D image data includes facial image data of the subject and coil image data of the stimulation coil; Step S2.3: The data processing terminal (4) anonymizes the received original RGB-D image data to obtain anonymous RGB-D image data and saves it.
5. The method for recording and collecting data of the entire transcranial magnetic stimulation based on anonymity according to claim 3, Features: The step S3 is specifically as follows: Step S3.1, the data processing terminal (4) extracts key frame image data from the anonymous RGB-D image data and saves it; Step S3.2, the data processing terminal (4) completes the three-dimensional modeling of the internal structure of the subject's brain and the stimulation coil according to the magnetic resonance data (5) and the anonymous RGB-D image data to obtain three-dimensional modeling data; Step S3.3, the data processing terminal (4) simulates the focus of the magnetic field effect according to the three-dimensional modeling data, obtains the stimulation error between the target target point and the actual target point in the key frame image data, and visualizes the stimulation error; Step S3.4: The data processing terminal (4) classifies, stores and shares the original RGB-D image data, the anonymous RGB-D image data and the key frame image data.
6. The method for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity according to claim 4, Features: The specific method of anonymizing the original RGB-D image data in step S2.3 is: Firstly, the facial features of the subject are extracted from the facial image data through the facial feature deep learning algorithm, and the RGB pixels of the subject's eye sockets and the RGB pixels below the lips are identified; then, the eye socket data and the facial data below the lips in the facial image data are deleted through the corresponding point cloud relationship between the depth image and the RGB image of the subject's face, and the anonymous facial image data of the subject is obtained and saved.
7. The method for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity according to claim 5, Features: In step S3.1, the specific method of extracting key frame image data from anonymous RGB-D image data is as follows: extracting three frames of RGB-D images before and after the stimulation occurs from each frame of anonymous RGB-D image data as key frame image data; the key frame image data includes the depth image and RGB image of the stimulation coil, and the anonymous depth image and RGB image of the subject's face.
8. The method for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity according to claim 5, Features: The step S3.2 is specifically as follows: Step S3.2.1, the data processing terminal (4) detects facial features of the subject according to the facial RGB image in the anonymous RGB-D image data, establishes facial mask point cloud data of the subject through the depth data corresponding to the facial features in the facial depth image, deletes the facial feature data in the magnetic resonance data (5) according to the facial mask point cloud data, and then uses the facial mask point cloud data and the magnetic resonance data (5) with the facial feature data deleted to align, match and integrate to obtain the internal brain structure model of the subject in the facial depth camera coordinate system; Step S3.2.2, the data processing terminal (4) detects the coil image area based on the RGB image of the stimulation coil, establishes a local three-dimensional model of the stimulation coil through the depth data corresponding to the coil image area in the depth image of the stimulation coil, and aligns, matches and integrates the local three-dimensional model of the stimulation coil with the complete three-dimensional model of the stimulation coil to obtain the spatial position information of the entire stimulation coil.
9. The method for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity according to claim 5, Features: The step S3.3 is specifically as follows: the data processing terminal (4) simulates the focus of the magnetic field effect based on the complete three-dimensional model and spatial position information of the stimulation coil, obtains the position of the actual target in the internal structure model of the brain, and then uses the position error between the actual target and the target target in each key frame image as the stimulation error, and visualizes the stimulation error in the data processing terminal (4), and determines the stimulation effectiveness at the current stimulation moment based on the stimulation error displayed in real time: If the stimulation error is less than the preset error threshold, it means that the actual target position meets the requirements, and the stimulation at the moment of stimulation is determined to be effective stimulation; Otherwise, it indicates that the actual target position does not meet the requirements, and the stimulation at the moment of occurrence is judged as invalid stimulation.
10. The method for recording and collecting data of the entire transcranial magnetic stimulation process based on anonymity according to claim 5, Features: In step S3.4, the data processing terminal (4) classifies and stores various types of image data and shares them in the following manner: First, the data processing terminal (4) encrypts all stored image data and sets sharing permissions, wherein the image data includes original RGB-D image data, anonymous RGB-D image data, key frame image data, and a full-process picture after three-dimensional modeling; the source user of the original RGB-D image data has permissions to view and download the original RGB-D image data, anonymous RGB-D image data, key frame image data, a full-process picture of three-dimensional modeling, and all stimulation errors of the stimulation process, and other shared users have permissions to view and download anonymous RGB-D image data, key frame image data, a full-process picture of three-dimensional modeling, and stimulation errors of the full-process stimulation.
Citation Information
Patent Citations
TMS (Transcranial Magnetic Stimulation) automatic navigation system and method based on depth camera and magnetic resonance
CN111729200A
Data processing method, device and system for video remote fast playback
CN113207040A
Transcranial magnetoacoustic stimulation and stimulation current monitoring device and method
CN114632265A
Magnetic stimulation device and method for promoting muscle strength enhancement based on magnetic field induction TRPC1
CN116650843A
Transcranial magnetic stimulation navigation positioning system and method
CN118319287A
