Multi-view vision system for brain magnetic automatic registration
Through the multi-eye vision system of multi-eye vision system, deep neural network and stereo vision technology are used, combined with least squares method and iterative close-point algorithm, the problem of insufficient registration accuracy of brain magneto-signal acquisition is solved, and efficient and accurate Magnetic brain signal collection, suitable for brain function research and clinical diagnosis.
Patent Information
- Application Number
- CN202411088666.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-09
AI Technical Summary
The existing magnetic brain signal acquisition technology has limitations in accuracy and efficiency, especially when dealing with complex head structures or movements, the registration accuracy is insufficient, resulting in the impact of data acquisition quality.
Using multi-eye vision systems, including rigid helmets and multi-eye vision camera systems, uses deep neural networks and stereo vision technology to identify face feature points, combined with least squares method and iterative closest point algorithm to realize helmet-to-head and head-to-MRI Accurate registration of images.
It significantly improves the accuracy and efficiency of brain magnetic signal acquisition, reduces artificial errors, ensures high quality and high reliability of data, and is suitable for brain function research and clinical diagnosis.
Smart Images

Figure CN119048594A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of biomedical engineering technology, and more specifically, to a multi-eye vision system for automatic brain magnetometry registration. Background Art
[0002] In the field of MEG signal acquisition, traditional technologies mainly rely on single-view visual systems or manual marker point registration, which have obvious limitations in accuracy and efficiency. Existing technologies often require complex and time-consuming manual correction processes to ensure accurate acquisition of MEG signals. In addition, due to the lack of synchronous information from multiple perspectives, the existing systems have insufficient registration accuracy when dealing with complex head structures or movements, which affects the quality of data acquisition.
[0003] Although the existing technology has achieved automatic registration of brain magnetic signals to a certain extent, in the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the existing technology: the instability of helmet positioning, the limitations of the multi-eye vision camera system in capturing facial feature points, and the error in the registration between the MRI image and the actual head structure. These problems or defects result in the registration accuracy of brain magnetic signals not meeting the requirements of high-precision research, limiting the wide application of magnetoencephalography technology in clinical and research fields. Summary of the invention
[0004] The present invention provides a multi-eye vision system for automatic brain magnetoencephalography registration, comprising:
[0005] A rigid helmet having a plurality of known coordinate markers at predetermined locations for providing initial sensor coordinates and assisting in helmet-head registration;
[0006] A multi-view camera system, arranged around the subject, is used to capture the feature points of the helmet and the subject's head and face;
[0007] The registration system is used to realize the registration from helmet to head based on helmet markers, and the registration from head to MRI based on facial feature points.
[0008] Furthermore, the coordinate markers of the rigid helmet provide accurate initial coordinate references for the multi-view camera system.
[0009] Furthermore, its multi-view camera system includes at least two high-resolution cameras for facial feature point extraction and three-dimensional reconstruction, and several other cameras for capturing marker points on the helmet.
[0010] Furthermore, its registration system uses the known coordinate marking points on the helmet to achieve accurate positioning of the helmet on the subject's head.
[0011] Furthermore, its registration system realizes the registration of head to MRI through the following steps:
[0012] Use deep neural networks to identify facial feature points, including the root of the nose, the tip of the nose, and the wings of the nose;
[0013] Identify MRI feature points, where the nose tip point is defined as the maximum value point on the Y axis, and locate other feature points based on this.
[0014] Furthermore, the three-dimensional reconstruction of its feature points includes:
[0015] Use stereo vision technology to match feature points captured by a multi-view camera system;
[0016] The triangulation method is used to calculate the positions of facial feature points in three-dimensional space.
[0017] Furthermore, the MRI registration steps include:
[0018] Preprocess and correct the posture of MRI images;
[0019] Based on the nose tip point, other feature points are determined to achieve registration between the MRI image and the face.
[0020] Furthermore, its registration algorithm based on marker points includes the least squares method and the ICP algorithm, and the specific steps are as follows:
[0021] Define a set of marked points {P i} and {Q i}, where {P i} is the coordinate of the mark point on the helmet, {Q i} are the coordinates of the corresponding points on the head;
[0022] Solve the rigid transformation matrix R and the translation vector t so that
[0023] Use the SVD algorithm to calculate the transformation matrix and vector.
[0024] Furthermore, the facial feature point-based registration algorithm includes the following steps:
[0025] Capture multiple perspective images of human faces through a multi-view camera system;
[0026] Use deep neural network to extract facial feature points;
[0027] Using the three-dimensional coordinates of the feature points, they are aligned with the feature points in the MRI image through rigid transformation;
[0028] The least squares method or ICP algorithm is used to calculate the rigid transformation matrix between the head and the MRI image to complete the registration.
[0029] Furthermore, the overall accuracy of its registration system is ensured by the three-dimensional reconstruction of the marker points and the reconstruction of the facial feature points of the multi-eye vision camera system, realizing the accurate collection and analysis of the magnetic brain signals. The positions of the marker points in the rigid helmet are precisely measured and calibrated, and their three-dimensional coordinates are known in the system, providing a stable reference system.
[0030] According to the above-mentioned embodiment of the present invention, at least the following beneficial effects are achieved: the multi-eye vision system can provide accurate references for the initial coordinates of the sensor and auxiliary helmet-head registration by using known coordinate markers on the rigid helmet and a multi-eye vision camera system. The system uses deep neural networks and stereo vision technology to efficiently identify and match facial feature points, and achieve accurate registration from helmet to head and head to MRI images. This registration method can not only improve the speed and convenience of registration, but also significantly improve the accuracy of brain magnetic signal acquisition. In addition, the least squares method and iterative closest point (ICP) algorithm adopted by the system can further optimize the registration process and ensure the three-dimensional reconstruction accuracy of the helmet and facial feature points. Through the application of these algorithms, the system can automatically adjust and optimize the registration parameters, reduce human errors, and thus ensure the high quality and reliability of the data during the acquisition and analysis of brain magnetic signals. This high-precision registration technology is of great significance for brain function research and clinical diagnosis, and can provide a powerful tool for researchers and doctors in related fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:
[0032] Figure 1 A schematic diagram of the structure of a multi-camera vision system for automatic magnetoencephalography registration provided by an embodiment of the present invention;
[0033] Figure 2 A schematic diagram of a multi-eye vision system provided by an embodiment of the present invention;
[0034] Figure 3 A top view of a fully automatic registration device according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION
[0035] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.
[0036] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.
[0037] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.
[0038] Example 1
[0039] Reference below Figure 1 , Figure 1 This is a schematic diagram of the structure of a multi-eye vision system for automatic brain magnetoencephalography registration provided by an embodiment of the present invention. Figure 1 As shown, a multi-eye vision system 100 for automatic brain magnetoencephalography registration includes:
[0040] A rigid helmet 101, on which a plurality of known coordinate marking points at predetermined positions are provided, for providing initial coordinates of the sensor and assisting in helmet-head registration;
[0041] A multi-eye vision camera system 102 is arranged around the subject to capture feature points of the helmet and the subject's head and face;
[0042] The registration system 103 is used to realize the registration from helmet to head based on helmet markers, and the registration from head to MRI based on facial feature points.
[0043] It should be noted that the core function of the multi-eye vision system for automatic brain magnetoencephalography registration is to achieve accurate registration between the helmet and the subject's head, and between the head and the MRI image. This involves not only the design of the physical equipment, but also the application of algorithms to ensure that the entire system can work efficiently and accurately.
[0044] Specifically, the system includes a rigid helmet with multiple known coordinate markers at predetermined locations. These markers are key because they provide an accurate initial coordinate reference for the multi-view camera system. The multi-view camera system consists of at least two high-resolution cameras that capture the feature points of the helmet and the subject's head and face, as well as several other cameras that capture the markers on the helmet. The registration system uses this information to align the helmet and the head, as well as the head with the MRI image, through a specific algorithm.
[0045] Preferably, the rigid helmet can be made of a lightweight and durable material to ensure the comfort of the wearer while ensuring the stability of the marking points. The settings of the multi-eye vision camera system can be adjusted according to actual needs. For example, the resolution, focal length and capture angle of the camera can be optimized according to the size and shape of the subject's head. The registration system can use advanced deep learning algorithms, such as convolutional neural networks (CNN), to improve the recognition accuracy of facial feature points. In addition, the registration algorithm can combine the least squares method and the iterative closest point (ICP) algorithm to achieve a higher accuracy registration effect. In practical applications, the algorithm parameters can be adjusted according to different research purposes or clinical needs to achieve the best registration effect.
[0046] In some embodiments, the coordinate markers of the rigid helmet provide an accurate initial coordinate reference for the multi-vision camera system.
[0047] It should be noted that the coordinate markers on the rigid helmet provide an accurate initial coordinate reference for the multi-view camera system, which emphasizes the importance of the coordinate markers on the helmet in the system alignment process. These markers not only provide an initial reference point for the camera system, but their position accuracy directly affects the accuracy of the entire alignment process.
[0048] Specifically, the coordinate markers on the rigid helmet can be markers with specific geometric shapes and sizes, such as circles or squares, whose positions are predetermined by precision measuring equipment and recorded in the system. These markers are usually evenly distributed on the helmet to ensure that they can be captured by the camera system from different angles. The multi-view camera system consists of multiple high-resolution cameras, which are precisely calibrated to ensure that the captured images are consistent with the coordinates of the markers on the helmet.
[0049] Preferably, the coordinate marking points on the helmet can be made of high-contrast colors or reflective materials to improve recognizability under different lighting conditions. The camera system can include multiple lenses with different focal lengths to adapt to shooting requirements at different distances and angles. In addition, the system can use automatic exposure and white balance control to ensure image quality. During implementation, machine vision algorithms can be used to quickly identify and locate the marking points, and then use these data as initial coordinates to provide an accurate starting point for subsequent alignment algorithms. As an alternative, active markers can be considered to improve recognition accuracy and robustness in complex environments.
[0050] In some embodiments, the multi-vision camera system includes at least two high-resolution cameras for facial feature point extraction and three-dimensional reconstruction, and several other cameras for capturing marker points on the helmet.
[0051] It should be noted that the multi-view camera system includes at least two high-resolution cameras for facial feature point extraction and 3D reconstruction, and several other cameras for capturing markers on the helmet, which illustrates the key components of the system and their functions. These camera systems are designed to ensure that images captured from different angles can provide sufficient information for accurate registration.
[0052] Specifically, at least two high-resolution cameras in the multi-view camera system refer to devices with sufficient pixels and resolution to capture clear images, which are used to extract key feature points of the subject's face, such as eyes, nose and mouth, which are essential for three-dimensional reconstruction. Several other cameras may include cameras with different focal lengths and viewing angles, which are specifically used to capture known coordinate markers on the helmet to assist in spatial positioning.
[0053] Preferably, these high-resolution cameras can be configured with autofocus and image stabilization to ensure high-quality images at different distances and conditions. The resolution and frame rate of the camera can be adjusted as needed to accommodate the capture of fast movements or small changes. In addition, the system can use a synchronized trigger mechanism to ensure that all cameras capture images at the same time to facilitate subsequent image matching and data processing. As an alternative, structured light or time-of-flight (ToF) cameras can be considered to enhance the accuracy and speed of 3D reconstruction.
[0054] In some embodiments, the registration system uses known coordinate markers on the helmet to accurately position the helmet on the subject's head.
[0055] It should be noted that the registration system uses the known coordinate markers on the helmet to achieve accurate positioning of the helmet on the subject's head. This article emphasizes how the registration system uses the markers on the helmet to ensure accurate alignment between the helmet and the subject's head. This precise positioning is the basis for subsequent head-to-MRI registration.
[0056] Specifically, the known coordinate markers in the registration system are coordinate points that are pre-determined and recorded in the system during the manufacturing process. These markers provide a stable reference frame for the system, allowing the registration algorithm to accurately determine the position of the helmet relative to the subject's head. This process usually requires a set of algorithms that can process the image data captured by the multi-view camera system and match it with the coordinates of the markers on the helmet.
[0057] Preferably, the registration system can use advanced image processing and pattern recognition techniques to automatically identify and track markers on the helmet. For example, a feature matching algorithm can be used to identify markers in the image and determine the position of the helmet by calculating the spatial relationship between the camera and the markers. In addition, the system can integrate a calibration module that allows the user to accurately calibrate the camera system before actual use to ensure that the correspondence between the captured image data and the known coordinate markers is as accurate as possible. As an alternative, machine learning algorithms can be considered to improve the accuracy and robustness of marker recognition, especially in the case of subject head movement or changing lighting conditions.
[0058] In some embodiments, the registration system achieves head-to-MRI registration by the following steps:
[0059] Use deep neural networks to identify facial feature points, including the root of the nose, the tip of the nose, and the wings of the nose;
[0060] Identify MRI feature points, where the nose tip point is defined as the maximum value point on the Y axis, and locate other feature points based on this.
[0061] It should be noted that the registration system achieves head-to-MRI registration through the following steps, which outlines how the registration system accurately aligns the subject's head with the MRI image. This process is critical to ensuring the accuracy of the magnetoencephalography data.
[0062] Specifically, the registration system first uses a deep neural network to identify facial feature points, including the root of the nose, the tip of the nose, and the wings of the nose. The identification of these feature points is key to achieving head-to-MRI registration because they provide reference points for the facial structure. Next, the system identifies MRI feature points, where the tip of the nose is defined as the maximum value point on the Y axis, which is used to locate other feature points to achieve alignment between images.
[0063] Preferably, the deep neural network can be a pre-trained model specifically designed to identify and locate facial feature points in an image. This model is based on a convolutional neural network (CNN) architecture and is capable of processing high-resolution images provided by a multi-view camera system. In addition, the system can use image registration algorithms, such as feature-based registration or model-based registration, to further refine the identification and location of feature points. During the identification of MRI feature points, the system can use automated image analysis tools to determine the Y-axis maximum point and adjust the positions of other feature points accordingly.
[0064] More specifically, the registration system can adopt a multi-stage deep learning process, including preprocessing, feature extraction, feature matching and fine-tuning. Preprocessing may include steps such as image denoising and contrast enhancement to improve the accuracy of feature recognition. Feature extraction can use specific neural network layers to identify key facial landmarks. Feature matching determines their correspondence in three-dimensional space by comparing feature points under different perspectives. Finally, fine-tuning can optimize the position of feature points through an iterative algorithm to achieve the best registration effect. As an alternative, physical model-based methods, such as deformation field-based methods, can be considered to simulate and correct small differences between the head and MRI images.
[0065] In some embodiments, the three-dimensional reconstruction comprises:
[0066] Use stereo vision technology to match feature points captured by a multi-view camera system;
[0067] The triangulation method is used to calculate the positions of facial feature points in three-dimensional space.
[0068] It should be noted that stereo vision technology refers to the use of two or more cameras to capture images of the same object from different angles, and then determine the 3D shape and position of the object by comparing these images. In this system, the feature points captured by the multi-view camera system are key landmarks of the face, such as specific points of the eyes, nose and mouth. The triangulation algorithm is a method that uses the projections of these feature points from different camera perspectives to mathematically calculate their exact positions in 3D space.
[0069] Preferably, the process of 3D reconstruction can be further refined into the following steps: First, the images captured by each camera are preprocessed, including denoising, grayscale and edge detection, to improve the recognition accuracy of feature points. Next, feature matching algorithms such as SIFT (Scale Invariant Feature Transform) or SURF (Speeded Up Robust Features) are used to identify and match the same feature points in images from different perspectives. Then, based on the matched feature points and the internal and external parameters of the camera, the coordinates of these points in 3D space are calculated using triangulation.
[0070] More specifically, the following alternatives or refinement steps can be adopted: Use deep learning algorithms to enhance the recognition and matching process of feature points, for example, using convolutional neural networks (CNNs) to automatically extract key features in images and match them. In addition, in order to improve the accuracy of 3D reconstruction, multi-view geometry algorithms can be used to optimize the triangulation process, taking into account the relative positions and orientations between cameras. In practical applications, the performance of the stereo vision system can also be improved by increasing the number of cameras or adjusting the layout of the cameras.
[0071] In some embodiments, the MRI registration step includes:
[0072] Preprocess and correct the posture of MRI images;
[0073] Based on the nose tip point, other feature points are determined to achieve registration between the MRI image and the face.
[0074] It should be noted that the sentence "MRI registration step includes: preprocessing and posture correction of MRI images; determining other feature points based on the nose tip point to achieve registration of MRI images with human faces" mentioned in claim 7 summarizes the process of aligning MRI images with actual human faces. This is an important step to ensure that magnetoencephalogram data accurately corresponds to MRI images.
[0075] Specifically, the preprocessing of MRI images includes operations such as denoising, contrast enhancement, and image segmentation, with the goal of improving the quality of MRI images and making them more suitable for subsequent feature point recognition and registration processes. Posture correction refers to adjusting the direction and angle of the MRI image to ensure that it matches the actual posture of the face. Determining other feature points based on the nose tip point means using the nose tip as a reference point to locate other key feature points on the face, such as eyes and ears.
[0076] Preferably, the preprocessing step can use an automated image processing algorithm, such as wavelet transform or Gaussian filtering, to remove noise from the MRI image. Posture correction can be achieved by calculating rotation and translation parameters of the image, which can be determined based on the symmetry of the face or known anatomical landmarks. When determining the feature points, an image registration algorithm, such as feature point-based registration or deformation field-based registration, can be used to accurately match the feature points on the MRI image with the feature points of the face.
[0077] More specifically, the following refined operation steps or alternatives can be adopted: First, deep learning technology is used to automatically identify and segment brain structures in MRI images. Then, posture correction is achieved by calculating the rotation matrix and translation vector of the image. In the process of determining the feature points, three-dimensional morphometrics technology can be used to improve the recognition accuracy of the feature points. In addition, the iterative closest point (ICP) algorithm can be used to optimize the registration process of the feature points to ensure the best alignment between the MRI image and the face. As an alternative, model-based methods, such as statistical shape models, can be considered to guide the recognition and registration process of feature points.
[0078] In some embodiments, the registration algorithm based on the markers includes the least squares method and the ICP algorithm, and the specific steps are:
[0079] Define a set of marked points {P i} and {Q i}, where {Pi} is the coordinate of the mark point on the helmet, {Q i} are the coordinates of the corresponding points on the head;
[0080] Solve the rigid transformation matrix R and the translation vector t so that
[0081] Use the SVD algorithm to calculate the transformation matrix and vector.
[0082] It should be noted that the registration algorithm based on markers includes the least squares method and the ICP algorithm, which outlines two main algorithms that are used to calculate and optimize the transformation parameters in the registration process. These algorithms are crucial to ensure the accuracy of the registration.
[0083] Specifically, the least squares method is a mathematical optimization technique used to find the parameters of the best fitting curve given a set of data. In the registration algorithm, it can be used to minimize the sum of squared errors between the actual point and the target point. The ICP algorithm, or iterative closest point algorithm, is a method that gradually optimizes the correspondence between point sets through an iterative process until the best registration is found. Definition The marker point set refers to a set of points pre-set in the system for registration, including the coordinates of the marker points on the helmet and the coordinates of the corresponding points on the head.
[0084] Preferably, the implementation of the registration algorithm can be further refined into the following steps: First, the coordinates of the marker points on the helmet are accurately captured by spatial positioning technology. Then, the coordinates of the corresponding points on the head are extracted using deep learning or other image processing techniques. Next, the least squares method is applied to calculate the initial rigid transformation matrix and translation vector, which parameters will minimize the error between the marker point sets. Finally, the ICP algorithm is used for iterative optimization to further improve the accuracy of the registration.
[0085] More specifically, the following refinement steps or alternatives can be adopted: In the implementation of the least squares method, a regularization term can be introduced to prevent overfitting and improve the stability of the algorithm. In the ICP algorithm, the number of iterations and the convergence threshold can be set as parameters to control the accuracy and efficiency of the algorithm. In addition, a multi-resolution strategy can be adopted to accelerate the convergence of the ICP algorithm. As an alternative, machine learning-based registration methods can be considered, which can learn registration patterns from a large amount of registration data and may provide higher registration accuracy and speed.
[0086] In some embodiments, the facial feature point-based registration algorithm includes the following steps:
[0087] Capture multiple perspective images of human faces through a multi-view camera system;
[0088] Use deep neural network to extract facial feature points;
[0089] Using the three-dimensional coordinates of the feature points, they are aligned with the feature points in the MRI image through rigid transformation;
[0090] The least squares method or ICP algorithm is used to calculate the rigid transformation matrix between the head and the MRI image to complete the registration.
[0091] It should be noted that the first step of the registration algorithm is to capture multiple perspective images of the face through a multi-view camera system. This involves using multiple cameras to capture the subject's face from different angles to obtain comprehensive facial feature information. Next, facial feature points are extracted using deep neural networks, which are trained to identify and locate key features of the face, such as eyes, nose, and mouth. The three-dimensional coordinates of the feature points are then used to align them with the feature points in the MRI image through a rigid transformation. This step involves calculating and applying a transformation so that the feature points of the face match the corresponding points in the MRI image. Finally, the rigid transformation matrix between the head and the MRI image is calculated using the least squares method or ICP algorithm to complete the registration.
[0092] Preferably, the settings of the multi-view camera system can be adjusted according to the facial features of the subject and the required accuracy. The deep neural network can be a model specially trained for facial feature recognition, such as a convolutional neural network-based architecture. During the feature point extraction process, threshold parameters can be set to determine the recognition accuracy of the feature points. The calculation of the rigid transformation can include rotation and translation, and these parameters can be determined by an optimization algorithm. The application of the least squares method or ICP algorithm can be further refined to ensure the accurate calculation of the transformation matrix.
[0093] More specifically, the following refinement steps or alternatives can be adopted: In the configuration of the multi-view camera system, the layout and resolution of the cameras can be optimized to improve the quality and coverage of the image. The training of the deep neural network can include a large number of facial images to improve its generalization ability. In the extraction of feature points, additional verification steps can be introduced to ensure the accuracy of the extracted feature points. In the calculation of rigid transformation, numerical optimization techniques can be used to improve the stability and accuracy of the calculation. As an alternative, feature-based non-rigid registration methods can be considered, which can handle more complex head deformation situations.
[0094] In some embodiments, the overall accuracy of its registration system is ensured by three-dimensional reconstruction of marker points and reconstruction of facial feature points of a multi-eye vision camera system, so as to achieve accurate collection and analysis of magnetic brain signals, wherein the positions of marker points in the rigid helmet are precisely measured and calibrated, and their three-dimensional coordinates are known in the system, providing a stable reference system.
[0095] The above-mentioned embodiments of the present invention have the following beneficial effects: the present invention can significantly improve the accuracy and efficiency of brain magnetic automatic registration by adopting a multi-eye vision system and known coordinate markers on a rigid helmet. The system uses the helmet and the subject's head and facial feature points captured by a multi-eye vision camera, combined with deep neural networks and stereo vision technology, to accurately achieve helmet-to-head and head-to-MRI image registration. This high-precision registration can not only ensure the accurate acquisition and analysis of brain magnetic signals, but also provide strong technical support for brain function research and clinical diagnosis. In addition, the registration system of the present invention can further optimize the registration process, reduce human errors, and improve the stability and reliability of the system through the application of advanced algorithms such as the least squares method and the iterative closest point (ICP) algorithm. The position of the markers in the rigid helmet is accurately measured and calibrated, and its three-dimensional coordinates are known in the system, which can provide a stable reference system. This can not only ensure the accuracy of the registration process, but also adapt to different subjects and experimental conditions, and has good versatility and flexibility.
[0096] Example 2
[0097] Exemplarily, in some embodiments, the multi-eye vision system is intended to achieve automated and high-precision registration and tracking of magnetoencephalography devices. The composition and operation flow of the system are described in detail below.
[0098] 1. System composition
[0099] In some embodiments, the multi-eye vision system mainly includes the following components:
[0100] Helmet: Equipped with multiple markers with known coordinates for registration with the subject's head.
[0101] Multi-camera system: It consists of multiple cameras, including at least two high-resolution cameras for facial feature point extraction and other cameras for capturing feature points on the helmet.
[0102] Feature point markers: set on the helmet and the surface of the subject's head to facilitate image capture and feature point extraction by the multi-camera system.
[0103] Data processing unit: responsible for receiving data captured by the multi-camera system and performing image processing, feature point matching, registration and tracking algorithms.
[0104] User interface: Provides an operation interface that displays the registration and tracking results in real time, allowing users to adjust parameters and export data.
[0105] 2. Initial Registration Process
[0106] The system first performs an initial registration of the helmet and the subject's head. The multi-camera captures images of the helmet and head, extracts feature points and matches them. Using calibration technology and registration algorithms, the initial position and posture relationship between the helmet and the head is calculated.
[0107] 3. Real-time tracking and registration
[0108] During the experiment, the multi-camera system captured images of the helmet and head in real time, extracted feature points and matched them. The relative movement of the helmet and head was calculated through the tracking algorithm, and the position information of the helmet was updated in real time. Figure 1 As shown, examples of the relative positions and some feature points of the helmet and the multi-camera system are shown.
[0109] 4. Helmet design
[0110] The helmet is designed for comfort, stability and adjustability. The interior of the helmet is equipped with soft padding and reflective markings are evenly distributed on the surface to ensure that it can be clearly captured in different lighting conditions. Figure 2 As shown, the design of the helmet and the distribution of the marking points are shown in detail.
[0111] 5. Camera Arrangement and Synchronization
[0112] The multi-cameras are evenly arranged around the subject to form a circular structure to ensure full coverage of all marking points on the helmet. Figure 3 As shown in the figure, a top view of the camera arrangement and the specific position of each camera are provided. Each camera is fixed on an adjustable bracket to adapt to the height and sitting posture of different subjects. The hardware trigger method is used to achieve synchronous control of multiple cameras.
[0113] 6. Data processing unit
[0114] The data processing unit includes an image acquisition module, an image processing module and a posture estimation module. The image acquisition module synchronously receives image data from multiple cameras. The image processing module performs preprocessing operations such as image enhancement, denoising, and marker recognition. The posture estimation module calculates the position and posture of the helmet relative to the subject's head based on the preprocessed image data using three-dimensional reconstruction and posture estimation algorithms.
[0115] 7. Algorithm Implementation
[0116] The facial feature point extraction and 3D reconstruction use the MTCNN algorithm and stereo vision technology. The helmet feature point capture and registration use feature point template matching and ICP algorithm. 8. System advantages
[0117] The multi-eye vision system in some embodiments improves the measurement accuracy and reliability of the magnetoencephalography device through high-precision registration and tracking. Automated operation simplifies the experimental process and reduces the difficulty of operation. It is suitable for a variety of brain science research and clinical application scenarios, and helps in the early diagnosis and treatment of brain diseases.
[0118] Example 3
[0119] The electronic devices in some embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
[0120] Electronic devices may include processing devices (e.g., central processing units, graphics processing units, etc.), which may perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) or programs loaded from a storage device into a random access memory (RAM). In RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0121] Typically, the following devices can be connected to the I / O interface: input devices such as touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices such as liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices such as magnetic tapes, hard disks, etc.; and communication devices. The communication device can allow the electronic device to communicate with other devices wirelessly or wired to exchange data. Further, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.
[0122] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention to form a technical solution.
Claims
1. A multi-eye vision system for automatic brain magnetoencephalography registration, characterized in that: include: A rigid helmet having a plurality of known coordinate markers at predetermined locations for providing initial sensor coordinates and assisting in helmet-head registration; A multi-view camera system is arranged around the subject to capture the feature points of the helmet and the subject's head and face; The registration system is used to realize the registration from helmet to head based on helmet markers, and the registration from head to MRI based on facial feature points.
2. The system according to claim 1, characterized in that The coordinate markers of the rigid helmet provide accurate initial coordinate references for the multi-view camera system.
3. The system according to claim 1, wherein the multi-view camera system comprises at least two high-resolution cameras for facial feature point extraction and three-dimensional reconstruction, and a plurality of other cameras for capturing marker points on the helmet.
4. The system according to claim 1, wherein the registration system uses known coordinate markers on the helmet to achieve accurate positioning of the helmet on the subject's head.
5. The system according to claim 1, wherein the registration system realizes the registration of the head to the MRI by the following steps: Use deep neural networks to identify facial feature points, including the root of the nose, the tip of the nose, and the wings of the nose; Identify MRI feature points, where the nose tip point is defined as the maximum value point on the Y axis, and locate other feature points based on this.
6. The system according to claim 1, wherein the three-dimensional reconstruction of the feature points comprises: Use stereo vision technology to match feature points captured by a multi-view camera system; The triangulation method is used to calculate the positions of facial feature points in three-dimensional space.
7. The system according to claim 1, wherein the MRI registration step comprises: Preprocess and correct the posture of MRI images; Based on the nose tip point, other feature points are determined to achieve registration between the MRI image and the face.
8. The system according to claim 1, wherein the registration algorithm based on the markers comprises the least square method and the ICP algorithm, and the specific steps are: Define a set of marked points {P i } and {Q i }, where {P i } is the coordinate of the mark point on the helmet, {Q i } are the coordinates of the corresponding points on the head; Solve the rigid transformation matrix R and the translation vector t so that Use the SVD algorithm to calculate the transformation matrix and vector.
9. The system according to claim 1, wherein the registration algorithm based on facial feature points comprises the following steps: Capture multiple perspective images of human faces through a multi-view camera system; Use deep neural network to extract facial feature points; Using the three-dimensional coordinates of the feature points, they are aligned with the feature points in the MRI image through rigid transformation; The least squares method or ICP algorithm is used to calculate the rigid transformation matrix between the head and the MRI image to complete the registration.
10. According to the system of claim 1, the overall accuracy of its registration system is ensured by the three-dimensional reconstruction of the marker points and the reconstruction of the facial feature points of the multi-eye vision camera system, so as to realize the accurate collection and analysis of the magnetic brain signal, wherein the positions of the marker points in the rigid helmet are accurately measured and calibrated, and their three-dimensional coordinates are known in the system, providing a stable reference system.
Citation Information
Patent Citations
Helmet pose tracking system and method for IMU-assisted visual feature point capture
CN115690910A
Automatic registration method for SERF atom magnetometer and head nuclear magnetic resonance image
CN116630384A
Method of locating EEG and meg sensors on a head
WO2013026749A1
Method and system for aligning neurophysiological sensors
WO2023187785A1
Cited By
Registration method and device of magnetoencephalography, equipment, storage medium and program product
CN120259389A
MRI head movement monitoring system based on machine learning
CN120837046A
