A multi-view vision system for automatic brain-magnetic registration

Abstract

This invention provides a multi-view vision system for automatic registration of magnetoencephalography (MEG), comprising: a rigid helmet with multiple pre-defined coordinate markers for providing initial coordinates for sensors and assisting helmet-to-head registration; a multi-view vision camera system positioned around the subject to capture feature points of the helmet and the subject's head and face; and a registration system for helmet-to-head registration based on helmet markers and head-to-MRI registration based on facial feature points. This invention can improve the accuracy and reliability of MEG signal acquisition, providing strong technical support for brain function research and clinical diagnosis.

Description

Technical Field

[0001] This invention relates to the field of biomedical engineering technology, and more specifically, to a multi-view vision system for automatic brain magnetic registration. Background Technology

[0002] In the field of magnetoencephalography (MEG) signal acquisition, traditional techniques mainly rely on single-view visual systems or manual marker registration, which have significant limitations in accuracy and efficiency. Existing technologies often require complex and time-consuming manual calibration processes to ensure accurate acquisition of MEG signals. Furthermore, due to the lack of multi-view synchronization information, existing systems suffer from insufficient registration accuracy when handling complex head structures or movements, thus affecting the quality of data acquisition.

[0003] Although existing technologies have achieved automatic registration of magnetoencephalography (MEG) signals to some extent, the inventors discovered at least the following problems or defects in the prior art during the implementation of the embodiments of this invention: instability of helmet positioning, limitations of multi-view camera systems in capturing facial feature points, and registration errors between MRI images and actual head structures. These problems or defects result in the registration accuracy of MEG signals not meeting the requirements of high-precision research, thus limiting the widespread application of MEG technology in clinical and research fields. Summary of the Invention

[0004] This invention provides a multi-view vision system for automatic electroencephalogram (EEG) registration, comprising:

[0005] A rigid helmet with multiple pre-defined coordinate markers at predetermined locations to provide initial coordinates for sensors and assist in helmet-head registration; A multi-view vision camera system is deployed around the subject to capture feature points of the helmet and the subject's head and face; The registration system is used to achieve helmet-to-head registration based on helmet markers, and head-to-MRI registration based on facial feature points.

[0006] Furthermore, the coordinate markers on the rigid helmet provide a precise initial coordinate reference for the multi-view vision camera system.

[0007] Furthermore, its multi-view vision camera system includes at least two high-resolution cameras for facial feature point extraction and 3D reconstruction, and several other cameras for capturing marker points on the helmet.

[0008] Furthermore, its registration system utilizes known coordinate markers on the helmet to achieve precise positioning of the helmet on the subject's head.

[0009] Furthermore, its registration system achieves head-to-MRI registration through the following steps: Deep neural networks are used to identify facial feature points, including the root of the nose, the tip of the nose, and the wings of the nose. Identify MRI feature points, with the nasal tip point defined as the maximum value point on the Y-axis, and locate other feature points accordingly.

[0010] Furthermore, the three-dimensional reconstruction of its feature points includes: Using stereo vision technology to match feature points captured by a multi-view camera system; The positions of facial feature points in three-dimensional space are calculated using triangulation.

[0011] Furthermore, the MRI registration steps include: Preprocessing and orientation correction of MRI images; Other feature points are determined based on the tip of the nose to achieve registration between MRI images and human faces.

[0012] Furthermore, its marker-based registration algorithm includes the least squares method and the ICP algorithm, with the specific steps as follows: Define a set of marker points and ,in These are the coordinates of the marker points on the helmet. These are the coordinates of the corresponding point on the head; Solve for the rigid transformation matrix R and the translation vector t such that ; The transformation matrix and vectors are calculated using the SVD algorithm.

[0013] Furthermore, its registration algorithm based on facial feature points includes the following steps: Capture images of a human face from multiple perspectives using a multi-view vision camera system; Use deep neural networks to extract facial feature points; Using the three-dimensional coordinates of the feature points, they are aligned with the feature points in the MRI image through rigid transformation; The rigid transformation matrix between the head and the MRI image is calculated using the least squares method or the ICP algorithm to complete the registration.

[0014] Furthermore, the overall accuracy of its registration system is ensured through the 3D reconstruction of marker points and facial feature points of the multi-view vision camera system, enabling precise acquisition and analysis of magnetoencephalogram (MEG) signals. The positions of the marker points in the rigid helmet are precisely measured and calibrated, and their 3D coordinates are known in the system, providing a stable reference system.

[0015] The embodiments of the present invention have at least the following beneficial effects: This multi-view vision system, by using known coordinate markers on a rigid helmet and a multi-view vision camera system, can provide initial sensor coordinates and accurate references for helmet-to-head registration. Utilizing deep neural networks and stereo vision technology, the system can efficiently identify and match facial feature points, achieving accurate registration from helmet to head and from head to MRI images. This registration method not only improves the speed and convenience of registration but also significantly enhances the accuracy of magnetoencephalography (MEG) signal acquisition. Furthermore, the least squares method and Iterative Closest Point (ICP) algorithm employed by the system can further optimize the registration process, ensuring the accuracy of 3D reconstruction of the helmet and facial feature points. Through the application of these algorithms, the system can automatically adjust and optimize registration parameters, reducing human error, thereby ensuring high-quality and high-reliability data during the acquisition and analysis of MEG signals. This high-precision registration technology is of great significance for brain function research and clinical diagnosis, providing a powerful tool for researchers and physicians in related fields. Attached Figure Description

[0016] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated in the drawings by way of example and not limitation, wherein: Figure 1 This is a schematic diagram of the structure of a multi-view vision system for automatic electroencephalogram (EEG) registration according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a multi-view vision system provided in an embodiment of the present invention; Figure 3 A top view of a fully automatic registration apparatus according to an embodiment of the present invention is shown schematically. Detailed Implementation

[0017] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make the invention more thorough and complete, and to fully convey the scope of the invention to those skilled in the art.

[0018] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present invention can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0019] It should be noted that the number of any elements in the accompanying drawings is for illustrative purposes only and not as a limitation, and any naming is for distinction only and has no limiting meaning.

[0020] Example 1 The following is for reference. Figure 1 , Figure 1 This is a schematic diagram of the structure of a multi-view vision system for automatic electroencephalogram (EEG) registration according to an embodiment of the present invention. Figure 1 As shown, a multi-view vision system 100 for automatic brain-magnetic registration includes: A rigid helmet 101 has multiple known coordinate markers at predetermined positions to provide initial coordinates for sensors and assist in helmet-head registration; A multi-view vision camera system 102 is positioned around the subject to capture feature points of the helmet and the subject's head and face; The registration system 103 is used to perform helmet-to-head registration based on helmet markers and head-to-MRI registration based on facial feature points.

[0021] It should be noted that the core function of the multi-view vision system for automatic EEG registration is to achieve precise registration between the helmet and the subject's head, as well as precise registration between the head and MRI images. This involves not only the design of the physical equipment but also the application of algorithms to ensure that the entire system can work efficiently and accurately.

[0022] Specifically, the system includes a rigid helmet with multiple pre-defined coordinate markers on its surface. These markers are crucial because they provide a precise initial coordinate reference for the multi-view camera system. The multi-view camera system consists of at least two high-resolution cameras responsible for capturing feature points on the helmet and the subject's head and face, as well as several other cameras for capturing the markers on the helmet. The registration system then uses this information to register the helmet and head, as well as the head and MRI images, using specific algorithms.

[0023] Preferably, the rigid helmet can be made of lightweight and durable materials to ensure wearer comfort while guaranteeing the stability of the marker points. The settings of the multi-view vision camera system can be adjusted according to actual needs; for example, the camera resolution, focal length, and capture angle can be optimized based on the subject's head size and shape. The registration system can employ advanced deep learning algorithms, such as convolutional neural networks (CNNs), to improve the accuracy of facial feature point recognition. Furthermore, the registration algorithm can combine least squares and iterative nearest point (ICP) algorithms to achieve even higher registration accuracy. In practical applications, algorithm parameters can be adjusted according to different research objectives or clinical needs to achieve the best registration results.

[0024] In some embodiments, the coordinate markers on the rigid helmet provide a precise initial coordinate reference for the multi-view vision camera system.

[0025] It should be noted that the coordinate markers on the rigid helmet provide a precise initial coordinate reference for the multi-view vision camera system, highlighting the importance of these markers in the system registration process. These markers not only provide an initial reference point for the camera system, but their positional accuracy directly affects the accuracy of the entire registration process.

[0026] Specifically, the coordinate markers on a rigid helmet can be markers with specific geometric shapes and sizes, such as circles or squares. Their positions are predetermined using precision measuring equipment and recorded in the system. These markers are typically evenly distributed across the helmet to ensure they can be captured by the camera system from different angles. The multi-view camera system consists of multiple high-resolution cameras that are precisely calibrated to ensure that the captured images match the coordinates of the markers on the helmet.

[0027] Preferably, the coordinate markers on the helmet can use high-contrast colors or reflective materials to improve visibility under different lighting conditions. The camera system can include multiple lenses with different focal lengths to adapt to shooting needs at different distances and angles. Furthermore, the system can employ automatic exposure and white balance control to ensure image quality. During implementation, machine vision algorithms can be used to quickly identify and locate the markers, and then this data can be used as initial coordinates to provide a precise starting point for subsequent registration algorithms. As an alternative, active markers can be considered to improve recognition accuracy and robustness in complex environments.

[0028] In some embodiments, the multi-view camera system includes at least two high-resolution cameras for facial feature point extraction and 3D reconstruction, and several other cameras for capturing marker points on the helmet.

[0029] It should be noted that the multi-view vision camera system includes at least two high-resolution cameras for facial feature point extraction and 3D reconstruction, and several other cameras for capturing marker points on the helmet, illustrating the key components of the system and their functions. These camera systems are designed to ensure that images captured from different angles provide sufficient information for accurate registration.

[0030] Specifically, at least two high-resolution cameras in a multi-view camera system refer to devices with sufficient pixels and resolution to capture clear images. These cameras are used to extract key feature points of the subject's face, such as the eyes, nose, and mouth, which are crucial for 3D reconstruction. Several other cameras may include cameras with different focal lengths and angles of view, specifically designed to capture known coordinate markers on the helmet to aid in spatial localization.

[0031] Preferably, these high-resolution cameras can be configured with autofocus and image stabilization to ensure high-quality images are acquired at varying distances and under different conditions. The camera resolution and frame rate can be adjusted as needed to accommodate the capture requirements of fast-moving or minutely changing objects. Furthermore, the system can employ a synchronous triggering mechanism to ensure all cameras capture images simultaneously, facilitating subsequent image matching and data processing. Alternatively, structured light or Time-of-Flight (ToF) cameras can be considered to enhance the accuracy and speed of 3D reconstruction.

[0032] In some embodiments, the registration system utilizes known coordinate markers on the helmet to achieve precise positioning of the helmet on the subject's head.

[0033] It should be noted that the registration system utilizes known coordinate markers on the helmet to achieve precise positioning of the helmet on the subject's head, emphasizing how the registration system uses these markers to ensure accurate alignment between the helmet and the subject's head. This precise positioning is fundamental to subsequent head-to-MRI registration.

[0034] Specifically, the known coordinate markers in the registration system refer to coordinate points that are pre-determined and recorded in the system during the manufacturing process. These markers provide a stable reference frame for the system, enabling the registration algorithm to accurately determine the position of the helmet relative to the subject's head. This process typically requires an algorithm that can process image data captured by a multi-view camera system and match it with the coordinates of the markers on the helmet.

[0035] Preferably, the registration system can employ advanced image processing and pattern recognition technologies to automatically identify and track markers on the helmet. For example, feature matching algorithms can be used to identify markers in the image, and the helmet's position can be determined by calculating the spatial relationship between the camera and the markers. Furthermore, the system can integrate a calibration module, allowing users to precisely calibrate the camera system before actual use to ensure the most accurate correspondence between the captured image data and the known coordinate markers. Alternatively, machine learning algorithms can be considered to improve the accuracy and robustness of marker recognition, especially under conditions of subject head movement or changes in lighting conditions.

[0036] In some embodiments, the registration system achieves head-to-MRI registration through the following steps: Deep neural networks are used to identify facial feature points, including the root of the nose, the tip of the nose, and the wings of the nose. Identify MRI feature points, with the nasal tip point defined as the maximum value point on the Y-axis, and locate other feature points accordingly.

[0037] It should be noted that the registration system achieves head-to-MRI registration through the following steps. This outlines how the registration system precisely aligns the subject's head with the MRI image. This process is crucial for ensuring the accuracy of the magnetoencephalogram (MEG) data.

[0038] Specifically, the registration system first uses a deep neural network to identify facial feature points, including the root of the nose, the tip of the nose, and the alar of the nose. Identifying these feature points is crucial for head-to-MRI registration, as they provide reference points for facial structures. Next, the system identifies MRI feature points, with the tip of the nose defined as the point of maximum Y-axis value. This point is used to locate other feature points, thereby achieving alignment between images.

[0039] Preferably, the deep neural network can be a pre-trained model specifically designed for recognizing and locating facial feature points in images. This model, based on a convolutional neural network (CNN) architecture, is capable of processing high-resolution images provided by multi-view vision camera systems. Furthermore, the system can employ image registration algorithms, such as feature-based registration or model-based registration, to further refine the recognition and localization of feature points. During the recognition of MRI feature points, the system can use automated image analysis tools to determine the maximum Y-axis point and adjust the positions of other feature points accordingly.

[0040] More specifically, the registration system can employ a multi-stage deep learning workflow, including preprocessing, feature extraction, feature matching, and fine-tuning. Preprocessing may include steps such as image denoising and contrast enhancement to improve the accuracy of feature recognition. Feature extraction can utilize specific neural network layers to identify key facial landmarks. Feature matching determines the correspondence between feature points in three-dimensional space by comparing feature points from different viewpoints. Finally, fine-tuning optimizes the position of feature points using iterative algorithms to achieve the best registration result. As an alternative, physical model-based methods, such as deformation field-based methods, can be considered to simulate and correct for subtle differences between the head and MRI images.

[0041] In some embodiments, three-dimensional reconstruction includes: Using stereo vision technology to match feature points captured by a multi-view camera system; The positions of facial feature points in three-dimensional space are calculated using triangulation.

[0042] It should be noted that stereo vision technology refers to using two or more cameras to capture images of the same object from different angles, and then determining the object's three-dimensional shape and position by comparing these images. In this system, the feature points captured by the multi-view camera system are key facial landmarks, such as specific points of the eyes, nose, and mouth. Triangulation is a mathematical method that uses the projections of these feature points from different camera viewpoints to calculate their exact positions in three-dimensional space.

[0043] Preferably, the 3D reconstruction process can be further refined into the following steps: First, preprocess the images captured by each camera, including denoising, grayscale conversion, and edge detection, to improve the accuracy of feature point recognition. Next, feature matching algorithms, such as SIFT (Scale Invariant Feature Transform) or SURF (Speeded Robust Feature Transform), are used to identify and match identical feature points in images from different viewpoints. Then, based on the matched feature points and the camera's intrinsic and extrinsic parameters, triangulation is applied to calculate the coordinates of these points in 3D space.

[0044] More specifically, alternative solutions or refined operational steps can be adopted, such as using deep learning algorithms to enhance the feature point recognition and matching process, for example, using convolutional neural networks (CNNs) to automatically extract and match key features in images. Furthermore, to improve the accuracy of 3D reconstruction, multi-view geometric algorithms can be employed to optimize the triangulation process, taking into account the relative positions and orientations between cameras. In practical applications, the performance of the stereo vision system can also be improved by increasing the number of cameras or adjusting their layout.

[0045] In some embodiments, the MRI registration step includes: Preprocessing and orientation correction of MRI images; Other feature points are determined based on the tip of the nose to achieve registration between MRI images and human faces.

[0046] It should be noted that the statement "MRI registration steps include: preprocessing and posture correction of MRI images; determining other feature points based on the nasal tip point to achieve registration between MRI images and the human face" summarizes the process of aligning MRI images with actual human faces. This is a crucial step in ensuring accurate correspondence between magnetoencephalography (MEG) data and MRI images.

[0047] Specifically, MRI image preprocessing includes operations such as noise reduction, contrast enhancement, and image segmentation, aiming to improve the quality of MRI images and make them more suitable for subsequent feature point recognition and registration processes. Pose correction refers to adjusting the orientation and angle of the MRI image to ensure it matches the actual posture of the face. Determining other feature points based on the nose tip means using the nose tip as a reference point to locate other key facial features, such as the eyes and ears.

[0048] Preferably, the preprocessing step can employ automated image processing algorithms, such as wavelet transform or Gaussian filtering, to remove noise from the MRI images. Pose correction can be achieved by calculating the rotation and translation parameters of the image, which can be determined based on facial symmetry or known anatomical landmarks. When determining feature points, image registration algorithms, such as feature point-based registration or deformation field-based registration, can be used to accurately map feature points on the MRI image to feature points on the face.

[0049] More specifically, the following refined operational steps or alternatives can be adopted: First, use deep learning techniques to automatically identify and segment brain structures in MRI images. Then, pose correction is achieved by calculating the image's rotation matrix and translation vector. During feature point determination, three-dimensional morphometry techniques can be used to improve feature point recognition accuracy. Furthermore, the Iterative Closest Point (ICP) algorithm can be used to optimize the feature point registration process, ensuring optimal alignment between the MRI image and the face. As an alternative, model-based methods, such as statistical shape models, can be considered to guide the feature point recognition and registration process.

[0050] In some embodiments, the registration algorithm based on marker points includes the least squares method and the ICP algorithm, and the specific steps are as follows: Define a set of marker points and ,in These are the coordinates of the marker points on the helmet. These are the coordinates of the corresponding point on the head; Solve for the rigid transformation matrix R and the translation vector t such that ; The transformation matrix and vectors are calculated using the SVD algorithm.

[0051] It should be noted that marker-based registration algorithms include the least squares method and the ICP algorithm. Two main algorithms are outlined here, used to calculate and optimize transformation parameters during the registration process. These algorithms are crucial for ensuring registration accuracy.

[0052] Specifically, the least squares method is a mathematical optimization technique used to find the parameters of the best-fit curve given a set of data. In registration algorithms, it can be used to minimize the sum of squared errors between the actual points and the target points. The ICP algorithm, or Iterative Closest Point algorithm, is a method that iteratively optimizes the correspondence between point sets until the best registration is found. The set of marker points refers to a pre-defined set of points used for registration in the system, including the coordinates of marker points on the helmet and the corresponding coordinates of points on the head.

[0053] Preferably, the registration algorithm can be further refined into the following steps: First, the coordinates of marker points on the helmet are accurately captured using spatial positioning technology. Then, the coordinates of corresponding points on the head are extracted using deep learning or other image processing techniques. Next, the least squares method is applied to calculate the initial rigid transformation matrix and translation vector, which minimize the error between the sets of marker points. Finally, the ICP algorithm is used for iterative optimization to further improve the registration accuracy.

[0054] More specifically, the following refined operational steps or alternatives can be adopted: In the implementation of the least squares method, a regularization term can be introduced to prevent overfitting and improve the stability of the algorithm. In the ICP algorithm, the number of iterations and the convergence threshold can be set as parameters to control the accuracy and efficiency of the algorithm. Furthermore, a multi-resolution strategy can be employed to accelerate the convergence of the ICP algorithm. As an alternative, machine learning-based registration methods can be considered, which can learn registration patterns from large amounts of registration data and may provide higher registration accuracy and speed.

[0055] In some embodiments, the registration algorithm based on facial feature points includes the following steps: Capture images of a human face from multiple perspectives using a multi-view vision camera system; Use deep neural networks to extract facial feature points; Using the three-dimensional coordinates of the feature points, they are aligned with the feature points in the MRI image through rigid transformation; The rigid transformation matrix between the head and the MRI image is calculated using the least squares method or the ICP algorithm to complete the registration.

[0056] It's important to note that the first step in the registration algorithm is capturing multiple viewpoint images of the face using a multi-view camera system. This involves using multiple cameras to photograph the subject's face from different angles to obtain comprehensive facial feature information. Next, deep neural networks are used to extract facial feature points. These networks are trained to recognize and locate key facial features such as the eyes, nose, and mouth. Then, using the 3D coordinates of the feature points, a rigid transformation is applied to align them with feature points in the MRI image. This step involves calculating and applying a transformation that matches the facial feature points with corresponding points in the MRI image. Finally, the rigid transformation matrix between the head and the MRI image is calculated using the least squares method or the ICP algorithm to complete the registration.

[0057] Preferably, the setup of the multi-view vision camera system can be adjusted based on the subject's facial features and the required accuracy. The deep neural network can be a model specifically trained for facial feature recognition, such as a convolutional neural network-based architecture. During feature point extraction, threshold parameters can be set to determine the accuracy of feature point recognition. The calculation of rigid transformations can include rotations and translations, and these parameters can be determined through optimization algorithms. The application of least squares or ICP algorithms can further refine the process to ensure accurate calculation of the transformation matrix.

[0058] More specifically, the following refined operational steps or alternatives can be adopted: In the configuration of a multi-view vision camera system, the camera layout and resolution can be optimized to improve image quality and coverage. Training of deep neural networks can include a large number of facial images to improve their generalization ability. In feature point extraction, additional verification steps can be introduced to ensure the accuracy of the extracted feature points. In the calculation of rigid transformations, numerical optimization techniques can be employed to improve computational stability and accuracy. As an alternative, feature-based non-rigid registration methods can be considered, which can handle more complex head deformations.

[0059] In some embodiments, the overall accuracy of the registration system is ensured by the 3D reconstruction of marker points and facial feature point reconstruction of the multi-view vision camera system, thereby achieving accurate acquisition and analysis of magnetoencephalogram (MEG) signals. The positions of the marker points in the rigid helmet are precisely measured and calibrated, and their 3D coordinates are known in the system, providing a stable reference system.

[0060] The various embodiments of the present invention have the following beneficial effects: By employing a multi-view vision system and known coordinate marker points on a rigid helmet, the present invention can significantly improve the accuracy and efficiency of automatic EEG registration. The system utilizes the helmet and subject's head and facial feature points captured by the multi-view vision camera, combined with deep neural networks and stereo vision technology, to accurately achieve helmet-to-head and head-to-MRI image registration. This high-precision registration not only ensures accurate acquisition and analysis of EEG signals but also provides strong technical support for brain function research and clinical diagnosis. Furthermore, the registration system of the present invention, through the application of advanced algorithms such as least squares and iterative nearest point (ICP) algorithms, can further optimize the registration process, reduce human error, and improve the stability and reliability of the system. The marker point positions in the rigid helmet are precisely measured and calibrated, and their three-dimensional coordinates are known in the system, providing a stable reference frame. This not only ensures the accuracy of the registration process but also adapts to different subjects and experimental conditions, exhibiting excellent versatility and flexibility.

[0061] Example 2 Exemplary examples show that multi-view vision systems in some embodiments are designed to automate and achieve high-precision registration and tracking of magnetoencephalography (MEG) devices. The composition and operation of the system are described in detail below.

[0062] 1. System Composition In some embodiments, a multi-view vision system mainly includes the following components: Helmet: Equipped with multiple markers with known coordinates for registration with the subject's head.

[0063] Multi-camera system: consists of multiple cameras, including at least two high-resolution cameras for facial feature point extraction, and other cameras for capturing feature points on the helmet.

[0064] Feature point marking: Set on the helmet and subject's head surface to enable the multi-view camera system to capture images and extract feature points.

[0065] Data processing unit: responsible for receiving data captured by the multi-camera system and performing image processing, feature point matching, registration and tracking algorithms.

[0066] User interface: Provides an operation interface that displays registration and tracking results in real time, allowing users to adjust parameters and export data.

[0067] 2. Initial registration process The system first performs initial registration between the helmet and the subject's head. Multi-view cameras capture images of the helmet and head, extract feature points, and perform matching. Using calibration techniques and registration algorithms, the initial position and pose relationship between the helmet and head is calculated.

[0068] 3. Real-time tracking and registration During the experiment, the multi-camera system captured images of the helmet and head in real time, extracted feature points, and performed matching. A tracking algorithm calculated the relative motion between the helmet and head, updating the helmet's position information in real time. Figure 1 As shown, an example of the relative positions and partial feature points of the helmet and multi-camera system is presented.

[0069] 4. Helmet Design The helmet is designed with comfort, stability, and adjustability in mind. The interior features soft padding, and the surface is evenly covered with reflective markers to ensure clear visibility under varying lighting conditions. Figure 2 As shown, the design of the helmet and the distribution of the marker points are illustrated in detail.

[0070] 5. Camera Setup and Synchronization Multiple cameras were evenly arranged around the subject, forming a circular structure to ensure complete coverage of all marked points on the helmet. For example... Figure 3The diagram shows a top-down view of the camera setup and the specific location of each camera. Each camera is mounted on an adjustable stand to accommodate different subject heights and sitting postures. Hardware triggering is used to achieve synchronized control of the multiple cameras.

[0071] 6. Data Processing Unit The data processing unit includes an image acquisition module, an image processing module, and a pose estimation module. The image acquisition module simultaneously receives image data from multiple cameras. The image processing module performs preprocessing operations such as image enhancement, denoising, and marker recognition. Based on the preprocessed image data, the pose estimation module uses 3D reconstruction and pose estimation algorithms to calculate the position and pose of the helmet relative to the subject's head.

[0072] 7. Algorithm Implementation Facial feature point extraction and 3D reconstruction utilize the MTCNN algorithm and stereo vision techniques. Helmet feature point capture and registration employ feature point template matching and the ICP algorithm. Figure 2 The diagram shows a flowchart of feature point matching and 3D reconstruction.

[0073] 8. System Advantages In some embodiments, the multi-view vision system improves the measurement accuracy and reliability of magnetoencephalography (MEG) devices through high-precision registration and tracking. Automated operation simplifies experimental procedures, reduces operational complexity, and is suitable for various neuroscience research and clinical applications, contributing to the early diagnosis and treatment of brain diseases.

[0074] Example 3 The electronic devices in some embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers.

[0075] Electronic devices may include processing units (such as central processing units, graphics processing units, etc.) that perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or loaded from storage devices into random access memory (RAM). RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0076] Typically, the following devices can be connected to the I / O interface: input devices such as touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices such as liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices such as magnetic tapes, hard disks, etc.; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Further, the storage medium in the embodiments of this application stores program instructions capable of implementing all the above methods. These program instructions can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0077] The above description is merely an explanation of some preferred embodiments of the present invention and the technical principles employed. Those skilled in the art should understand that the scope of the invention as described in the embodiments of the present invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A multi-view vision system for automatic electroencephalogram (EEG) registration, characterized in that, include: A rigid helmet with multiple pre-defined coordinate markers at predetermined locations to provide initial coordinates for sensors and assist in helmet-head registration; A multi-view vision camera system is deployed around the subject to capture feature points of the helmet and the subject's head and face; The registration system is used to achieve helmet-to-head registration based on helmet markers, and head-to-MRI registration based on facial feature points. The multi-view vision camera system includes at least two high-resolution cameras for facial feature point extraction and 3D reconstruction, and several other cameras for capturing marker points on the helmet; The registration system achieves head-to-MRI registration through the following steps: Deep neural networks are used to identify facial feature points, including the root of the nose, the tip of the nose, and the wings of the nose. Identify MRI feature points, where the nasal tip is defined as the maximum point on the Y-axis, and use this as a basis to locate other feature points; 3D reconstruction of feature points includes: Using stereo vision technology to match feature points captured by a multi-view camera system; The positions of facial feature points in three-dimensional space are calculated using triangulation. Marker-based registration algorithms include least squares and ICP algorithms, with the following specific steps: Define a set of marker points and ,in These are the coordinates of the marker points on the helmet. These are the coordinates of the corresponding point on the head; Solve for the rigid transformation matrix R and the translation vector t such that ; The SVD algorithm is used to calculate the transformation matrix and vectors; The registration algorithm based on facial feature points includes the following steps: Capture images of a human face from multiple perspectives using a multi-view vision camera system; Use deep neural networks to extract facial feature points; Using the three-dimensional coordinates of the feature points, they are aligned with the feature points in the MRI image through rigid transformation; The rigid transformation matrix between the head and the MRI image is calculated using the least squares method or the ICP algorithm to complete the registration.

2. The system according to claim 1, characterized in that, The coordinate markers on the rigid helmet provide a precise initial coordinate reference for the multi-view vision camera system.

3. The system according to claim 1, wherein the registration system utilizes known coordinate markers on the helmet to achieve precise positioning of the helmet on the subject's head.

4. The system according to claim 1, wherein the MRI registration step comprises: Preprocessing and orientation correction of MRI images; Other feature points are determined based on the tip of the nose to achieve registration between MRI images and human faces.

5. The system according to claim 1, wherein the overall accuracy of its registration system is guaranteed by the three-dimensional reconstruction of marker points and facial feature point reconstruction of the multi-view vision camera system, thereby achieving accurate acquisition and analysis of the magnetoencephalogram (MEG) signal, wherein the position of the marker points in the rigid helmet is accurately measured and calibrated, and their three-dimensional coordinates are known in the system, providing a stable reference system.

Images