OPM-MEG and MRI registration method and system based on binocular camera

Abstract

The invention provides an OPM-MEG and MRI registration method and system based on a binocular camera, and the method comprises the steps: firstly carrying out the calibration of the binocular camera, and obtaining a calibration parameter through a checkerboard calibration method; then, binocular images of the testee wearing the brain magnetic measurement device are collected; performing three-dimensional reconstruction, obtaining an initial disparity map through epipolar correction and an SGBM binocular stereo matching algorithm, optimizing the disparity map to generate a three-dimensional point cloud, and performing noise reduction and smoothing processing by using MLS and Poisson surface reconstruction technologies; in the face point cloud registration stage, point cloud is cut by taking a nose tip point as a sphere center, ICP registration is carried out, and a transformation matrix from a camera coordinate system to an MRI coordinate system is obtained; and finally, in a helmet positioning stage, detecting a three-dimensional coordinate of a ball target on the helmet, calculating a transformation matrix from a helmet coordinate system to a camera coordinate system, and unifying the helmet coordinate system to an MRI coordinate system. According to the invention, a one-button registration process can be realized, the registration time is shortened, the registration precision is improved, and a basis is provided for improving the source positioning precision.

Description

Technical Field

[0001] The present invention relates to the field of brain magnetism research, and in particular to an OPM-MEG and MRI registration method based on a binocular camera. Background Art

[0002] Magnetoencephalography can help researchers locate the origins of neural signals, a process known as source localization. Source localization requires a cortical model generated from magnetic resonance imaging (MRI), but OPM-MEG and MRI are acquired from different devices; therefore, it is necessary to construct the position and orientation of the magnetic measurement sensor relative to the MRI (i.e., registration). The accuracy of source localization depends largely on the accuracy of registration. For example, accurate estimates of the orientation of cortical columns and the spatial extent of neuronal activation can improve source localization, but accurate MEG and MRI registration is required. A variety of registration schemes have been developed for OPM-MEG, but a convenient and feasible automatic registration scheme has not yet emerged.

[0003] In the field of stereo vision, binocular camera technology is widely used to obtain three-dimensional information of a scene. By simulating the human binocular vision system, the binocular camera can capture the left and right parallax of the scene and then reconstruct the point cloud in the three-dimensional space. This technology has a wide range of applications in robot navigation, autonomous vehicles, and augmented reality. Despite this, the application of binocular camera technology to medical imaging, especially the registration with high-precision imaging technologies such as MRI and MEG, still faces many technical challenges.

[0004] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: most of the existing OPM-MEG registration schemes are based on manual registration of manually selected target points, and their registration accuracy is limited by the operator's experience and is time-consuming and labor-intensive, which is not conducive to the promotion and application of OPM-MEG, and lacks sufficient automation and accuracy, resulting in a time-consuming and error-prone registration process. Summary of the invention

[0005] The present invention provides an OPM-MEG and MRI registration method and system based on a binocular camera.

[0006] In a first aspect of the present invention, a binocular camera-based OPM-MEG and MRI registration method is provided, comprising:

[0007] In the binocular camera calibration stage, the binocular camera is calibrated using the checkerboard calibration method to obtain calibration parameters;

[0008] In the subject image acquisition phase, binocular images of the subject wearing the magnetoencephalometric device are obtained;

[0009] In the 3D reconstruction stage, the acquired binocular images are subjected to epipolar correction, and the SGBM binocular stereo matching algorithm is used to obtain the initial disparity map. The initial disparity map is then subjected to Gaussian mask filling and median filtering to optimize the disparity map and generate a 3D point cloud. The MLS and Poisson surface reconstruction techniques are then used to reduce noise and smooth the 3D point cloud.

[0010] In the face point cloud registration stage, the point cloud is cut with the nose tip as the sphere center, and ICP registration is performed to obtain the transformation matrix from the camera coordinate system to the MRI coordinate system;

[0011] During the helmet positioning stage, the 3D coordinates of the three spherical targets on the helmet are detected through the spherical center detection algorithm, and the transformation matrix from the helmet coordinate system to the camera coordinate system is calculated, and finally the helmet coordinate system is converted to the MRI coordinate system.

[0012] Furthermore, in the binocular camera calibration stage, it also includes setting the working distance and baseline distance, using the camera calibrator software of MATLAB, and obtaining the calibration parameters by shooting the checkerboard.

[0013] Furthermore, in the 3D reconstruction stage, the Bouguet epipolar correction algorithm is used for epipolar correction, and the SGBM algorithm is used for binocular stereo matching to obtain the initial disparity map, which is then filled with Gaussian masks and processed with median filtering. Finally, the MLS and Poisson surface reconstruction techniques are used to denoise and smooth the point cloud.

[0014] Furthermore, in the face point cloud registration stage, dlib is used to detect the two-dimensional coordinates of the nose tip point, and the corresponding three-dimensional coordinates are obtained according to the projection matrix. Then, the point cloud is cut with the nose tip point as the center of the sphere, and ICP registration is performed to obtain the transformation matrix from the camera coordinate system to the MRI coordinate system.

[0015] Furthermore, during the rough registration of point clouds in the face point cloud registration stage, principal component analysis is performed on the reconstructed 3D face key points to extract the main direction vectors, and then the ICP algorithm is used to minimize the error between the two point clouds to obtain the initial transformation matrix.

[0016] Furthermore, in the helmet positioning stage, a spherical center positioning algorithm is used to detect the three-dimensional coordinates of the three spherical targets on the helmet, and a transformation matrix from the helmet coordinate system to the camera coordinate system is calculated based on a known helmet data file.

[0017] Furthermore, the sphere center positioning algorithm includes performing Canny edge detection on the image, performing distortion correction on the edge point coordinates, solving the sphere center imaging point coordinates by nonlinear optimization, and calculating the three-dimensional coordinates of the sphere center of the spherical target according to the epipolar correction projection matrix Q.

[0018] In a second aspect of the present invention, an OPM-MEG and MRI registration system based on a binocular camera is provided, comprising:

[0019] A binocular camera calibration module is used to calibrate the binocular camera using a checkerboard calibration method to obtain calibration parameters;

[0020] A subject image acquisition module, used to obtain binocular images of the subject wearing the magnetoencephalometric measurement device;

[0021] The 3D reconstruction module is used to perform epipolar correction on the acquired binocular images and obtain the initial disparity map using the SGBM binocular stereo matching algorithm. The initial disparity map is then subjected to Gaussian mask filling and median filtering to optimize the disparity map and generate a 3D point cloud. The MLS and Poisson surface reconstruction techniques are then used to perform noise reduction and smoothing on the 3D point cloud.

[0022] The face point cloud registration module is used to cut the point cloud with the nose tip as the center of the sphere and perform ICP registration to obtain the transformation matrix from the camera coordinate system to the MRI coordinate system;

[0023] The helmet positioning module is used to detect the three-dimensional coordinates of the three spherical targets on the helmet through the spherical center detection algorithm, and calculate the transformation matrix from the helmet coordinate system to the camera coordinate system, and finally transform the helmet coordinate system to the MRI coordinate system.

[0024] According to the above-mentioned embodiments of the present invention, at least the following beneficial effects are achieved: the OPM-MEG and MRI registration method based on the binocular camera of the present invention can improve the registration efficiency and accuracy of OPM-MEG. By using the checkerboard calibration method to accurately calibrate the binocular camera, accurate calibration parameters are obtained, and high-quality binocular images are obtained in the subject image acquisition stage. In the three-dimensional reconstruction stage, epipolar correction and SGBM binocular stereo matching algorithms are used, combined with Gaussian mask filling and median filtering, to optimize the disparity map and generate an accurate three-dimensional point cloud. Further using MLS and Poisson surface reconstruction surface reconstruction technology to reduce noise and smooth the point cloud can ensure the high quality of the point cloud data. In the face point cloud registration stage, the point cloud is cut with the nose tip as the center of the sphere, and ICP registration is performed to obtain an accurate transformation matrix from the camera coordinate system to the MRI coordinate system, which can achieve high-precision registration. In addition, this method adopts a spherical center detection algorithm in the helmet positioning stage. By detecting the three-dimensional coordinates of the three spherical targets on the helmet, the transformation matrix from the helmet coordinate system to the camera coordinate system is calculated, and finally the helmet coordinate system is transformed into the MRI coordinate system.

[0025] This method can effectively solve the problems of insufficient registration accuracy and low degree of automation in existing technologies, improve the efficiency and reliability of the registration process, and has important clinical significance for improving the diagnosis and treatment of brain diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0027] Figure 1 A schematic diagram of a flow chart of an OPM-MEG and MRI registration method based on a binocular camera provided in one embodiment of the present invention;

[0028] Figure 2 A schematic diagram of the structure of an OPM-MEG and MRI registration system based on a binocular camera provided in one embodiment of the present invention.

[0029] Figure 3 It is a schematic diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0030] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0031] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0032] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0033] Reference below Figure 1 , Figure 1 The following is a flow chart of an OPM-MEG and MRI registration method based on a binocular camera according to an embodiment of the present invention. Figure 1 As shown, an OPM-MEG and MRI registration method 100 based on a binocular camera includes:

[0034] Step 101, a binocular camera calibration phase, wherein the binocular camera is calibrated using a checkerboard calibration method to obtain calibration parameters;

[0035] Step 102, during the subject image acquisition phase, a binocular image of the subject wearing the magnetoencephalometric measurement device is acquired;

[0036] Step 103, 3D reconstruction stage, epipolar correction is performed on the acquired binocular image, and the SGBM binocular stereo matching algorithm is used to obtain the initial disparity map, and then the initial disparity map is subjected to Gaussian mask filling and median filtering to optimize the disparity map and generate a 3D point cloud, and then the MLS and Poisson surface reconstruction techniques are used to perform noise reduction and smoothing on the 3D point cloud;

[0037] Step 104, the face point cloud registration stage, cutting the point cloud with the nose tip as the sphere center, and performing ICP registration to obtain a transformation matrix from the camera coordinate system to the MRI coordinate system;

[0038] Step 105, the helmet positioning stage, detects the three-dimensional coordinates of the three ball targets on the helmet through the spherical center detection algorithm, and calculates the transformation matrix from the helmet coordinate system to the camera coordinate system, and finally converts the helmet coordinate system to the MRI coordinate system.

[0039] The OPM-MEG and MRI registration method of the present invention first includes a binocular camera calibration stage, which is to obtain the calibration parameters of the camera to ensure the accuracy of subsequent image acquisition and three-dimensional reconstruction. Here, a binocular camera refers to a system consisting of two cameras that simulate human eyes and obtain depth information by capturing two different perspectives of the same scene. Calibration is an important step in machine vision, which is used to determine the internal parameters (such as focal length, principal point coordinates) and external parameters (such as the position and posture of the camera relative to a reference coordinate system) of the camera.

[0040] Specifically, the calibration stage of the binocular camera can be achieved through the checkerboard calibration method, which is a commonly used camera calibration method that calculates camera parameters by shooting a checkerboard pattern with known grid sizes. In actual operation, you can use Matlab's camera calibrator software tool to first determine the side length of each grid on the checkerboard, then fix the camera, move the checkerboard to take 20 to 30 photos at different angles and distances, calibrate, and finally output the calibration parameters. This process needs to ensure that the checkerboard covers a sufficient spatial range and angle to obtain accurate calibration results.

[0041] Preferably, in the binocular camera calibration stage, the setting of working distance and baseline distance is also very important. Working distance refers to the distance from the camera to the object being photographed, while baseline distance refers to the distance between the two cameras. These two parameters directly affect the depth perception ability of the stereo vision system.

[0042] Furthermore, in this embodiment, the baseline distance is set to 30cm and the camera is 1.5m away from the person. Such a setting can ensure the accuracy of depth estimation while avoiding the loss of matching accuracy caused by too close a distance. In addition, uniform lighting conditions are also the key to obtaining high-quality stereo matching results. Therefore, during calibration and subsequent data collection, it is necessary to ensure uniform lighting in the shooting environment.

[0043] In some embodiments, during the binocular camera calibration stage, it also includes setting a working distance and a baseline distance, using MATLAB's camera calibrator software, and obtaining calibration parameters by photographing a checkerboard.

[0044] The 3D reconstruction stage in this embodiment is performed after the binocular camera calibration and subject image acquisition are completed. The purpose of this stage is to generate a 3D point cloud using binocular images, which is achieved through epipolar correction and binocular stereo matching algorithms. Here, epipolar correction refers to adjusting the images taken by the left and right cameras so that the corresponding image rows are parallel in space, which is an important step in improving matching accuracy in stereo vision. The SGBM binocular stereo matching algorithm is an advanced algorithm used to calculate the parallax between the left and right images to generate 3D information.

[0045] Specifically, the Bouguet epipolar correction algorithm is used for epipolar correction, and the SGBM algorithm is used for binocular stereo matching to obtain the initial disparity map, which is then filled with Gaussian masks and processed with median filters. Finally, the MLS and Poisson surface reconstruction techniques are used to denoise and smooth the point cloud.

[0046] When using the Bouguet epipolar correction algorithm for epipolar correction, first obtain the intrinsic parameter matrix obtained in the binocular camera calibration phase, including focal length, principal point coordinates, and extrinsic parameter matrix. Then calculate the rotation matrix and projection matrix of the left and right cameras based on these calibration parameters.

[0047] For the left camera, the correction rotation matrix and projection matrix are obtained, and the correction rotation matrix and projection matrix are calculated for the right camera in the same way. Then, each pixel in the left image and the right image is remapped using the calculated correction rotation matrix and projection matrix, and the new coordinates in the corrected image are calculated through the remapping function (based on the principle of projective geometry). It is necessary to traverse the entire image and process all pixels one by one.

[0048] In the remapping process, the coordinate values ​​of the pixels are adjusted according to the correction transformation matrix so that the epipolar lines are in the horizontal direction (i.e., the same row) in the corrected image. This involves complex coordinate transformation calculations, including determining the new pixel position based on the focal length, principal point coordinates, and rotation and translation information. Finally, the remapped pixels are recombined to generate the corrected left and right images, and the image boundary problem is processed. For pixels that exceed the boundaries of the original image, their pixel values ​​can be determined according to a specific interpolation method (such as bilinear interpolation) to ensure that the corrected image is complete and reasonable. After completing the epipolar correction, the corrected image can be used for subsequent operations such as the SGBM binocular stereo matching algorithm, providing a better image basis for 3D reconstruction.

[0049] Use the SGBM algorithm for binocular stereo matching to generate an initial disparity map. The SGBM algorithm is a semi-global block matching algorithm that can obtain a more accurate disparity map while maintaining high computational efficiency. After the initial disparity map is generated, it needs to be optimized, including Gaussian mask filling and median filtering, to eliminate noise and smooth the disparity map, thereby generating a higher quality 3D point cloud.

[0050] Preferably, the point cloud optimization process in the 3D reconstruction stage can be further refined. For example, Gaussian mask filling can be performed using windows of different sizes to accommodate holes of different sizes. The window size of the median filter can also be adjusted according to the actual disparity map quality.

[0051] Furthermore, in order to further improve the quality of the point cloud, the moving least squares (MLS) method and Poisson surface reconstruction technology can be used to further reduce noise and smooth the point cloud. MLS is a local surface fitting technology that can reduce noise while retaining the details of the point cloud. Poisson surface reconstruction is a gradient-based reconstruction method that can reconstruct a smooth surface from the point cloud.

[0052] In some embodiments, during the three-dimensional reconstruction stage, the Bouguet epipolar correction algorithm is used to perform epipolar correction, and the SGBM algorithm is used to perform binocular stereo matching to obtain an initial disparity map, and the disparity map is Gaussian mask filled and median filtered, and finally the point cloud is denoised and smoothed using MLS and Poisson surface reconstruction techniques.

[0053] The 3D reconstruction stage in this embodiment focuses on the post-processing of the disparity map. This step is to improve the quality of the disparity map and ensure the accuracy and reliability of the generated 3D point cloud data. Here, disparity map optimization refers to further processing of the initial disparity map to eliminate errors and discontinuities to obtain more accurate depth information. Gaussian mask filling is an image processing technique used to fill holes or missing parts in an image, while median filtering is a nonlinear filtering technique used to remove image noise and maintain edge information.

[0054] Specifically, the disparity map optimization process first involves filling the missing pixels in the disparity map using a Gaussian mask. The Gaussian mask is a weighted average filter whose weights are determined by a Gaussian distribution, which can effectively smooth the image and fill the holes.

[0055] More specifically, in this embodiment, the window size and standard deviation of the Gaussian mask can be adjusted according to the specific situation of the disparity map to achieve the best filling effect. Then, a median filter is used to suppress the remaining outliers and smooth the facial surface. The median filter removes noise by replacing each pixel value with the median value in its neighborhood. This method is very effective in maintaining edge information, especially in the extraction of facial features.

[0056] Preferably, the disparity map optimization process may also include other advanced image processing techniques. For example, bilateral filtering may be used to further smooth the disparity map while maintaining edge clarity. Bilateral filtering is a filtering method that considers both spatial proximity and pixel similarity, and is particularly suitable for smoothing while maintaining edge features.

[0057] Furthermore, adaptive threshold processing can be used to enhance the features in the disparity map. This method can dynamically adjust the threshold according to the local image characteristics to better handle the disparity map under different lighting and texture conditions. Through these optimization steps, the quality of the disparity map can be significantly improved, thereby providing more accurate data for subsequent 3D reconstruction and registration.

[0058] In some embodiments, during the face point cloud registration stage, dlib is used to detect the two-dimensional coordinates of the nose tip point, and the corresponding three-dimensional coordinates are obtained based on the projection matrix. The point cloud is then cut with the nose tip point as the center of the sphere, and ICP registration is performed to obtain the transformation matrix from the camera coordinate system to the MRI coordinate system.

[0059] The facial point cloud registration stage is a key step in the entire OPM-MEG and MRI registration method. Its purpose is to obtain the transformation matrix between the camera coordinate system and the MRI coordinate system by accurately aligning the three-dimensional point cloud of the face with the MRI data. In this stage, the nose tip point is selected as the reference point of the spherical center cutting point cloud as a significant and easily recognizable facial feature. Due to the deformation of the soft tissue of the face and the change of expression, direct full-face registration may have errors, so only the nose area with less change can be registered. ICP registration, or iterative closest point registration, is a method to find the optimal transformation by minimizing the distance between two sets of point clouds.

[0060] Specifically, the face point cloud registration stage first needs to use the dlib library to detect the two-dimensional coordinates of the nose tip. dlib is a C++ library that contains modern machine learning algorithms. It provides a series of tools to process images and perform facial feature detection. In the left-eye image, the position of the nose tip can be accurately found through the dlib face detection algorithm. Then, using the projection matrix Q after epipolar correction, the two-dimensional coordinates can be converted to the corresponding three-dimensional coordinates. This projection matrix contains the intrinsic and extrinsic information of the camera, and can convert image coordinates into three-dimensional space coordinates.

[0061] Preferably, the facial point cloud registration stage can be further refined. In the coarse registration stage, the main direction vectors of the facial key points can be extracted by principal component analysis (PCA). PCA is a statistical method that can determine the main directions of variation in the data and project the data in these directions to reduce the dimension.

[0062] Furthermore, PCA can help determine the orientation of the face and ensure that it is basically aligned with the standard orientation of MRI during the initial registration. In the fine registration stage, the iterative closest point (ICP) algorithm can be used to minimize the error between the two point clouds. During the execution of the ICP algorithm, the registration results can be optimized by adjusting the number of iterations and setting a suitable threshold. In addition, a multi-resolution strategy can be considered to accelerate the convergence of the ICP algorithm and improve the robustness of the registration.

[0063] In some embodiments, during the ICP registration stage, principal component analysis is performed on the reconstructed three-dimensional facial key points to extract the main direction vectors, and then the ICP algorithm is used to minimize the error between the two point clouds to obtain a transformation matrix.

[0064] This implementation describes the specific steps of the face point cloud registration stage, which is to obtain the transformation matrix from the camera coordinate system to the MRI coordinate system by cutting the point cloud with the nose tip as the center of the sphere for ICP registration. In this context, the nose tip point refers to the most prominent point of the nose in the middle of the face, and its position in three-dimensional space is crucial for facial recognition and registration. ICP registration, also known as iterative closest point algorithm, is an algorithm that finds the best match between two sets of point clouds by minimizing the distance between them.

[0065] Specifically, the implementation of the face point cloud registration stage involves the use of specific software tools to detect and process three-dimensional point cloud data. First, the location of the nose tip point needs to be determined in the MRI data, which can usually be achieved by finding the maximum value point along the y-axis. For the three-dimensional point cloud of the face reconstructed from the binocular camera system, the two-dimensional coordinates of the nose tip point can be detected by an image processing library such as dlib, and then these two-dimensional coordinates are converted into three-dimensional coordinates using the projection matrix Q after epipolar correction. This process involves segmenting the point cloud data, centered on the nose tip point, and extracting the corresponding nose area point cloud in preparation for subsequent ICP registration.

[0066] Preferably, some advanced image processing and point cloud processing techniques can be used in the face point cloud registration stage to improve the accuracy and efficiency of registration. For example, when detecting the two-dimensional coordinates of the nose tip point, the detection results of multiple facial feature points can be combined to improve the accuracy of positioning. In the point cloud segmentation process, a reasonable search radius can be set to more accurately extract point cloud data around the nose tip point.

[0067] Furthermore, when performing ICP registration, a multi-resolution strategy can be adopted to first converge quickly at a lower resolution and then make fine adjustments at a higher resolution to improve the robustness of the algorithm and the quality of registration.

[0068] In some embodiments, during the helmet positioning stage, a spherical center positioning algorithm is used to detect the three-dimensional coordinates of three spherical targets on the helmet, and a transformation matrix from the helmet coordinate system to the camera coordinate system is calculated based on a known helmet data file.

[0069] The helmet positioning stage is one of the key steps in the entire OPM-MEG and MRI registration method. The purpose of this stage is to accurately detect the three-dimensional coordinates of the ball target on the helmet and then calculate the transformation matrix of the helmet coordinate system relative to the camera coordinate system. In this context, the spherical center positioning algorithm refers to a calculation method for determining the center position of a sphere, and the spherical target refers to a spherical marker fixed on the helmet for positioning.

[0070] Specifically, the helmet positioning stage begins with detecting the three-dimensional coordinates of the three ball targets on the helmet using a spherical center positioning algorithm. This step involves performing Canny edge detection on the image, which is a popular edge detection algorithm that can identify linear structures in the image. Next, the edge point coordinates are subjected to distortion correction, which usually involves correcting the image distortion caused by lens distortion. Then, the coordinates of the spherical center imaging point are solved by nonlinear optimization. This step may require the use of least squares or other optimization algorithms to accurately determine the position of the sphere center. Finally, the three-dimensional coordinates of the spherical target are calculated based on the epipolar correction projection matrix Q, which involves the process of converting the two-dimensional image coordinates into three-dimensional space coordinates.

[0071] Preferably, the operation steps of the helmet positioning stage can be further refined. For example, when performing Canny edge detection, different thresholds can be selected to balance the number of detected edges and the noise level. During the distortion correction process, the distortion parameters obtained in the camera calibration stage can be used to adjust the edge point coordinates.

[0072] Furthermore, when solving the sphere center coordinates by nonlinear optimization, different initial guess values ​​and optimization algorithm parameters can be set to improve the accuracy and convergence speed of the solution. In addition, it is also possible to consider using multiple camera perspectives to improve the robustness of sphere center positioning, or use machine learning methods to automatically identify and track the ball target, thereby providing more accurate three-dimensional coordinate information.

[0073] In some embodiments, the sphere center positioning algorithm includes performing Canny edge detection on the image, performing distortion correction on the edge point coordinates, solving the sphere center imaging point coordinates by nonlinear optimization, and calculating the three-dimensional coordinates of the sphere center of the spherical target based on the epipolar correction projection matrix Q.

[0074] This implementation describes how to optimize the processing flow of point cloud data by principal component analysis (PCA) during the ICP registration stage. The purpose of this step is to extract the main direction vectors of the point cloud data so that the two point cloud data sets can be aligned more accurately during registration. Here, principal component analysis is a statistical method used to identify the main directions of change in the data and to reduce the dimensionality of the data in these directions.

[0075] Specifically, the ICP registration stage first involves principal component analysis of the reconstructed 3D facial key points. This process involves collecting the coordinate data of the key points and then using the PCA algorithm to find the main directions of change in the data, namely the principal components. These principal components correspond to the directions with the largest changes in the point cloud dataset and can be used to guide the alignment process of the point cloud data.

[0076] More specifically, in actual operation, PCA can be implemented by calculating the covariance matrix and performing eigenvalue decomposition on it. The corresponding eigenvector with the largest eigenvalue is the first principal component, which indicates the main change trend in the point cloud data.

[0077] Preferably, the PCA process in the ICP registration stage can be combined with other preprocessing steps. For example, before performing PCA, the point cloud data can be denoised to reduce the impact of noise on the principal component analysis results. In addition, data normalization can be used to ensure that the data in each dimension has the same scale, thereby avoiding excessive impact of some dimensions on the PCA results due to their large value range.

[0078] Furthermore, after PCA, you can choose to retain a certain number of principal components for subsequent registration calculations. This number can be determined based on the complexity of the point cloud data and the required accuracy. For example, you can choose to retain the principal component of 95% energy, which means that only those eigenvectors with a cumulative contribution rate of 95% are retained.

[0079] In some embodiments, the generation of the three-dimensional point cloud includes converting a disparity map obtained by an SGBM algorithm into a three-dimensional point cloud, and performing MLS and Poisson surface reconstruction processing on the generated three-dimensional point cloud.

[0080] It should be noted that this embodiment describes in detail the specific steps of generating a 3D point cloud, including converting the disparity map obtained by the SGBM algorithm into a 3D point cloud, and performing subsequent moving least squares (MLS) and Poisson surface reconstruction processing on the generated 3D point cloud. In this context, the moving least squares (MLS) method is a technique for smoothing and interpolating point cloud data, which can reduce noise while retaining point cloud details; Poisson surface reconstruction is a method for generating a smooth surface based on point cloud data, which uses the gradient information of the point cloud data to construct a continuous surface.

[0081] Specifically, the generation of a 3D point cloud first involves using the SGBM algorithm to process the corrected binocular image to generate a disparity map. The disparity map reflects the pixel-level displacement between corresponding points in the left and right images, and this displacement information can be used to calculate the depth of each point in the scene. After optimizing the initial disparity map, the depth information can be converted into a 3D space coordinate by converting the disparity map into a depth map, and then combined with the camera's intrinsic and extrinsic parameters to generate the initial point cloud data. Next, the MLS algorithm is applied to these point cloud data for smoothing. The MLS algorithm achieves a smoothing effect by fitting a polynomial surface in the local neighborhood of each point, and then estimating the position of the point based on this surface. Finally, the Poisson surface reconstruction algorithm is used to reconstruct the surface of the point cloud, which uses the gradient information of the point cloud data to construct a continuous surface that can better approximate the original geometric shape.

[0082] Preferably, the generation and processing steps of the 3D point cloud can be further refined. For example, in the process of converting the disparity map to the depth map, a depth correction step can be introduced to compensate for the depth error caused by the camera lens distortion or other factors. When applying the MLS algorithm, different neighborhood sizes and the order of the fitting polynomial can be selected to balance the smoothing effect and detail retention.

[0083] Furthermore, for Poisson surface reconstruction, the regularization parameters in the reconstruction process can be adjusted to control the smoothness of the reconstructed surface. In addition, it is also possible to consider introducing additional steps between MLS and Poisson surface reconstruction, such as removing outliers or using feature-preserving filters, to further improve the quality of the point cloud.

[0084] The above-mentioned embodiments of the present invention have the following beneficial effects: The registration method described in the present invention can bring significant technological progress to the field of medical imaging. First, by using the checkerboard calibration method to accurately calibrate the binocular camera, it can provide a basis for the subsequent high-precision three-dimensional reconstruction. In the three-dimensional reconstruction stage, the SGBM binocular stereo matching algorithm combined with Gaussian mask filling and median filtering can be used to optimize the disparity map and generate a high-quality three-dimensional point cloud, and then the point cloud is denoised and smoothed by MLS and Poisson surface reconstruction technology, so as to obtain a more refined and accurate three-dimensional model.

[0085] In addition, in the face point cloud registration stage mentioned in the claims, by using the nose tip point as the sphere center to cut the point cloud and perform ICP registration, the camera coordinate system can be effectively aligned with the MRI coordinate system, thereby obtaining an accurate transformation matrix. In the helmet positioning stage, the spherical target on the helmet is detected by the sphere center detection algorithm and the transformation matrix from the helmet coordinate system to the camera coordinate system is calculated, which can ensure the accurate positioning of the helmet and ultimately achieve accurate conversion from the helmet coordinate system to the MRI coordinate system. The combination of these technologies can improve the registration efficiency and accuracy of OPM-MEG, which has important clinical value for improving the diagnosis and treatment of brain diseases.

[0086] like Figure 2 As shown, in some embodiments, an OPM-MEG and MRI registration system 200 based on a binocular camera is implemented, and the system 200 includes:

[0087] A binocular camera calibration module 201 is used to calibrate the binocular camera using a checkerboard calibration method to obtain calibration parameters;

[0088] The subject image acquisition module 202 is used to obtain binocular images of the subject wearing the magnetoencephalography measurement device;

[0089] The 3D reconstruction module 203 is used to perform epipolar correction on the acquired binocular images, and obtain an initial disparity map using the SGBM binocular stereo matching algorithm, and then perform Gaussian mask filling and median filtering on the initial disparity map to optimize the disparity map and generate a 3D point cloud, and then use MLS and Poisson surface reconstruction techniques to perform noise reduction and smoothing on the 3D point cloud;

[0090] The face point cloud registration module 204 is used to cut the point cloud with the nose tip as the sphere center and perform ICP registration to obtain a transformation matrix from the camera coordinate system to the MRI coordinate system;

[0091] The helmet positioning module 205 is used to detect the three-dimensional coordinates of the three ball targets on the helmet through a spherical center detection algorithm, and calculate the transformation matrix from the helmet coordinate system to the camera coordinate system, and finally transform the helmet coordinate system to the MRI coordinate system.

[0092] It is understandable that the modules described in the OPM-MEG and MRI registration system 200 based on binocular camera are similar to those described in the reference Figure 1 The steps in the described OPM-MEG and MRI registration method based on a binocular camera correspond to each other. Therefore, the operations, features and beneficial effects described above for the OPM-MEG and MRI registration method based on a binocular camera are also applicable to the OPM-MEG and MRI registration system 200 based on a binocular camera and the modules contained therein, and will not be repeated here.

[0093] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention to form a technical solution.

Claims

1. A binocular camera-based OPM-MEG and MRI registration method, characterized in that The following steps are involved: In the binocular camera calibration stage, the binocular camera is calibrated using the checkerboard calibration method to obtain calibration parameters; In the subject image acquisition phase, binocular images of the subject wearing the magnetoencephalometric device are obtained; In the 3D reconstruction stage, the acquired binocular images are subjected to epipolar correction, and the SGBM binocular stereo matching algorithm is used to obtain the initial disparity map. The initial disparity map is then subjected to Gaussian mask filling and median filtering to optimize the disparity map and generate a 3D point cloud. The MLS and Poisson surface reconstruction techniques are then used to reduce noise and smooth the 3D point cloud. In the face point cloud registration stage, the point cloud is cut with the nose tip as the sphere center, and ICP registration is performed to obtain the transformation matrix from the camera coordinate system to the MRI coordinate system; During the helmet positioning stage, the 3D coordinates of the three spherical targets on the helmet are detected through the spherical center detection algorithm, and the transformation matrix from the helmet coordinate system to the camera coordinate system is calculated, and finally the helmet coordinate system is converted to the MRI coordinate system.

2. The method according to claim 1, characterized in that In the binocular camera calibration stage, the working distance and baseline distance are also set, and the calibration parameters are obtained by shooting the checkerboard using the MATLAB camera calibrator software.

3. The method according to claim 1, characterized in that In the 3D reconstruction stage, the Bouguet epipolar correction algorithm is used for epipolar correction, and the SGBM algorithm is used for binocular stereo matching to obtain the initial disparity map, which is then filled with Gaussian masks and processed with median filters. Finally, the MLS and Poisson surface reconstruction techniques are used to denoise and smooth the point cloud.

4. The method according to claim 1, characterized in that: In the face point cloud registration stage, dlib is used to detect the two-dimensional coordinates of the nose tip point, and the corresponding three-dimensional coordinates are obtained according to the projection matrix. Then, the point cloud is cut with the nose tip point as the center of the sphere, and ICP registration is performed to obtain the transformation matrix from the camera coordinate system to the MRI coordinate system.

5. The method according to claim 1, characterized in that During the rough registration of point clouds in the face point cloud registration stage, principal component analysis is performed on the reconstructed 3D face key points to extract the main direction vectors, and then the ICP algorithm is used to minimize the error between the two point clouds to obtain the initial transformation matrix.

6. The method according to claim 1, characterized in that In the helmet positioning stage, the spherical center positioning algorithm is used to detect the three-dimensional coordinates of the three spherical targets on the helmet, and the transformation matrix from the helmet coordinate system to the camera coordinate system is calculated based on the known helmet data file.

7. The method according to claim 6, characterized in that The sphere center positioning algorithm includes Canny edge detection on the image, distortion correction on the edge point coordinates, nonlinear optimization to solve the sphere center imaging point coordinates, and the three-dimensional coordinates of the sphere center according to the polar line correction projection matrix Q.

8. An OPM-MEG and MRI registration system based on a binocular camera, characterized in that: include: A binocular camera calibration module is used to calibrate the binocular camera using a checkerboard calibration method to obtain calibration parameters; A subject image acquisition module, used to obtain binocular images of the subject wearing the magnetoencephalometric measurement device; The 3D reconstruction module is used to perform epipolar correction on the acquired binocular images and obtain the initial disparity map using the SGBM binocular stereo matching algorithm. The initial disparity map is then subjected to Gaussian mask filling and median filtering to optimize the disparity map and generate a 3D point cloud. The MLS and Poisson surface reconstruction techniques are then used to perform noise reduction and smoothing on the 3D point cloud. The face point cloud registration module is used to cut the point cloud with the nose tip as the center of the sphere and perform ICP registration to obtain the transformation matrix from the camera coordinate system to the MRI coordinate system; The helmet positioning module is used to detect the three-dimensional coordinates of the three spherical targets on the helmet through the spherical center detection algorithm, and calculate the transformation matrix from the helmet coordinate system to the camera coordinate system, and finally transform the helmet coordinate system to the MRI coordinate system.

Images