Head MRI registration method, system, and program product based on a 3D scanner

Abstract

The application relates to the technical field of transcranial magnetic stimulation target positioning, and provides a head MRI registration method, system and program product based on a 3D scanner, which obtains MRI three-dimensional data of a patient; performs facial scanning on the patient through a 3D scanner to obtain scanning data, and generates three-dimensional model data according to the scanning data; the scanning data comprises a depth map; coordinates of an MRI space and a 3D scanner are subjected to coordinate conversion according to the MRI three-dimensional data and the three-dimensional model data, so that a first conversion matrix is obtained; a face region on the depth map is cropped to obtain a face partial region; the face partial region and the MRI three-dimensional data are registered according to the first conversion matrix, the MRI three-dimensional data and three-dimensional model data of the face partial region, so that a second conversion matrix is obtained; therefore, the accuracy and efficiency of head MRI registration can be improved through the 3D scanner, and the accuracy and efficiency of transcranial magnetic stimulation treatment are improved.

Description

Technical Field

[0001] This application relates to the field of transcranial magnetic stimulation therapy technology, and in particular to a head MRI registration method, system, and program product based on a 3D scanner. Background Technology

[0002] During transcranial magnetic stimulation (TMS) therapy using a robotic arm, the end effector of the magnetic therapy device needs to be guided to the target location on the patient using an optical locator. During treatment, head displacement may occur, affecting the treatment outcome. The optical locator tracks the patient's head displacement in real time to improve the treatment effect. Before guidance, registration of the patient's magnetic resonance imaging (MRI) data with the patient's actual physical space is required. This necessitates three-dimensional reconstruction of the patient's MRI data, followed by spatial registration of the reconstructed three-dimensional facial data with the patient's actual facial data during treatment.

[0003] The existing head registration process involves marking facial feature points in MRI space, then using the marked facial feature points to acquire corresponding feature points on the patient's face with a probe. The doctor uses the probe to acquire feature points on the patient's face. By combining the marked facial feature points with the feature points acquired by the probe at preset locations, the coordinates of the MRI space and the physical space of the actual patient are transformed, thus unifying the spatial coordinates of the MRI data and the spatial coordinates of the actual patient.

[0004] The disadvantage of this registration process is that when doctors use the acquisition probe to collect feature points on the patient's head, it is difficult to accurately acquire the points that correspond to the MRI space. Moreover, registration requires high skill from the doctor, the registration effect is not ideal, the registration time is long, and the registration efficiency is low. Summary of the Invention

[0005] In order to solve or at least partially solve the above-mentioned technical problems, this application provides a head MRI registration method, system and program product based on a 3D scanner, which can avoid errors caused by human acquisition, achieve more accurate registration, and improve the speed of head MRI registration.

[0006] In a first aspect, this application provides a head MRI registration method based on a 3D scanner, including:

[0007] Acquire the patient's MRI three-dimensional data;

[0008] A 3D scanner is used to scan the patient's face to obtain scan data, and a three-dimensional model data is generated based on the scan data; the scan data includes a depth map.

[0009] Based on the MRI three-dimensional data and the three-dimensional model data, coordinate transformation is performed between the MRI spatial coordinates and the 3D scanner spatial coordinates to obtain the first transformation matrix;

[0010] The face region on the depth map is cropped to obtain the face portion region;

[0011] The face region and the MRI three-dimensional data are registered based on the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix.

[0012] Secondly, this application provides a head MRI registration system based on a 3D scanner, comprising:

[0013] The acquisition module is used to acquire the patient's MRI three-dimensional data;

[0014] The acquisition module is further configured to perform a facial scan on the patient using a 3D scanner to obtain scan data, and generate three-dimensional model data based on the scan data; the scan data includes a depth map;

[0015] The registration module is used to perform coordinate transformation between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix.

[0016] The cropping module is used to crop the face region on the depth map to obtain the face portion region;

[0017] The registration module is further configured to register the face region and the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data and the three-dimensional model data of the face region, to obtain a second transformation matrix.

[0018] Thirdly, embodiments of this application also provide a head MRI registration system based on a 3D scanner, including a processor and a memory; and one or more programs stored in the memory and configured to be executed by the processor, the programs including steps for the head MRI registration method based on a 3D scanner as described in the first aspect.

[0019] Fourthly, embodiments of this application also provide a computer program product, which, when executed by a computer, enables the computer to perform the head MRI registration method based on a 3D scanner as described in the first aspect or the head MRI navigation method as described in the second aspect.

[0020] The head MRI registration method based on a 3D scanner according to the embodiments of this application involves acquiring the patient's MRI three-dimensional data; performing a facial scan on the patient using a 3D scanner to obtain scan data; generating three-dimensional model data based on the scan data; the scan data including a depth map; performing coordinate transformation between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; cropping the facial region on the depth map to obtain a partial facial region; and registering the partial facial region and the MRI three-dimensional data based on the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the partial facial region to obtain a second transformation matrix. Therefore, the accuracy and efficiency of head MRI registration can be improved using a 3D scanner, thereby improving the accuracy and efficiency of transcranial magnetic stimulation (TMS) therapy. Attached Figure Description

[0021] To more clearly illustrate the embodiments of this application, the relevant drawings will be briefly described below. It is understood that the drawings described below are only for illustrating some embodiments of this application, and those skilled in the art can obtain many other technical features and connections not mentioned herein based on these drawings.

[0022] Figure 1 A schematic flowchart of a head MRI registration system provided for embodiments of this application;

[0023] Figure 2 This is a schematic diagram illustrating how a first facial feature point is obtained by performing facial recognition on MRI three-dimensional data using a preset facial recognition model, as provided in an embodiment of this application.

[0024] Figure 3 This is a schematic diagram illustrating how a second facial feature point is obtained by performing facial recognition on three-dimensional model data using a preset facial recognition model, as provided in an embodiment of this application.

[0025] Figure 4 A schematic diagram illustrating a rectangular region of a face captured from an RGB image, provided as an embodiment of this application.

[0026] Figure 5 A schematic diagram illustrating how a rectangular region of a face is mapped to a depth map, provided for the purpose of implementing this application.

[0027] Figure 6 This is a schematic diagram illustrating how a rectangular region of a face in an RGB image is mapped to MRI 3D data during fine registration, as provided in an embodiment of this application.

[0028] Figure 7 A schematic flowchart of another head MRI registration method provided for an embodiment of this application;

[0029] Figure 8 A schematic diagram of the structure of a head MRI registration system provided for an embodiment of this application;

[0030] Figure 9 This is a schematic diagram of the structure of a head MRI registration system provided for an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0033] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0034] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0035] Example 1

[0036] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a head MRI registration method based on a 3D scanner, provided as an embodiment of this application. One embodiment of this application proposes a head MRI registration method based on a 3D scanner, which includes:

[0037] 101. Obtain the patient's MRI three-dimensional data.

[0038] Among them, magnetic resonance imaging (MRI) data of the subjects can be collected using a magnetic resonance imaging instrument, and MRI three-dimensional data can be generated based on the MRI data.

[0039] 102. Perform a facial scan on the patient using a 3D scanner to obtain scan data, and generate three-dimensional model data based on the scan data.

[0040] The scan data includes GRB maps, depth maps, and point clouds corresponding to the depth maps. Specifically, the 3D scanner can be registered with the optical locator so that the origins of the 3D scanner and the optical locator are in the same coordinate system to determine the relative positional relationship between the 3D scanner and the optical locator. The 3D scanner is used to scan the patient's face to obtain scan data. Based on the scan data, a first three-dimensional model data is generated to mark the feature points corresponding to the MRI three-dimensional data in the model.

[0041] 103. Based on the MRI three-dimensional data and the three-dimensional model data, perform coordinate transformation on the MRI spatial coordinates and the 3D scanner spatial coordinates to obtain the first transformation matrix.

[0042] The process of performing coordinate transformation to obtain the first transformation matrix can be understood as coarse registration, the purpose of which is to maximize the overlap between the two sets of data through the transformation matrix (rotation and translation).

[0043] Specifically, feature points can be labeled where the MRI 3D data and the 3D model data have the same characteristics, and then the matrix transformation of the two sets of data can be obtained through the transformation of feature points.

[0044] 104. Cropping the face region on the depth map to obtain the face portion region.

[0045] By cropping the face region on the depth map to obtain the face region, and then performing fine registration on the face region, the registration speed and accuracy can be improved.

[0046] 105. Based on the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region, the face region and the MRI three-dimensional data are registered to obtain the second transformation matrix.

[0047] The first transformation matrix is ​​used to determine the initial position for registering the MRI 3D data and the 3D model data of the face region. The process of registering to obtain the second transformation matrix can be understood as fine registration. In this scheme, coarse registration is used to initially align the two sets of data (MRI 3D data and 3D model data of the face region) to find a rough matching relationship, providing a better initial position for subsequent precise registration. Further, through fine registration, the optimal transformation matrix is ​​calculated to minimize the distance between corresponding point pairs in the two spaces (MRI space and 3D scanner space) to achieve high-precision registration.

[0048] Specifically, the initial position for registering the MRI three-dimensional data and the three-dimensional model data of the face region can be determined according to the first transformation matrix; then, the three-dimensional model data and the MRI three-dimensional data of the face region are registered according to the initial position to obtain the second transformation matrix, thereby achieving fine registration.

[0049] The head MRI registration method based on a 3D scanner provided in this application involves acquiring the patient's MRI three-dimensional data; performing a facial scan on the patient using a 3D scanner to obtain scan data; generating three-dimensional model data based on the scan data; the scan data including a depth map; performing coordinate transformation between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; cropping the facial region on the depth map to obtain a partial facial region; and registering the partial facial region and the MRI three-dimensional data based on the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the partial facial region to obtain a second transformation matrix. Therefore, the accuracy and efficiency of head MRI registration can be improved using a 3D scanner, thereby improving the accuracy and efficiency of transcranial magnetic stimulation (TMS) therapy.

[0050] Optionally, acquiring the patient's MRI three-dimensional data includes:

[0051] Acquire the patient's magnetic resonance imaging data;

[0052] The magnetic resonance imaging data is reconstructed in three dimensions to obtain MRI three-dimensional data.

[0053] Magnetic resonance brain imaging can include functional magnetic resonance imaging (fMRI) and structural magnetic resonance imaging (sMRI). Three-dimensional reconstruction can be performed based on fMRI and sMRI to obtain MRI three-dimensional data.

[0054] Optionally, the step of performing coordinate transformation between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix includes:

[0055] The first facial feature point is obtained by performing facial recognition on the MRI three-dimensional data using a preset facial recognition model.

[0056] The preset face recognition model is used to identify feature points in the three-dimensional model data to obtain a second face feature point corresponding to the first face feature point in the MRI three-dimensional data.

[0057] Based on the first and second facial feature points, coordinate transformation is performed between the MRI spatial coordinates and the 3D scanner spatial coordinates to obtain the first transformation matrix.

[0058] like Figure 2 The diagram illustrates a method for obtaining first facial feature points by performing facial recognition on MRI 3D data using a pre-set facial recognition model. This solution, by employing a facial recognition model, can mark facial feature points on MRI 3D models; as shown... Figure 3 The diagram shows a method for performing face recognition on 3D model data using a preset face recognition model to obtain second face feature points. This method uses a face recognition model to mark face feature points on 3D models scanned by a 3D scanner. Through coarse registration, two sets of data (MRI 3D data and second 3D model data) are initially aligned to find a rough matching relationship, providing a better initial position for subsequent precise registration.

[0059] By using a transformation matrix (rotation and translation), the overlap between the two sets of data can be maximized. Feature points can be labeled at locations with the same characteristics in the two sets of data (second 3D model data and MRI 3D data). Then, the matrix transformation between the two sets of data is obtained through the transformation of feature points, resulting in the first transformation matrix.

[0060] Specifically, a face recognition model can be pre-trained by acquiring three-dimensional point cloud sample data of a face through a 3D scanner; acquiring MRI point cloud sample data; and training the three-dimensional point cloud sample data and MRI point cloud sample data using a PointNet network model to obtain the pre-trained face recognition model.

[0061] In this solution, facial feature point recognition can be performed using an AI model. Specifically, a preset facial recognition model is used to perform facial recognition on the MRI 3D data to obtain a first facial feature point. The preset facial recognition model is then used to perform feature point recognition on the 3D model data to obtain a second facial feature point corresponding to the first facial feature point in the MRI 3D data. Subsequently, the first and second facial feature points are used for coarse registration to obtain a first transformation matrix. By using the preset facial recognition model, facial feature points can be identified more accurately and used for coarse registration.

[0062] Optionally, the scan data includes a depth map; the step of registering the face region and the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix includes:

[0063] The initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region is determined based on the first transformation matrix;

[0064] Point cloud sampling and preprocessing are performed on the MRI 3D data and the 3D model data of the face region to obtain the preprocessed target point cloud and input point cloud; the input point cloud corresponds to the 3D model data of the face region, and the target point cloud corresponds to the MRI 3D data;

[0065] Based on the initial position, point cloud matching is performed between the target point cloud set and the input point cloud set. Specifically, a transformation matrix is ​​constructed to transform the point cloud of the input point cloud set to the coordinate system corresponding to the target point cloud set. The error function between the transformed source and target point clouds is estimated. When the error function is determined to converge, a second transformation matrix is ​​obtained. The target point cloud set P = {p1, p2, ..., p...} can be defined as follows: n} and the input point cloud X = {x1, x2, ..., x} n Perform point cloud matching, construct the transformation matrix (R, t), and the error function is:

[0066] .

[0067] Where N is the number of point clouds of the target point.

[0068] Optionally, the step of registering the face region and the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix includes:

[0069] The initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region is determined based on the first transformation matrix;

[0070] Point cloud sampling and preprocessing are performed on the MRI 3D data and the 3D model data of the face region to obtain the preprocessed target point cloud and input point cloud; the input point cloud corresponds to the 3D model data of the face region, and the target point cloud corresponds to the MRI 3D data;

[0071] Based on the initial position, point cloud matching is performed between the target point cloud set and the input point cloud set. Specifically, a transformation matrix is ​​constructed to transform the point cloud set of the input point cloud set to the coordinate system corresponding to the target point cloud set. The error function between the transformed source point cloud and the target point cloud is estimated. When the error function is determined to converge, the second transformation matrix is ​​obtained.

[0072] The target point set P = {p1, p2, ..., p...} can be defined as follows: n} and the input point cloud X = {x1, x2, ..., x} n Perform point cloud matching, construct the transformation matrix (R, t), and the error function is:

[0073] .

[0074] Where N is the number of point clouds of the target point.

[0075] Optionally, determining the initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region based on the first transformation matrix includes:

[0076] The MRI 3D data and the 3D model data of the face region are rotated and translated according to the first transformation matrix to successfully match the MRI 3D data and the 3D model data of the face region, thereby obtaining the initial position.

[0077] Optionally, the scan data further includes an RGB image, and the step of cropping the face region on the depth image to obtain the face portion region includes:

[0078] Capture the rectangular region of the face from the RGB image;

[0079] The rectangular region of the face is mapped onto the depth map to obtain the face portion region of the depth map rectangle.

[0080] like Figure 4 The image shown is a schematic diagram illustrating the process of capturing a rectangular region of a face from an RGB image. Figure 5 The image shown is a schematic diagram illustrating the mapping of a rectangular region of a face onto a depth map; as shown... Figure 6The diagram illustrates how a rectangular region of a face in an RGB image is mapped onto MRI 3D data during fine registration. By acquiring a rectangular region of a face from an RGB image and mapping it to the depth map, the rectangular region of the face in the depth map is obtained. Fine registration of this face region can improve both registration speed and accuracy.

[0081] Example 2

[0082] like Figure 7 As shown, one embodiment of this application proposes another head MRI registration method based on a 3D scanner. Figure 7 This application provides a schematic flowchart of another head MRI registration method based on a 3D scanner, which includes:

[0083] 201. Obtain the patient's magnetic resonance imaging data.

[0084] 202. Perform three-dimensional reconstruction on the magnetic resonance imaging data to obtain MRI three-dimensional data.

[0085] 203. Perform a facial scan on the patient using a 3D scanner to obtain scan data, and generate three-dimensional model data based on the scan data; the scan data includes a depth map and an RGB map.

[0086] 204. Perform face recognition on the MRI three-dimensional data using a preset face recognition model to obtain the first face feature points.

[0087] 205. Using the preset face recognition model, feature points are identified in the three-dimensional model data to obtain second face feature points corresponding to the first face feature points in the MRI three-dimensional data.

[0088] 206. Based on the first facial feature points and the second facial feature points, perform coordinate transformation on the MRI spatial coordinates and the 3D scanner spatial coordinates to obtain the first transformation matrix.

[0089] 207. Collect the rectangular area of ​​the face from the RGB image.

[0090] 208. Map the rectangular region of the face onto the depth map to obtain the face portion region of the depth map rectangle.

[0091] 209. Determine the initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region based on the first transformation matrix.

[0092] 210. Perform point cloud sampling and preprocessing on the MRI three-dimensional data and the three-dimensional model data of the face region to obtain the preprocessed target point cloud and input point cloud.

[0093] 211. Based on the initial position, perform point cloud matching on the target point cloud set and the input point cloud set, wherein the point cloud of the input point cloud set is transformed to the coordinate system corresponding to the target point cloud set by constructing a transformation matrix, the error function between the source point cloud and the target point cloud after transformation is estimated, and when the error function is determined to converge, the second transformation matrix is ​​obtained.

[0094] In this embodiment, a first facial feature point is obtained by performing facial recognition on MRI 3D data using a preset facial recognition model; a second facial feature point is obtained by performing feature point recognition on 3D model data using the same preset facial recognition model, corresponding to the first facial feature point in the MRI 3D data; a coordinate transformation is performed between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the first and second facial feature points, resulting in a first transformation matrix; the initial position for registering the MRI 3D data and the 3D model data of the facial region is determined based on the first transformation matrix; point cloud sampling and preprocessing are performed on the MRI 3D data and the 3D model data of the facial region to obtain a preprocessed target point cloud and an input point cloud; point cloud matching is performed on the target point cloud and the input point cloud based on the initial position, wherein the point cloud of the input point cloud is transformed to the coordinate system corresponding to the target point cloud by constructing a transformation matrix, and the error function between the transformed source point cloud and the target point cloud is estimated. When the error function is determined to converge, a second transformation matrix is ​​obtained, thereby improving the accuracy and speed of registration.

[0095] Example 3

[0096] like Figure 8 , Figure 8 The head MRI registration system 300 based on a 3D scanner provided in the application includes:

[0097] Acquisition module 301 is used to acquire the patient's MRI three-dimensional data;

[0098] The acquisition module 301 is further configured to perform a facial scan on the patient using a 3D scanner to obtain scan data, and generate three-dimensional model data based on the scan data; the scan data includes a depth map;

[0099] The registration module 302 is used to perform coordinate transformation between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix.

[0100] The cropping module 303 is used to crop the face region on the depth map to obtain a partial face region;

[0101] The registration module 302 is further configured to register the face region and the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data and the three-dimensional model data of the face region to obtain a second transformation matrix.

[0102] Optionally, in acquiring the patient's MRI three-dimensional data, the acquisition module 301 is specifically used for:

[0103] Acquire the patient's magnetic resonance imaging data;

[0104] The magnetic resonance imaging data is reconstructed in three dimensions to obtain MRI three-dimensional data.

[0105] Optionally, in the step of performing coordinate transformation between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix, the registration module 302 is specifically used for:

[0106] The first facial feature point is obtained by performing facial recognition on the MRI three-dimensional data using a preset facial recognition model.

[0107] The preset face recognition model is used to identify feature points in the three-dimensional model data to obtain a second face feature point corresponding to the first face feature point in the MRI three-dimensional data.

[0108] Based on the first and second facial feature points, coordinate transformation is performed between the MRI spatial coordinates and the 3D scanner spatial coordinates to obtain the first transformation matrix.

[0109] Optionally, the registration module 302 is further configured to:

[0110] Acquire 3D point cloud sample data of human face using a 3D scanner; acquire MRI point cloud sample data;

[0111] The preset face recognition model is obtained by training the PointNet network model on 3D point cloud sample data and MRI point cloud sample data.

[0112] Optionally, in the process of registering the face region and the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix, the registration module 302 is used for:

[0113] The initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region is determined based on the first transformation matrix;

[0114] Point cloud sampling and preprocessing are performed on the MRI 3D data and the 3D model data of the face region to obtain the preprocessed target point cloud and input point cloud; the input point cloud corresponds to the 3D model data of the face region, and the target point cloud corresponds to the MRI 3D data;

[0115] Based on the initial position, point cloud matching is performed between the target point cloud set and the input point cloud set. Specifically, a transformation matrix is ​​constructed to transform the point cloud set of the input point cloud set to the coordinate system corresponding to the target point cloud set. The error function between the transformed source point cloud and the target point cloud is estimated. When the error function is determined to converge, the second transformation matrix is ​​obtained.

[0116] Optionally, in determining the initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region based on the first transformation matrix, the registration module 302 is specifically used for:

[0117] The MRI 3D data and the 3D model data of the face region are rotated and translated according to the first transformation matrix to successfully match the MRI 3D data and the 3D model data of the face region, thereby obtaining the initial position.

[0118] Optionally, the scan data further includes an RGB image. Regarding the cropping of the face region on the depth map to obtain a partial face region, the registration module 302 is specifically used for:

[0119] Capture the rectangular region of the face from the RGB image;

[0120] The rectangular region of the face is mapped onto the depth map to obtain the face portion region of the depth map rectangle.

[0121] In this embodiment, the patient's MRI three-dimensional data is acquired; a 3D scanner is used to scan the patient's face to obtain scan data, and three-dimensional model data is generated based on the scan data; the scan data includes a depth map; coordinate transformation is performed on the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; the facial region on the depth map is cropped to obtain a partial facial region; the partial facial region and the MRI three-dimensional data are registered based on the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the partial facial region to obtain a second transformation matrix. Therefore, the accuracy and efficiency of head MRI registration can be improved using a 3D scanner, thereby improving the accuracy and efficiency of transcranial magnetic stimulation (TMS) therapy.

[0122] Example 4

[0123] like Figure 9 , Figure 9 The application provides a head MRI registration system, which includes a processor 410 and a memory 420; and one or more programs stored in the memory. The memory 420 can be a high-speed RAM or non-volatile memory, such as a disk storage device. The memory 420 is used to store a set of program codes, and the processor 410 is used to call the program codes stored in the memory 420 to perform the following operations:

[0124] Acquire the patient's MRI three-dimensional data;

[0125] A 3D scanner is used to scan the patient's face to obtain scan data, and a three-dimensional model data is generated based on the scan data; the scan data includes a depth map.

[0126] Based on the MRI three-dimensional data and the three-dimensional model data, coordinate transformation is performed between the MRI spatial coordinates and the 3D scanner spatial coordinates to obtain the first transformation matrix;

[0127] The face region on the depth map is cropped to obtain the face portion region;

[0128] The face region and the MRI three-dimensional data are registered based on the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix.

[0129] In one possible example, regarding the acquisition of the patient's MRI three-dimensional data, the processor 410 is specifically used for:

[0130] Acquire the patient's magnetic resonance imaging data;

[0131] The magnetic resonance imaging data is reconstructed in three dimensions to obtain MRI three-dimensional data.

[0132] In one possible example, regarding the coordinate transformation of the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix, the processor 410 is specifically configured to:

[0133] The first facial feature point is obtained by performing facial recognition on the MRI three-dimensional data using a preset facial recognition model.

[0134] The preset face recognition model is used to identify feature points in the three-dimensional model data to obtain a second face feature point corresponding to the first face feature point in the MRI three-dimensional data.

[0135] Based on the first and second facial feature points, coordinate transformation is performed between the MRI spatial coordinates and the 3D scanner spatial coordinates to obtain the first transformation matrix.

[0136] In one possible example, the processor 410 is also used for:

[0137] Acquire 3D point cloud sample data of human face using a 3D scanner; acquire MRI point cloud sample data;

[0138] The preset face recognition model is obtained by training the PointNet network model on 3D point cloud sample data and MRI point cloud sample data.

[0139] In one possible example, regarding the process of registering the face region and the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix, the processor 410 is specifically configured to:

[0140] The initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region is determined based on the first transformation matrix;

[0141] Point cloud sampling and preprocessing are performed on the MRI 3D data and the 3D model data of the face region to obtain the preprocessed target point cloud and input point cloud; the input point cloud corresponds to the 3D model data of the face region, and the target point cloud corresponds to the MRI 3D data;

[0142] Based on the initial position, point cloud matching is performed between the target point cloud set and the input point cloud set. Specifically, a transformation matrix is ​​constructed to transform the point cloud set of the input point cloud set to the coordinate system corresponding to the target point cloud set. The error function between the transformed source point cloud and the target point cloud is estimated. When the error function is determined to converge, the second transformation matrix is ​​obtained.

[0143] In one possible example, regarding the determination of the initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region based on the first transformation matrix, the processor 410 is specifically configured to:

[0144] The MRI 3D data and the 3D model data of the face region are rotated and translated according to the first transformation matrix to successfully match the MRI 3D data and the 3D model data of the face region, thereby obtaining the initial position.

[0145] In one possible example, where the scan data also includes an RGB image, the processor 410 is specifically configured to: crop the face region on the depth image to obtain a partial face region;

[0146] Capture the rectangular region of the face from the RGB image;

[0147] The rectangular region of the face is mapped onto the depth map to obtain the face portion region of the depth map rectangle.

[0148] This embodiment of the head MRI registration method based on a 3D scanner involves acquiring the patient's 3D MRI data; performing a facial scan on the patient using a 3D scanner to obtain scan data; generating 3D model data based on the scan data; the scan data including a depth map; performing coordinate transformation between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the MRI 3D data and the 3D model data to obtain a first transformation matrix; cropping the facial region on the depth map to obtain a partial facial region; and registering the partial facial region with the MRI 3D data based on the first transformation matrix, the MRI 3D data, and the 3D model data of the partial facial region to obtain a second transformation matrix. Therefore, the accuracy and efficiency of head MRI registration can be improved using a 3D scanner, thereby improving the accuracy and efficiency of transcranial magnetic stimulation (TMS) therapy.

[0149] This application also provides a computer program product, comprising a non-transitory computer-readable program product storing a computer program operable to cause a computer to perform some or all of the steps described in any of the 3D scanner-based head MRI registration methods recounted in this application. The computer program product may be a software installation package.

[0150] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0151] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The computer program may be stored / distributed in a suitable medium, provided with or as part of other hardware, or may take other distribution forms, such as via the Internet or other wired or wireless telecommunications systems.

[0152] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable vehicle trajectory analysis device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable vehicle trajectory analysis device, generate instructions for implementing the process... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0153] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable human and vehicle trajectory analysis device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable human and vehicle trajectory analysis device, causing a series of operational steps to be executed on the computer or other programmable device to produce computer-implemented processing, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0155] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A head MRI registration method based on a 3D scanner, characterized in that, include: Acquire the patient's MRI three-dimensional data; A 3D scanner is used to scan the patient's face to obtain scan data, and a three-dimensional model data is generated based on the scan data; the scan data includes a depth map and an RGB map. The first facial feature point is obtained by performing facial recognition on the MRI three-dimensional data using a preset facial recognition model. The preset face recognition model is used to identify feature points in the three-dimensional model data to obtain a second face feature point corresponding to the first face feature point in the MRI three-dimensional data. Based on the first facial feature points and the second facial feature points, coordinate transformation is performed between the MRI spatial coordinates and the 3D scanner spatial coordinates to obtain the first transformation matrix; The face region on the depth map is cropped to obtain the face portion region; Capture the rectangular region of the face from the RGB image; The rectangular region of the face is mapped onto the depth map to obtain the face portion region of the depth map rectangle; The face region and the MRI three-dimensional data are registered based on the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix.

2. The head MRI registration method based on a 3D scanner according to claim 1, characterized in that, The acquisition of the patient's MRI three-dimensional data includes: Acquire the patient's magnetic resonance imaging data; The magnetic resonance imaging data is reconstructed in three dimensions to obtain MRI three-dimensional data.

3. The head MRI registration method based on a 3D scanner according to claim 1, characterized in that, The method further includes: Acquire 3D point cloud sample data of human face using a 3D scanner; acquire MRI point cloud sample data; The preset face recognition model is obtained by training the PointNet network model on 3D point cloud sample data and MRI point cloud sample data.

4. The head MRI registration method based on a 3D scanner according to any one of claims 1-3, characterized in that, The step of registering the face region and the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix includes: The initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region is determined based on the first transformation matrix; Point cloud sampling and preprocessing are performed on the MRI 3D data and the 3D model data of the face region to obtain the preprocessed target point cloud and input point cloud; the input point cloud corresponds to the 3D model data of the face region, and the target point cloud corresponds to the MRI 3D data; Based on the initial position, point cloud matching is performed between the target point cloud set and the input point cloud set. Specifically, a transformation matrix is ​​constructed to transform the point cloud set of the input point cloud set to the coordinate system corresponding to the target point cloud set. The error function between the transformed source point cloud and the target point cloud is estimated. When the error function is determined to converge, the second transformation matrix is ​​obtained.

5. The head MRI registration method based on a 3D scanner according to claim 4, characterized in that, The step of determining the initial position for registering the MRI three-dimensional data and the three-dimensional model data of the facial region based on the first transformation matrix includes: The MRI 3D data and the 3D model data of the face region are rotated and translated according to the first transformation matrix to successfully match the MRI 3D data and the 3D model data of the face region, thereby obtaining the initial position.

6. A head MRI registration system based on a 3D scanner, characterized in that, include: The acquisition module is used to acquire the patient's MRI three-dimensional data; The acquisition module is also used to perform a facial scan on the patient using a 3D scanner to obtain scan data, and generate three-dimensional model data based on the scan data; the scan data includes a depth map and an RGB map; The registration module is used to perform face recognition on the MRI three-dimensional data using a preset face recognition model to obtain a first face feature point; to perform feature point recognition on the three-dimensional model data using the preset face recognition model to obtain a second face feature point corresponding to the first face feature point of the MRI three-dimensional data; and to perform coordinate transformation between the MRI spatial coordinates and the 3D scanner spatial coordinates based on the first face feature point and the second face feature point to obtain a first transformation matrix. The cropping module is used to crop the face region on the depth map to obtain the face portion region; The registration module is also used for: Capture the rectangular region of the face from the RGB image; The rectangular region of the face is mapped onto the depth map to obtain the face portion region of the depth map rectangle; The face region and the MRI three-dimensional data are registered based on the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the face region to obtain the second transformation matrix.

7. A head MRI registration system based on a 3D scanner, characterized in that, It includes a processor and a memory; and one or more programs stored in the memory and configured to be executed by the processor, the programs including steps for a 3D scanner-based head MRI registration method as claimed in any one of claims 1 to 5.

8. A computer program product, characterized in that, When the computer program of the computer program product is executed by a processor, it can implement the steps of the head MRI registration method based on a 3D scanner as described in any one of claims 1 to 5.

Images