3D scanner-based head MRI registration method, system and program product

Abstract

The invention relates to the technical field of transcranial magnetic stimulation target spot positioning, and provides a head MRI registration method, system and program product based on a 3D scanner, and the method comprises the steps: obtaining the MRI three-dimensional data of a patient; the method comprises the following steps: scanning the face of a patient through a 3D scanner to obtain scanning data, and generating three-dimensional model data according to the scanning data; the scanning data comprises a depth map; performing coordinate transformation on MRI space coordinates and space coordinates of a 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; cutting a face region on the depth map to obtain a face partial region; and according to the first conversion matrix, the MRI three-dimensional data and the three-dimensional model data of the face partial region, registering the face partial region and the MRI three-dimensional data to obtain a second conversion matrix, so that the precision and efficiency of head MRI registration can be improved through a 3D scanner, and the precision and efficiency of transcranial magnetic stimulation treatment are improved.

Description

Technical Field

[0001] The present application relates to the technical field of transcranial magnetic stimulation therapy, and in particular to a head MRI registration method, system and program product based on a 3D scanner. Background Art

[0002] During the transcranial magnetic stimulation treatment with the help of a robotic arm, the magnetic therapy device at the end of the robotic arm needs to be guided to the target position of the patient's treatment with the help of an optical locator. During the treatment, the patient's head may deviate, affecting the treatment effect. The optical locator will follow the patient's head deviation in real time to improve the treatment effect. Before guidance, the patient's magnetic resonance imaging (MRI) data and the patient's real physical space need to be aligned, so the patient's MRI data needs to be reconstructed in three dimensions, and then the reconstructed facial three-dimensional data is spatially aligned with the patient's real facial data during treatment.

[0003] The existing head registration process is to mark facial feature points in MRI space, and then use the marked facial feature points to collect corresponding feature points on the corresponding patient's face with a collection probe, and the doctor uses the collection probe to collect feature points on the patient's face; the MRI space coordinates and the physical space coordinates of the real patient are converted through the marked facial feature points and the feature points at the preset point positions collected by the collection probe, so as to unify the MRI data space coordinates and the real patient space coordinates.

[0004] The disadvantage of this registration process is that when the doctor uses the acquisition probe to collect feature points on the patient's head, it is difficult to accurately collect the points corresponding to the MRI space. In addition, the registration requires high acquisition skills from the doctor, the registration effect is not ideal, the registration time is long, and the registration efficiency is low. Summary of the invention

[0005] In order to solve or at least partially solve the above technical problems, the present application provides a head MRI registration method, system and program product based on a 3D scanner, which can avoid errors caused by human acquisition, achieve registration more accurately, and improve the speed of head MRI registration.

[0006] In a first aspect, the present application provides a head MRI registration method based on a 3D scanner, comprising: Acquire MRI three-dimensional data of the patient; Scanning the patient's face with a 3D scanner to obtain scan data, and generating three-dimensional model data according to the scan data; the scan data includes a depth map; Performing coordinate transformation on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; Cropping the face area on the depth map to obtain a partial face area; The partial face area and the MRI three-dimensional data are registered according to the first conversion matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second conversion matrix.

[0007] In a second aspect, the present application provides a head MRI registration system based on a 3D scanner, comprising: An acquisition module, used for acquiring MRI three-dimensional data of a patient; The acquisition module is further used to scan the patient's face with a 3D scanner to obtain scan data, and generate three-dimensional model data according to the scan data; the scan data includes a depth map; The registration module is used to perform coordinate transformation on the MRI space coordinates and the 3D scanner space coordinates according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; A cropping module, used for cropping the face area on the depth map to obtain a partial face area; The registration module is further used to register the partial face area with the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second transformation matrix.

[0008] In a third aspect, an embodiment of the present application also provides a 3D scanner-based head MRI alignment system, comprising a processor and a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising steps for the 3D scanner-based head MRI alignment method as described in the first aspect.

[0009] In a fourth aspect, an embodiment of the present application further provides a computer program product, which, when executed by a computer, enables the computer to execute the 3D scanner-based head MRI alignment method of the first aspect or the head MRI navigation method of the second aspect.

[0010] The head MRI registration method based on a 3D scanner of the implementation mode of the present application obtains the patient's MRI three-dimensional data; performs a facial scan on the patient through a 3D scanner to obtain scanning data, and generates three-dimensional model data based on the scanning data; the scanning data includes a depth map; performs coordinate transformation on the MRI space coordinates and the 3D scanner space coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; crops the face area on the depth map to obtain a partial face area; and registers the partial face area with the MRI three-dimensional data based on the first transformation matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second transformation matrix. Thus, the accuracy and efficiency of the head MRI registration can be improved through the 3D scanner, thereby improving the accuracy and efficiency of transcranial magnetic stimulation therapy. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the implementation of the present application, the following is a brief introduction to the relevant drawings. It is understood that the drawings described below are only used to illustrate some implementations of the present application, and those skilled in the art can also obtain many other technical features and connection relationships not mentioned in this document based on these drawings.

[0012] Figure 1 A schematic diagram of a head MRI registration system provided in an embodiment of the present application; Figure 2 A schematic diagram showing a method of performing face recognition on MRI three-dimensional data by using a preset face recognition model to obtain a first face feature point provided in an embodiment of the present application; Figure 3 A schematic diagram showing a method of performing face recognition on three-dimensional model data by using a preset face recognition model to obtain a second face feature point provided by an embodiment of the present application; Figure 4 A schematic diagram showing a rectangular face area of ​​a human face captured in an RGB image provided by an embodiment of the present application; Figure 5 A schematic diagram showing a method of mapping a rectangular face area to a depth map provided in an embodiment of the present application; Figure 6 A schematic diagram showing the mapping of a rectangular face area of ​​an RGB image to MRI three-dimensional data during precise registration provided by an embodiment of the present application; Figure 7 A schematic diagram of a flow chart of another head MRI registration method provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of a head MRI registration system provided in an embodiment of the present application; Fig. 9 A schematic diagram of the structure of a head MRI registration system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0015] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0016] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application.

[0017] Embodiment 1 like Figure 1 As shown, Figure 1 A schematic diagram of a head MRI registration method based on a 3D scanner provided in an embodiment of the present application. An embodiment of the present application proposes a head MRI registration method based on a 3D scanner, the method comprising: 101. Obtain MRI three-dimensional data of the patient.

[0018] Among them, a magnetic resonance imaging device can be used to collect magnetic resonance imaging (MRI) data of the subject, and MRI three-dimensional data can be generated according to the magnetic resonance imaging data.

[0019] 102. Scan the patient's face with a 3D scanner to obtain scan data, and generate three-dimensional model data according to the scan data.

[0020] The scanning data includes a GRB map, a depth map and a point cloud corresponding to the depth map. Specifically, the 3D scanner can be registered with the optical locator so that the origins of the 3D scanner and the optical locator are in the same coordinate system to determine the relative position relationship between the 3D scanner and the optical locator; the patient's face is scanned with a 3D scanner to obtain scanning data, and first three-dimensional model data is generated based on the scanning data to mark feature points corresponding to the MRI three-dimensional data on the model.

[0021] 103. Perform coordinate transformation on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix.

[0022] Among them, the process of performing coordinate transformation to obtain the first transformation matrix can be understood as rough alignment, the purpose of which is to increase the overlap of the two sets of data as much as possible through the transformation matrix (rotation and translation).

[0023] Specifically, feature points may be marked where the MRI three-dimensional data and the three-dimensional model data have the same features, and then matrix transformation of the two sets of data may be obtained through transformation of the feature points.

[0024] 104. Crop the face region on the depth map to obtain a partial face region.

[0025] Among them, by cropping the face area on the depth map to obtain the partial face area, the partial face area is precisely registered, which can improve the registration speed and accuracy.

[0026] 105. Align the partial face region with the MRI three-dimensional data according to the first conversion matrix, the MRI three-dimensional data, and the three-dimensional model data of the partial face region to obtain a second conversion matrix.

[0027] Among them, the first transformation matrix is ​​used to determine the initial position for aligning the MRI three-dimensional data and the three-dimensional model data of the partial area of ​​the face; the process of performing alignment to obtain the second transformation matrix can be understood as fine alignment. In this scheme, the two sets of data (MRI three-dimensional data and three-dimensional model data of the partial area of ​​the face) are preliminarily aligned through coarse alignment to find a rough matching relationship, providing a better initial position for subsequent precise alignment; further, through fine alignment, the optimal transformation matrix is ​​calculated to minimize the distance between corresponding points in the two spaces (MRI space and 3D scanner space) to achieve high-precision alignment.

[0028] Specifically, the initial position for aligning the MRI three-dimensional data and the three-dimensional model data of the partial face area can be determined according to the first transformation matrix; then the three-dimensional model data and the MRI three-dimensional data of the partial face area are aligned according to the initial position to obtain a second transformation matrix, thereby achieving precise alignment.

[0029] The embodiment of the present application provides a head MRI registration method based on a 3D scanner, which obtains the patient's MRI three-dimensional data; performs a facial scan on the patient through a 3D scanner to obtain scanning data, and generates three-dimensional model data based on the scanning data; the scanning data includes a depth map; coordinate conversion is performed on the MRI space coordinates and the 3D scanner space coordinates according to the MRI three-dimensional data and the three-dimensional model data to obtain a first conversion matrix; the face area on the depth map is cropped to obtain a partial face area; and the partial face area and the MRI three-dimensional data are registered according to the first conversion matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second conversion matrix. Thus, the accuracy and efficiency of head MRI registration can be improved through a 3D scanner, thereby improving the accuracy and efficiency of transcranial magnetic stimulation treatment.

[0030] Optionally, acquiring MRI three-dimensional data of the patient includes: acquiring magnetic resonance imaging data of the patient; The magnetic resonance imaging data is three-dimensionally reconstructed to obtain MRI three-dimensional data.

[0031] Among them, magnetic resonance brain imaging may include functional magnetic resonance imaging (fMRI) and structural magnetic resonance imaging (sMRI). Three-dimensional reconstruction can be performed based on the functional magnetic resonance images and the structural magnetic resonance images to obtain MRI three-dimensional data.

[0032] Optionally, performing coordinate transformation on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix includes: Performing face recognition on the MRI three-dimensional data using a preset face recognition model to obtain a first face feature point; Performing feature point recognition on the three-dimensional model data by using the preset face recognition model to obtain second face feature points corresponding to the first face feature points of the MRI three-dimensional data; Coordinate transformation is performed on the MRI space coordinates and the 3D scanner space coordinates according to the first facial feature points and the second facial feature points to obtain a first transformation matrix.

[0033] like Figure 2 As shown in FIG. 1 , it is a demonstration schematic diagram of performing face recognition on MRI three-dimensional data by using a preset face recognition model to obtain a first face feature point. This solution can mark the face feature points of the MRI three-dimensional model by using a face recognition model; Figure 3 As shown, it is a demonstration diagram of performing face recognition on three-dimensional model data through a preset face recognition model to obtain the second face feature points. By adopting the face recognition model, this solution can mark the face feature points of the three-dimensional model scanned by the 3D scanner; through coarse registration, the two sets of data (MRI three-dimensional data and second three-dimensional model data) are preliminarily aligned to find a rough matching relationship, providing a better initial position for subsequent precise registration.

[0034] The overlap of the two sets of data can be increased as much as possible through the transformation matrix (rotation and translation). Feature points can be marked at the places where the two sets of data (the second three-dimensional model data and the MRI three-dimensional data) have the same features. Then, the matrix transformation of the two sets of data is obtained through the transformation of the feature points to obtain the first transformation matrix.

[0035] Specifically, a face recognition model can be pre-trained, and three-dimensional point cloud sample data of the face can be obtained through a 3D scanner; MRI point cloud sample data can be obtained; the three-dimensional point cloud sample data and MRI point cloud sample data can be trained through a PointNet network model to obtain the preset face recognition model.

[0036] In this solution, facial feature point recognition can be performed through an AI model. Specifically, facial recognition is performed on the MRI three-dimensional data through a preset face recognition model to obtain a first facial feature point. Feature point recognition is performed on the three-dimensional model data through the preset face recognition model to obtain a second facial feature point corresponding to the first facial feature point of the MRI three-dimensional data. The first facial feature point and the second facial feature point are then used to perform a rough alignment to obtain a first transformation matrix. The preset face recognition model can more accurately identify the facial feature points and use them for rough alignment.

[0037] Optionally, the scan data includes a depth map; and registering the partial face area with the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the partial face area to obtain a second transformation matrix includes: Determine an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix; Performing point cloud set sampling and preprocessing on the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a preprocessed target point cloud set and an input point cloud set; the input point cloud set corresponds to the three-dimensional model data of the partial face area, and the target point cloud set corresponds to the MRI three-dimensional data; According to the initial position, the target point cloud set and the input point cloud set are matched, wherein the point cloud of the input point cloud set is transformed into the coordinate system corresponding to the target point cloud set by constructing a transformation matrix, and the error function between the transformed source point cloud and the target point cloud is estimated. When it is determined that the error function converges, a second transformation matrix is ​​obtained. The target point cloud set P={p1, p2, ..., p n} and the input point cloud set X = {x1, x2, ..., x n}Perform point cloud matching and construct the transformation matrix (R, t). The error function is: .

[0038] Where N is the number of point clouds of target points.

[0039] Optionally, registering the partial face area with the MRI three-dimensional data according to the first conversion matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second conversion matrix includes: Determine an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix; Performing point cloud set sampling and preprocessing on the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a preprocessed target point cloud set and an input point cloud set; the input point cloud set corresponds to the three-dimensional model data of the partial face area, and the target point cloud set corresponds to the MRI three-dimensional data; The target point cloud set and the input point cloud set are matched with each other according to the initial position, wherein the point cloud of the input point cloud set is transformed into the coordinate system corresponding to the target point cloud set by constructing a transformation matrix, and the error function between the transformed source point cloud and the target point cloud is estimated. When it is determined that the error function converges, a second transformation matrix is ​​obtained.

[0040] Among them, the target point cloud set P = {p1, p2, ..., p n} and the input point cloud set X = {x1, x2, ..., x n}Perform point cloud matching and construct the transformation matrix (R, t). The error function is: .

[0041] Where N is the number of point clouds of target points.

[0042] Optionally, determining an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix includes: The MRI three-dimensional data and the three-dimensional model data of the partial face area are rotated and translated according to the first transformation matrix, so that the MRI three-dimensional data and the three-dimensional model data of the partial face area are successfully matched to obtain the initial position.

[0043] Optionally, the scan data further includes an RGB image, and the step of cropping the face area on the depth image to obtain a partial face area includes: Collect the face rectangular area of ​​the RGB image; The face rectangular area is mapped to the depth map to obtain the face partial area of ​​the depth map rectangle.

[0044] like Figure 4 As shown in FIG. 1 , a schematic diagram of collecting a rectangular face area of ​​an RGB image is shown. Figure 5 As shown in FIG. 1 , it is a demonstration diagram of mapping a rectangular face area to a depth map; Figure 6 As shown, it is a demonstration schematic diagram of mapping the face rectangular area of ​​the RGB image to the MRI three-dimensional data during precise registration. By collecting the face rectangular area of ​​the RGB image, mapping the face rectangular area to the depth map, obtaining the face partial area of ​​the depth map rectangle, and performing precise registration on the face partial area, the registration speed and accuracy can be improved.

[0045] Embodiment 2 like Figure 7 As shown, one embodiment of the present application proposes another head MRI registration method based on a 3D scanner, Figure 7 A schematic flow chart of another head MRI registration method based on a 3D scanner provided in an embodiment of the present application, the method comprising: 201. Obtain magnetic resonance imaging data of a patient.

[0046] 202. Perform three-dimensional reconstruction on the magnetic resonance imaging data to obtain MRI three-dimensional data.

[0047] 203. Scan the patient's face with a 3D scanner to obtain scan data, and generate three-dimensional model data according to the scan data; the scan data includes a depth map and an RGB map.

[0048] 204. Perform face recognition on the MRI three-dimensional data using a preset face recognition model to obtain a first face feature point.

[0049] 205. Perform feature point recognition on the three-dimensional model data using the preset face recognition model to obtain second face feature points corresponding to the first face feature points of the MRI three-dimensional data.

[0050] 206. Perform coordinate transformation on the MRI space coordinates and the 3D scanner space coordinates according to the first facial feature points and the second facial feature points to obtain a first transformation matrix.

[0051] 207. Collect the face rectangular area of ​​the RGB image.

[0052] 208. Map the rectangular face area to the depth map to obtain a partial face area of ​​the depth map rectangle.

[0053] 209. Determine an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial facial area according to the first transformation matrix.

[0054] 210. Perform point cloud set sampling and preprocessing on the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a preprocessed target point cloud set and an input point cloud set.

[0055] 211. Perform point cloud matching on the target point cloud set and the input point cloud set according to the initial position, wherein the point cloud of the input point cloud set is transformed into the coordinate system corresponding to the target point cloud set by constructing a transformation matrix, and the error function between the source point cloud and the target point cloud after the transformation is estimated. When it is determined that the error function converges, a second transformation matrix is ​​obtained.

[0056] In this embodiment, face recognition is performed on MRI three-dimensional data through a preset face recognition model to obtain a first face feature point; feature point recognition is performed on the three-dimensional model data through a preset face recognition model to obtain a second face feature point corresponding to the first face feature point of the MRI three-dimensional data; coordinate conversion is performed on the MRI spatial coordinates and the spatial coordinates of the 3D scanner according to the first face feature point and the second face feature point to obtain a first conversion matrix; an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area is determined according to the first conversion matrix; point cloud set sampling and preprocessing are performed on the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a preprocessed target point cloud set and an input point cloud set; point cloud matching is performed on the target point cloud set and the input point cloud set according to the initial position, wherein the point cloud of the input point cloud set is converted to the coordinate system corresponding to the target point cloud set by constructing a conversion matrix, the error function between the transformed source point cloud and the target point cloud is estimated, and when it is determined that the error function converges, a second conversion matrix is ​​obtained, thereby improving the accuracy and speed of registration.

[0057] Embodiment 3 like Figure 8 , Figure 8 The application embodiment provides a head MRI registration system 300 based on a 3D scanner, comprising: An acquisition module 301 is used to acquire MRI three-dimensional data of a patient; The acquisition module 301 is further used to scan the patient's face with a 3D scanner to obtain scan data, and generate three-dimensional model data according to the scan data; the scan data includes a depth map; The registration module 302 is used to perform coordinate transformation on the MRI space coordinates and the 3D scanner space coordinates according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; A cropping module 303 is used to crop the face area on the depth map to obtain a partial face area; The registration module 302 is further configured to register the partial face region with the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face region to obtain a second transformation matrix.

[0058] Optionally, in the aspect of acquiring the patient's MRI three-dimensional data, the acquisition module 301 is specifically used for: acquiring magnetic resonance imaging data of the patient; The magnetic resonance imaging data is three-dimensionally reconstructed to obtain MRI three-dimensional data.

[0059] Optionally, in the aspect of performing coordinate conversion on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first conversion matrix, the registration module 302 is specifically used for: Performing face recognition on the MRI three-dimensional data using a preset face recognition model to obtain a first face feature point; Performing feature point recognition on the three-dimensional model data by using the preset face recognition model to obtain second face feature points corresponding to the first face feature points of the MRI three-dimensional data; Coordinate transformation is performed on the MRI space coordinates and the 3D scanner space coordinates according to the first facial feature points and the second facial feature points to obtain a first transformation matrix.

[0060] Optionally, the registration module 302 is further configured to: Obtain three-dimensional point cloud sample data of the face through a 3D scanner; obtain MRI point cloud sample data; The three-dimensional point cloud sample data and the MRI point cloud sample data are trained through the PointNet network model to obtain the preset face recognition model.

[0061] Optionally, in registering the partial face area with the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the partial face area to obtain a second transformation matrix, the registration module 302 is used to: Determine an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix; Performing point cloud set sampling and preprocessing on the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a preprocessed target point cloud set and an input point cloud set; the input point cloud set corresponds to the three-dimensional model data of the partial face area, and the target point cloud set corresponds to the MRI three-dimensional data; The target point cloud set and the input point cloud set are matched with each other according to the initial position, wherein the point cloud of the input point cloud set is transformed into the coordinate system corresponding to the target point cloud set by constructing a transformation matrix, and the error function between the transformed source point cloud and the target point cloud is estimated. When it is determined that the error function converges, a second transformation matrix is ​​obtained.

[0062] Optionally, in determining the initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix, the registration module 302 is specifically used to: The MRI three-dimensional data and the three-dimensional model data of the partial face area are rotated and translated according to the first transformation matrix, so that the MRI three-dimensional data and the three-dimensional model data of the partial face area are successfully matched to obtain the initial position.

[0063] Optionally, the scan data further includes an RGB image. In terms of cropping the face area on the depth image to obtain a partial face area, the registration module 302 is specifically configured to: Collect the face rectangular area of ​​the RGB image; The face rectangular area is mapped to the depth map to obtain the face partial area of ​​the depth map rectangle.

[0064] In this embodiment, MRI three-dimensional data of the patient is acquired; a facial scan of the patient is performed through a 3D scanner to obtain scan data, and three-dimensional model data is generated according to the scan data; the scan data includes a depth map; coordinate conversion is performed on the MRI spatial coordinates and the spatial coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first conversion matrix; the face area on the depth map is cropped to obtain a partial face area; the partial face area and the MRI three-dimensional data are registered according to the first conversion matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second conversion matrix, thereby, the accuracy and efficiency of head MRI registration can be improved through the 3D scanner, thereby improving the accuracy and efficiency of transcranial magnetic stimulation treatment.

[0065] Embodiment 4 like Fig. 9 , Fig. 9 The head MRI registration system provided in the application embodiment includes a processor 410 and a memory 420; and one or more programs, one or more programs are stored in the memory, the memory 420 can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a disk memory. The memory 420 is used to store a set of program codes, and the processor 410 is used to call the program code stored in the memory 420 to perform the following operations: Acquire MRI three-dimensional data of the patient; Scanning the patient's face with a 3D scanner to obtain scan data, and generating three-dimensional model data according to the scan data; the scan data includes a depth map; Performing coordinate transformation on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; Cropping the face area on the depth map to obtain a partial face area; The partial face area and the MRI three-dimensional data are registered according to the first conversion matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second conversion matrix.

[0066] In a possible example, in the aspect of acquiring the patient's MRI three-dimensional data, the processor 410 is specifically used for: acquiring magnetic resonance imaging data of the patient; The magnetic resonance imaging data is three-dimensionally reconstructed to obtain MRI three-dimensional data.

[0067] In a possible example, in the aspect of performing coordinate conversion on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain the first conversion matrix, the processor 410 is specifically used to: Performing face recognition on the MRI three-dimensional data using a preset face recognition model to obtain a first face feature point; Performing feature point recognition on the three-dimensional model data by using the preset face recognition model to obtain second face feature points corresponding to the first face feature points of the MRI three-dimensional data; Coordinate transformation is performed on the MRI space coordinates and the 3D scanner space coordinates according to the first facial feature points and the second facial feature points to obtain a first transformation matrix.

[0068] In a possible example, the processor 410 is further configured to: Obtain three-dimensional point cloud sample data of the face through a 3D scanner; obtain MRI point cloud sample data; The three-dimensional point cloud sample data and the MRI point cloud sample data are trained through the PointNet network model to obtain the preset face recognition model.

[0069] In a possible example, in the aspect of registering the partial face region with the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data, and the three-dimensional model data of the partial face region to obtain the second transformation matrix, the processor 410 is specifically configured to: Determine an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix; Performing point cloud set sampling and preprocessing on the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a preprocessed target point cloud set and an input point cloud set; the input point cloud set corresponds to the three-dimensional model data of the partial face area, and the target point cloud set corresponds to the MRI three-dimensional data; The target point cloud set and the input point cloud set are matched with each other according to the initial position, wherein the point cloud of the input point cloud set is transformed into the coordinate system corresponding to the target point cloud set by constructing a transformation matrix, and the error function between the transformed source point cloud and the target point cloud is estimated. When it is determined that the error function converges, a second transformation matrix is ​​obtained.

[0070] In a possible example, in determining the initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix, the processor 410 is specifically configured to: The MRI three-dimensional data and the three-dimensional model data of the partial face area are rotated and translated according to the first transformation matrix, so that the MRI three-dimensional data and the three-dimensional model data of the partial face area are successfully matched to obtain the initial position.

[0071] In a possible example, the scan data further includes an RGB image, and in terms of clipping the face area on the depth image to obtain a partial face area, the processor 410 is specifically configured to: Collect the face rectangular area of ​​the RGB image; The face rectangular area is mapped to the depth map to obtain the face partial area of ​​the depth map rectangle.

[0072] The head MRI registration method based on a 3D scanner of the present embodiment obtains the patient's MRI three-dimensional data; performs a facial scan on the patient through a 3D scanner to obtain scanning data, and generates three-dimensional model data based on the scanning data; the scanning data includes a depth map; performs coordinate transformation on the MRI space coordinates and the 3D scanner space coordinates based on the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; crops the face area on the depth map to obtain a partial face area; and registers the partial face area with the MRI three-dimensional data based on the first transformation matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second transformation matrix. Thus, the accuracy and efficiency of the head MRI registration can be improved through the 3D scanner, thereby improving the accuracy and efficiency of transcranial magnetic stimulation treatment.

[0073] The present application also provides a computer program product, wherein the computer program product includes a non-transitory computer-readable program product storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in any of the head MRI registration methods based on a 3D scanner as described in the present application. The computer program product may be a software installation package.

[0074] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other changes to the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0075] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment), or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The computer program is stored / distributed in a suitable medium, provided together with other hardware or as part of the hardware, or may take other distribution forms, such as through the Internet or other wired or wireless telecommunications systems.

[0076] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (devices) and computer program products of the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable human-vehicle trajectory analysis device to produce a machine, so that the instructions executed by the processor of the computer or other programmable human-vehicle trajectory analysis device generate instructions for implementing the processes in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable human-vehicle trajectory analysis device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable vehicle trajectory analysis device, so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0079] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A head MRI registration method based on a 3D scanner, characterized in that: include: Acquire MRI three-dimensional data of the patient; Scanning the patient's face with a 3D scanner to obtain scan data, and generating three-dimensional model data according to the scan data; the scan data includes a depth map; Performing coordinate transformation on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; Cropping the face area on the depth map to obtain a partial face area; The partial face area and the MRI three-dimensional data are registered according to the first conversion matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second conversion matrix.

2. The head MRI registration method based on 3D scanner according to claim 1, characterized in that: The step of acquiring MRI three-dimensional data of a patient comprises: acquiring magnetic resonance imaging data of the patient; The magnetic resonance imaging data is three-dimensionally reconstructed to obtain MRI three-dimensional data.

3. The head MRI registration method based on 3D scanner according to claim 1, characterized in that: The step of performing coordinate transformation on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix includes: Performing face recognition on the MRI three-dimensional data using a preset face recognition model to obtain a first face feature point; Performing feature point recognition on the three-dimensional model data by using the preset face recognition model to obtain second face feature points corresponding to the first face feature points of the MRI three-dimensional data; Coordinate transformation is performed on the MRI space coordinates and the 3D scanner space coordinates according to the first facial feature points and the second facial feature points to obtain a first transformation matrix.

4. The head MRI registration method based on 3D scanner according to claim 3, characterized in that: The method further comprises: Obtain three-dimensional point cloud sample data of the face through a 3D scanner; obtain MRI point cloud sample data; The three-dimensional point cloud sample data and the MRI point cloud sample data are trained through the PointNet network model to obtain the preset face recognition model.

5. The head MRI registration method based on a 3D scanner according to any one of claims 1 to 4, characterized in that: The registering the partial face region with the MRI three-dimensional data according to the first conversion matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face region to obtain a second conversion matrix includes: Determine an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix; Performing point cloud set sampling and preprocessing on the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a preprocessed target point cloud set and an input point cloud set; the input point cloud set corresponds to the three-dimensional model data of the partial face area, and the target point cloud set corresponds to the MRI three-dimensional data; The target point cloud set and the input point cloud set are matched with each other according to the initial position, wherein the point cloud of the input point cloud set is transformed into the coordinate system corresponding to the target point cloud set by constructing a transformation matrix, and the error function between the transformed source point cloud and the target point cloud is estimated. When it is determined that the error function converges, a second transformation matrix is ​​obtained.

6. The head MRI registration method based on 3D scanner according to claim 5, characterized in that: The step of determining an initial position for registering the MRI three-dimensional data and the three-dimensional model data of the partial face area according to the first transformation matrix includes: The MRI three-dimensional data and the three-dimensional model data of the partial face area are rotated and translated according to the first transformation matrix, so that the MRI three-dimensional data and the three-dimensional model data of the partial face area are successfully matched to obtain the initial position.

7. The head MRI registration method based on a 3D scanner according to any one of claims 1 to 4, characterized in that: The scan data also includes an RGB image, and the face area on the depth image is cropped to obtain a partial face area, including: Collect the face rectangular area of ​​the RGB image; The face rectangular area is mapped to the depth map to obtain the face partial area of ​​the depth map rectangle.

8. A head MRI registration system based on a 3D scanner, characterized in that: include: An acquisition module, used for acquiring MRI three-dimensional data of a patient; The acquisition module is further used to scan the patient's face with a 3D scanner to obtain scan data, and generate three-dimensional model data according to the scan data; the scan data includes a depth map; A registration module, used for performing coordinate transformation on the MRI space coordinates and the space coordinates of the 3D scanner according to the MRI three-dimensional data and the three-dimensional model data to obtain a first transformation matrix; A cropping module, used for cropping the face area on the depth map to obtain a partial face area; The registration module is further used to register the partial face area with the MRI three-dimensional data according to the first transformation matrix, the MRI three-dimensional data and the three-dimensional model data of the partial face area to obtain a second transformation matrix.

9. A head MRI registration system based on a 3D scanner, characterized in that: The invention comprises a processor and a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, wherein the programs include steps for a head MRI registration method based on a 3D scanner as described in any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program of the computer program product is executed by a processor, it is capable of implementing the steps of the head MRI registration method based on a 3D scanner according to any one of claims 1 to 7.

Images