Head image spatial registration method and device, equipment and storage medium
By constructing a 3D head model and combining iterative optimization with facial feature points and point cloud data, the problems of initial position deviation and insufficient accuracy in markerless registration methods are solved, achieving high-precision and robust head image registration.
Patent Information
- Application Number
- CN202511742574.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-02-27
AI Technical Summary
Existing markerless registration methods are prone to local optima due to initial positional deviations in head image registration, making it difficult to converge to the global optimum. Furthermore, they lack registration accuracy and robustness in areas with similar facial features or indistinct characteristics.
A 3D model is constructed from head image data, facial feature point sets are extracted, and preliminary registration is performed using a rigid body transformation matrix. Then, iterative optimization is performed by combining point cloud data around the facial feature points to determine the spatial transformation matrix for precise registration.
It improves registration accuracy and efficiency, avoids errors and patient discomfort caused by marker displacement, simplifies clinical procedures, and ensures the precision and stability of treatment.
Smart Images
Figure CN121582306A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical image processing and medical equipment technology, and in particular to a method, apparatus, device and storage medium for spatial registration of head images. Background Technology
[0002] In transcranial magnetic stimulation (TMS) therapy, precise spatial registration of pre-treatment brain medical images with the patient's actual head position during treatment is crucial to ensuring treatment accuracy. Currently, commonly used registration techniques include markerless registration methods, which primarily rely on traditional surface-to-surface registration strategies. This involves matching the surface data of the head image with the point cloud surface data of the patient's face obtained from a 3D scanner to achieve alignment.
[0003] However, existing label-free registration methods have the following shortcomings: First, the registration result depends on the initial spatial relative position of the two surfaces. If the initial deviation is large, the algorithm is prone to getting trapped in local optima and cannot converge to the global best registration state. Second, in areas with similar facial shapes or indistinct features, the accuracy and robustness of the registration are difficult to guarantee. Summary of the Invention
[0004] In view of this, embodiments of this application provide a spatial registration method, apparatus, device, and storage medium for head images, which improves registration accuracy and registration efficiency.
[0005] This application mainly includes the following aspects: In a first aspect, embodiments of this application provide a spatial registration method for head images, the method comprising: Based on the key head data extracted from the image data of the target object's head, a three-dimensional head model of the target object is constructed. The first facial feature points are extracted from the three-dimensional head model to obtain the first facial feature point set of the target object; Extract the second facial feature points from the three-dimensional point cloud data of the target object's face to obtain the second facial feature point set of the target object; Based on the first set of facial feature points and the second set of facial feature points, a first rigid body transformation matrix is determined for performing coordinate transformation on the three-dimensional head model. The coordinates of the three-dimensional head model are transformed using the first rigid body transformation matrix. Based on the coordinate-transformed 3D head model, a second rigid body transformation matrix is determined for further coordinate transformation of the coordinate-transformed 3D head model. Based on the first rigid body transformation matrix and the second rigid body transformation matrix, a spatial transformation matrix for performing coordinate transformation on the image data is determined; Based on the spatial transformation matrix, the image data is transformed from the image coordinate system to the physical coordinate system in which the head of the target object is located.
[0006] Furthermore, determining a first rigid body transformation matrix for coordinate transformation of the first facial feature point set based on the first facial feature point set and the second facial feature point set includes: Determine the first geometric center point corresponding to the first set of facial feature points and the second geometric center point corresponding to the second set of facial feature points, respectively. Based on the first geometric center point, determine the vector of each first facial feature point relative to the first geometric center point to obtain a first vector set; and based on the second geometric center point, determine the vector of each second facial feature point relative to the second geometric center point to obtain a second vector set. Construct a covariance matrix based on the first vector set and the second vector set; Decompose the covariance matrix; Based on the decomposed covariance matrix, a first rotation matrix is determined for rotating the first set of facial feature points. Based on the first geometric center point, the second geometric center point, and the first rotation matrix, a first translation vector is determined for translating the first set of facial feature points; Based on the first rotation matrix and the first translation vector, a first rigid body transformation matrix is determined for transforming the coordinates of the first set of facial feature points.
[0007] Furthermore, determining the first rotation matrix for rotating the first set of facial feature points based on the decomposed covariance matrix includes: The product of the left singular vector matrix and the transpose of the right singular vector matrix in the decomposed covariance matrix is determined as the initial rotation matrix; If the determinant of the initial rotation matrix is a preset value, then the initial rotation matrix is determined as the first rotation matrix; If the determinant of the initial rotation matrix is not a preset value, then the preset columns of the right singular vector matrix are inverted; the product of the left singular vector matrix and the transpose of the right singular vector matrix corresponding to the inverted preset columns is determined as the first rotation matrix.
[0008] Furthermore, the determination of a second rigid body transformation matrix for further coordinate transformation of the coordinate-transformed 3D head model based on the coordinate-transformed head model includes: Taking each second facial feature point in the second facial feature point set as the center, a preset region corresponding to each second facial feature point is divided in the three-dimensional point cloud data; For each scan point in each preset region, determine the surface point with the smallest distance from that scan point from the coordinate-transformed 3D head model; Based on all scan points and the surface points corresponding to all scan points, determine the spatial distance error and Jacobian matrix between all scan points and their corresponding surface points; Based on the spatial distance error between all scan points and the corresponding surface points and the Jacobian matrix, a second rotation matrix for rotating the coordinate-transformed 3D head model and a second translation vector for translating the coordinate-transformed 3D head model are determined. The second rotation matrix and the second translation vector are updated iteratively until the spatial distance error is no greater than a preset error value or the number of updates reaches a preset number, at which point the updating of the second rotation matrix and the second translation vector is stopped. Based on the last updated second rotation matrix and second translation vector, a second rigid body transformation matrix is determined for further coordinate transformation of the coordinate-transformed 3D head model.
[0009] Furthermore, determining the spatial transformation matrix for coordinate transformation of the image data based on the first rigid body transformation matrix and the second rigid body transformation matrix includes: The product of the first rigid body transformation matrix and the second rigid body transformation matrix is determined as the spatial transformation matrix used to perform coordinate transformation on the image data.
[0010] Secondly, embodiments of this application also provide a spatial registration device for head images, the device comprising: The building module is used to construct a 3D model of the head of the target object based on key head data extracted from the image data of the target object's head; The first extraction module is used to extract the first facial feature points from the three-dimensional head model to obtain the first facial feature point set of the target object. The second extraction module is used to extract second facial feature points from the three-dimensional point cloud data of the target object's face, and obtain the second facial feature point set of the target object. The first rigid body transformation matrix determination module is used to determine the first rigid body transformation matrix for performing coordinate transformation on the three-dimensional head model based on the first facial feature point set and the second facial feature point set. The first transformation module is used to perform coordinate transformation on the three-dimensional head model using the first rigid body transformation matrix; The second rigid body transformation matrix determination module is used to determine, based on the coordinate-transformed three-dimensional head model, a second rigid body transformation matrix for further coordinate transformation of the coordinate-transformed three-dimensional head model. The spatial transformation matrix determination module is used to determine the spatial transformation matrix for performing coordinate transformation on the image data based on the first rigid body transformation matrix and the second rigid body transformation matrix. The second transformation module is used to transform the image data from the image coordinate system to the physical coordinate system in which the head of the target object is located, based on the spatial transformation matrix.
[0011] Furthermore, the first rigid body transformation matrix determination module is specifically used for: Determine the first geometric center point corresponding to the first set of facial feature points and the second geometric center point corresponding to the second set of facial feature points, respectively. Based on the first geometric center point, determine the vector of each first facial feature point relative to the first geometric center point to obtain a first vector set; and based on the second geometric center point, determine the vector of each second facial feature point relative to the second geometric center point to obtain a second vector set. Construct a covariance matrix based on the first vector set and the second vector set; Decompose the covariance matrix; Based on the decomposed covariance matrix, a first rotation matrix is determined for rotating the first set of facial feature points. Based on the first geometric center point, the second geometric center point, and the first rotation matrix, a first translation vector is determined for translating the first set of facial feature points; Based on the first rotation matrix and the first translation vector, a first rigid body transformation matrix is determined for transforming the coordinates of the first set of facial feature points.
[0012] Furthermore, when determining the first rotation matrix for rotating the first set of facial feature points based on the decomposed covariance matrix, the first rigid body transformation matrix determination module is specifically used for: The product of the left singular vector matrix and the transpose of the right singular vector matrix in the decomposed covariance matrix is determined as the initial rotation matrix; If the determinant of the initial rotation matrix is a preset value, then the initial rotation matrix is determined as the first rotation matrix; If the determinant of the initial rotation matrix is not a preset value, then the preset columns of the right singular vector matrix are inverted; the product of the left singular vector matrix and the transpose of the right singular vector matrix corresponding to the inverted preset columns is determined as the first rotation matrix.
[0013] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus. The machine-readable instructions are executed by the processor to perform the steps of the spatial registration method for head images described in the first aspect or any possible implementation of the first aspect.
[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the spatial registration method for head images described in the first aspect or any possible implementation of the first aspect.
[0015] This application provides a method, apparatus, device, and storage medium for spatial registration of head images. The method involves extracting first facial feature points from a 3D head model to obtain a first set of facial feature points for the target object; extracting second facial feature points from the 3D point cloud data of the target object's face to obtain a second set of facial feature points; determining a first rigid body transformation matrix based on the first and second set of facial feature points; performing a coordinate transformation on the 3D head model using the first rigid body transformation matrix; determining a second rigid body transformation matrix based on the coordinate-transformed 3D head model; determining a spatial transformation matrix based on the first and second rigid body transformation matrices; and transforming the image data from the image coordinate system to the physical coordinate system of the target object's head based on the spatial transformation matrix. This improves registration accuracy and efficiency.
[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This document illustrates one of the flowcharts for a spatial registration method for head images provided in an embodiment of this application. Figure 2 This illustrates a second flowchart of a spatial registration method for head images provided in an embodiment of this application. Figure 3This illustrates a third flowchart of a spatial registration method for head images provided in an embodiment of this application; Figure 4 This illustration shows a schematic diagram of the structure of a spatial registration device for head images provided in an embodiment of this application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.
[0020] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0021] The methods, apparatus, electronic devices, or computer-readable storage media described in this application can be applied to any scenario requiring spatial registration of head images. This application does not limit specific application scenarios, and any scheme using the spatial registration method and apparatus for head images provided in this application is within the protection scope of this application.
[0022] It is worth noting that in transcranial magnetic stimulation (TMS) therapy, accurate spatial registration of the pre-treatment brain medical images with the patient's actual head position during treatment is crucial to ensuring treatment precision. Currently, commonly used registration techniques include label-free registration methods, primarily relying on traditional surface-to-surface registration strategies. This involves matching the surface data of the head image with the point cloud surface data of the patient's face acquired by a 3D scanner. However, existing label-free registration methods have the following shortcomings: First, the registration result depends on the initial relative spatial positions of the two surfaces. If the initial deviation is large, the algorithm is prone to getting trapped in local optima and failing to converge to the globally optimal registration state. Second, in areas with similar facial morphology or indistinct features, both the accuracy and robustness of the registration are difficult to guarantee.
[0023] To address the aforementioned issues, this application proposes a spatial registration method, apparatus, device, and storage medium for head images, which improves registration accuracy and efficiency.
[0024] To facilitate understanding of this application, the technical solutions provided in this application will be described in detail below with reference to specific embodiments.
[0025] In the embodiments of this application, current registration techniques primarily involve high-precision matching of pre-processed MRI or CT images with real-time optical tracking data. This process is achieved using a high-precision optical positioning camera and reflective markers attached to the head, thereby mapping the image space to the physical space. However, optical tracking technology has the following drawbacks: First, the attached markers may cause discomfort to the patient and may shift during treatment due to sweating or touching, introducing registration errors; second, attaching markers increases pre-treatment preparation time, affecting the efficiency of clinical work; finally, the selection and quality of marker placement are highly dependent on the operator's experience, increasing the complexity of the operation. To overcome these drawbacks, marker-free registration methods have emerged.
[0026] Please see Figure 1 , Figure 1 This is one of the flowcharts for a spatial registration method for head images provided in an embodiment of this application.
[0027] like Figure 1 As shown in the figure, the spatial registration method for head images provided in this application embodiment includes the following steps: Step S101: Construct a three-dimensional model of the head of the target object based on the key head data extracted from the image data of the target object's head.
[0028] Here, key head data may include, but is not limited to, the contours of the scalp and brain tissue. Image data of the target object's head can be acquired using medical imaging equipment (such as MRI or CT). Specifically, the steps involve first importing the image data of the target object's head into 3D reconstruction software; then, using region growing or thresholding algorithms to extract the contours of the scalp and brain tissue; finally, reconstructing the head using the moving cube algorithm to obtain a 3D model including the scalp surface. As an example, the 3D reconstruction software could be 3D Slicer, and the threshold range for the thresholding algorithm could be -500 to 1000 HU. Image data of the target object's head can be acquired using medical imaging equipment (such as MRI or CT).
[0029] Step S102: Extract the first facial feature points from the head 3D model to obtain the first facial feature point set of the target object.
[0030] Here, the first facial feature point is the location point of the facial features of the target object extracted from the 3D head model. Facial features may include, but are not limited to: the tip of the nose, the inner corner of the left eye, the inner corner of the right eye, the outer corner of the left eye, and the outer corner of the right eye. Facial feature points can be extracted by manually marking and selecting them on the 3D model, or automatically identifying and locating them using a facial recognition algorithm. The first set of facial feature points is... ,in, - This is the first facial feature point.
[0031] Step S103: Extract the second facial feature points from the three-dimensional point cloud data of the target object's face to obtain the second facial feature point set of the target object.
[0032] Here, the second facial feature point is the location point of the facial features of the target object extracted from the 3D point cloud data of the face. The 3D point cloud data can be acquired using a 3D structured light scanner or a laser scanner. Specifically, the face of the target object is first scanned using a 3D structured light scanner or a laser scanner to obtain the 3D point cloud data of the target object's face. ,in, , - The first step involves identifying the location points in the 3D point cloud data. Then, the Open3D library's preprocessing functions are used to downsample and denoise the 3D point cloud data. Next, a pre-trained Dlib face landmark detection model is used to identify 2D facial feature points on the corresponding 2D texture image. Finally, the 2D facial feature points are mapped back to the 3D point cloud to obtain the second set of facial feature points. for - This is the second facial feature point.
[0033] Step S104: Based on the first set of facial feature points and the second set of facial feature points, determine a first rigid body transformation matrix for coordinate transformation of the three-dimensional head model.
[0034] Here, the three-dimensional head model is aligned to the second facial feature point set coordinate system (i.e., the scanner coordinate system) by using the first rigid body transformation matrix.
[0035] The following is combined Figure 2 To explain in detail how to determine the first rigid body transformation matrix for performing coordinate transformation on the first facial feature point set based on the first facial feature point set and the second facial feature point set.
[0036] Please see Figure 2 , Figure 2 This is a second flowchart of a spatial registration method for head images provided in an embodiment of this application.
[0037] like Figure 2 As shown, regarding step S104, in a specific implementation, as an example, the following steps may be included: Step S1041: Determine the first geometric center point corresponding to the first facial feature point set and the second geometric center point corresponding to the second facial feature point set.
[0038] Here, as an example, the first geometric center point can be calculated using formula (1), and the second geometric center point can be calculated using formula (2).
[0039] (1), (2).
[0040] in, As the first geometric center point, The number of facial feature points. For the i-th first facial feature point, The second geometric center point, Let i be the i-th second facial feature point.
[0041] Step S1042: Based on the first geometric center point, determine the vector of each first facial feature point relative to the first geometric center point to obtain a first vector set; and based on the second geometric center point, determine the vector of each second facial feature point relative to the second geometric center point to obtain a second vector set.
[0042] Here, subtracting the first geometric center point from each of the first facial feature points yields a set of first vectors centered at the origin, and subtracting the second geometric center point from each of the second facial feature points yields a set of second vectors centered at the origin.
[0043] As an example, the first geometric center point can be calculated using formula (3), and the second geometric center point can be calculated using formula (4).
[0044] (3), (4).
[0045] in, Let be the first vector. This is the second vector.
[0046] Step S1043: Construct a covariance matrix based on the first vector set and the second vector set.
[0047] Here, a 3×3 covariance matrix can be constructed based on the transpose of the first and second vectors.
[0048] As an example, the covariance matrix can be calculated using formula (5).
[0049] (5).
[0050] in, Let be the covariance matrix.
[0051] Step S1044: Decompose the covariance matrix.
[0052] Here, we can perform singular value decomposition on the covariance matrix, that is, ,in, This is the notation for singular value decomposition. It is a left singular vector matrix. It is a singular value matrix. It is a right singular vector matrix.
[0053] Step S1045: Based on the decomposed covariance matrix, determine a first rotation matrix for rotating the first set of facial feature points.
[0054] Specifically, this step involves determining the initial rotation matrix as the product of the left singular vector matrix and the transpose of the right singular vector matrix in the decomposed covariance matrix. If the determinant of the initial rotation matrix is a preset value, then the initial rotation matrix is determined as the first rotation matrix. If the determinant of the initial rotation matrix is not a preset value, then a preset column of the right singular vector matrix is inverted. The product of the left singular vector matrix and the transpose of the right singular vector matrix corresponding to the inverted preset column is then determined as the first rotation matrix. Here, the preset value is 1, and the preset column of the right singular vector matrix is the last column of the right singular vector matrix.
[0055] Step S1046: Determine a first translation vector for translating the first set of facial feature points based on the first geometric center point, the second geometric center point, and the first rotation matrix.
[0056] Here, as an example, the first translation vector can be calculated using formula (6).
[0057] (6).
[0058] in, This is the first translation vector.
[0059] Step S1047: Based on the first rotation matrix and the first translation vector, determine a first rigid body transformation matrix for coordinate transformation of the head 3D model.
[0060] Here, as an example, the first rigid body transformation matrix can be calculated using formula (7).
[0061] (7).
[0062] in, Let be the first rigid body transformation matrix.
[0063] See again Figure 1 In step S104, the coordinates of the head 3D model are transformed using the first rigid body transformation matrix.
[0064] Here, using the first rigid body transformation matrix, the 3D head model can be initially aligned to the scanner coordinate system, that is, the 3D head model can be initially aligned to the coordinate system where the second set of facial feature points is located: .in, This is the set of all points in the 3D model of the head. This is the set of all points in the 3D head model after coordinate transformation.
[0065] Step S106: Based on the coordinate-transformed 3D head model, determine a second rigid body transformation matrix for performing another coordinate transformation on the coordinate-transformed 3D head model.
[0066] Here, the initial registration of facial feature points cannot completely eliminate spatial bias, so dense 3D point cloud data of the area surrounding the facial feature points is used for re-registration.
[0067] The following is combined Figure 3 This section will specifically explain how to determine the second rigid body transformation matrix for further coordinate transformation of the three-dimensional head model based on the coordinate-transformed model.
[0068] Please see Figure 3 , Figure 3 This is the third flowchart of a spatial registration method for head images provided in an embodiment of this application.
[0069] like Figure 3 As shown, regarding step S106, in a specific implementation, as an example, the following steps may be included: Step S1061: Using each of the second facial feature points in the second facial feature point set as the center, divide the three-dimensional point cloud data into a preset region corresponding to each of the second facial feature points.
[0070] Here, the preset region is a spherical region with radius r, and the set of all scan points within the spherical region is defined as the region point cloud. As an example, the radius can be 15mm.
[0071] Step S1062: For each scan point in each preset region, determine the surface point with the smallest distance from the scan point in the three-dimensional head model after coordinate transformation.
[0072] Here, for each scan point in each preset region, the surface point closest to the scan point is found in the set of surface points of the head 3D model after coordinate transformation.
[0073] Step S1063: Based on all scan points and the surface points corresponding to all scan points, determine the spatial distance error and Jacobian matrix between all scan points and their corresponding surface points.
[0074] In this embodiment, an objective function is set and iteratively optimized. The optimization objective is the sum of the squares of the distances between all scan points and their corresponding surface points. To enhance robustness, the Huber loss function is introduced to mitigate the influence of outliers during the optimization process. As an example, the objective function can be expressed by formula (8).
[0075] (8).
[0076] in, The spatial distance error between all scan points and their corresponding surface points. This is the second rotation matrix. n The number of scan points, The second translation vector, The loss function symbol, For surface points, These are the scan points.
[0077] In the embodiments of this application, nonlinear optimization algorithms such as the Levonburg-Marquardt algorithm can be used to solve the objective function.
[0078] Step S1064: Based on the spatial distance error between all scan points and their corresponding surface points and the Jacobian matrix, determine a second rotation matrix for rotating all scan points and a second translation vector for translating all scan points.
[0079] Step S1065: Update the second rotation matrix and the second translation vector iteratively until the spatial distance error is less than a preset error value or the number of updates reaches a preset number, then stop updating the second rotation matrix and the second translation vector.
[0080] Here, in each update iteration, the nonlinear optimization algorithm first finds the surface point with the minimum distance on the surface point set of the 3D head model after coordinate transformation for each scan point in the preset region; then, it calculates the spatial distance error and Jacobian matrix between all scan points and their corresponding surface points; finally, it updates the second rotation matrix and the second translation vector using the spatial distance error and the Jacobian matrix, rotates all surface points according to the updated second rotation matrix, and translates all surface points according to the second translation vector. This iterative cycle continues until the spatial distance error converges to a preset error threshold or the preset number of iterations is reached.
[0081] Step S1065: Based on the last updated second rotation matrix and second translation vector, determine a second rigid body transformation matrix for further coordinate transformation of the coordinate-transformed 3D head model.
[0082] Here, as an example, the second rigid body transformation matrix can be calculated using formula (9).
[0083] (9).
[0084] in, Let be the second rigid body transformation matrix.
[0085] Step S106 aims to find a more accurate coordinate transformation matrix, namely the second rigid body transformation matrix, so that the surface of the head model after the initial transformation is precisely aligned with the preset region. This step uses an iterative nearest-point algorithm framework to constrain the search within the preset region.
[0086] See again Figure 1 Step S107: Determine the spatial transformation matrix for performing coordinate transformation on the image data based on the first rigid body transformation matrix and the second rigid body transformation matrix.
[0087] Here, the product of the first rigid body transformation matrix and the second rigid body transformation matrix is determined as the spatial transformation matrix used to perform coordinate transformation on the image data. As an example, the spatial transformation matrix can be calculated using formula (10).
[0088] (10).
[0089] Step S108: Based on the spatial transformation matrix, transform the image data from the image coordinate system to the physical coordinate system where the head of the target object is located.
[0090] Here, a spatial transformation matrix is used to transform the image data of the target object's head from the virtual coordinate system defined by its acquisition device, i.e., the image coordinate system, to the actual physical coordinates representing the real position of the target object's head, i.e., spatial registration of the image data.
[0091] In practical applications, the above spatial registration method can be applied to a transcranial magnetic navigation system, which can map and overlay virtual stimulation targets on the actual head position of the patient (i.e., the target object) in real time on the screen, thereby enabling precise transcranial magnetic stimulation therapy.
[0092] This application provides a spatial registration method for head images that is highly accurate and robust without the need for physical markers. By combining coarse registration of facial feature points with fine registration of preset region points, it effectively overcomes the sensitivity to initial position and the tendency to get trapped in local optima that are common in traditional methods. This method avoids errors and patient discomfort caused by marker displacement, simplifies the clinical process, and improves registration accuracy and stability through a strategy of coarse feature point registration plus fine local point cloud registration. Furthermore, the good initial values provided by feature point registration enable rapid convergence of fine registration, thereby improving the overall registration efficiency.
[0093] This application provides a spatial registration method for head images, which improves registration accuracy and efficiency.
[0094] Based on the same concept, this application also provides a spatial registration device for head images corresponding to the spatial registration method for head images provided in the above embodiments. Since the principle of the device in this application is similar to the spatial registration method for head images in the above embodiments of this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.
[0095] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a spatial registration device for head images provided in an embodiment of this application.
[0096] like Figure 4 As shown in the illustration, the spatial registration device 410 for head images provided in this application embodiment includes: Module 411 is used to construct a three-dimensional model of the head of the target object based on key head data extracted from the image data of the head of the target object. The first extraction module 412 is used to extract the first facial feature points from the three-dimensional head model to obtain the first facial feature point set of the target object. The second extraction module 413 is used to extract second facial feature points from the three-dimensional point cloud data of the target object's face to obtain the second facial feature point set of the target object. The first rigid body transformation matrix determination module 414 is used to determine a first rigid body transformation matrix for performing coordinate transformation on the three-dimensional head model based on the first facial feature point set and the second facial feature point set. The first transformation module 415 is used to perform coordinate transformation on the three-dimensional head model using the first rigid body transformation matrix. The second rigid body transformation matrix determination module 416 is used to determine a second rigid body transformation matrix for further coordinate transformation of the three-dimensional head model based on the coordinate-transformed head model. The spatial transformation matrix determination module 417 is used to determine the spatial transformation matrix for performing coordinate transformation on the image data based on the first rigid body transformation matrix and the second rigid body transformation matrix. The second transformation module 418 is used to transform the image data from the image coordinate system to the physical coordinate system where the head of the target object is located, based on the spatial transformation matrix.
[0097] Furthermore, the first rigid body transformation matrix determination module 414 is specifically used for: Determine the first geometric center point corresponding to the first set of facial feature points and the second geometric center point corresponding to the second set of facial feature points, respectively. Based on the first geometric center point, determine the vector of each first facial feature point relative to the first geometric center point to obtain a first vector set; and based on the second geometric center point, determine the vector of each second facial feature point relative to the second geometric center point to obtain a second vector set. Construct a covariance matrix based on the first vector set and the second vector set; Decompose the covariance matrix; Based on the decomposed covariance matrix, a first rotation matrix is determined for rotating the first set of facial feature points. Based on the first geometric center point, the second geometric center point, and the first rotation matrix, a first translation vector is determined for translating the first set of facial feature points; Based on the first rotation matrix and the first translation vector, a first rigid body transformation matrix is determined for transforming the coordinates of the first set of facial feature points.
[0098] Furthermore, when determining the first rotation matrix for rotating the first set of facial feature points based on the decomposed covariance matrix, the first rigid body transformation matrix determination module 414 is specifically used for: The product of the left singular vector matrix and the transpose of the right singular vector matrix in the decomposed covariance matrix is determined as the initial rotation matrix; If the determinant of the initial rotation matrix is a preset value, then the initial rotation matrix is determined as the first rotation matrix; If the determinant of the initial rotation matrix is not a preset value, then the preset columns of the right singular vector matrix are inverted; the product of the left singular vector matrix and the transpose of the right singular vector matrix corresponding to the inverted preset columns is determined as the first rotation matrix.
[0099] Furthermore, the second rigid body transformation matrix determination module 416 is specifically used for: Taking each second facial feature point in the second facial feature point set as the center, a preset region corresponding to each second facial feature point is divided in the three-dimensional point cloud data; For each scan point in each preset region, determine the surface point with the smallest distance from that scan point from the coordinate-transformed 3D head model; Based on all scan points and the surface points corresponding to all scan points, determine the spatial distance error and Jacobian matrix between all scan points and their corresponding surface points; Based on the spatial distance error between all scan points and the corresponding surface points and the Jacobian matrix, a second rotation matrix for rotating the coordinate-transformed 3D head model and a second translation vector for translating the coordinate-transformed 3D head model are determined. The second rotation matrix and the second translation vector are updated iteratively until the spatial distance error is no greater than a preset error value or the number of updates reaches a preset number, at which point the updating of the second rotation matrix and the second translation vector is stopped. Based on the last updated second rotation matrix and second translation vector, a second rigid body transformation matrix is determined for further coordinate transformation of the coordinate-transformed 3D head model.
[0100] Furthermore, the second transformation module 418 is specifically used to: determine the product of the first rigid body transformation matrix and the second rigid body transformation matrix as a spatial transformation matrix for performing coordinate transformation on the image data.
[0101] This application provides a spatial registration device for head images, which improves registration accuracy and efficiency.
[0102] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0103] like Figure 5 As shown, the electronic device 500 includes a processor 510, a memory 520, and a bus 530.
[0104] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate via the bus 530. When the machine-readable instructions are executed by the processor 510, they can perform the operations described above. Figure 1 , Figure 2 and Figure 3 The steps of the spatial registration method for the head image in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.
[0105] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 , Figure 2 and Figure 3 The steps of the spatial registration method for the head image in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.
[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.
[0107] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0108] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0109] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0110] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method of spatial registration of head images, characterized by, The method comprises: constructing a head three-dimensional model of the target object based on head key data extracted from image data of the head of the target object; extracting first facial feature points from the head three-dimensional model to obtain a first facial feature point set of the target object; extracting second facial feature points from three-dimensional point cloud data of the face of the target object to obtain a second facial feature point set of the target object; determining a first rigid transformation matrix for coordinate transformation of the head three-dimensional model based on the first facial feature point set and the second facial feature point set; performing coordinate transformation on the head three-dimensional model by using the first rigid transformation matrix; determining a second rigid transformation matrix for re-coordinate transformation of the head three-dimensional model after coordinate transformation based on the head three-dimensional model after coordinate transformation; determining a spatial transformation matrix for coordinate transformation of the image data according to the first rigid transformation matrix and the second rigid transformation matrix; transforming the image data from an image coordinate system to a physical coordinate system in which the head of the target object is located based on the spatial transformation matrix.
2. The method of spatial registration of head images according to claim 1, wherein, The method comprises: determining a first geometric center point corresponding to the first facial feature point set and a second geometric center point corresponding to the second facial feature point set respectively; determining a vector of each of the first facial feature points relative to the first geometric center point to obtain a first vector set according to the first geometric center point, and determining a vector of each of the second facial feature points relative to the second geometric center point to obtain a second vector set according to the second geometric center point; constructing a covariance matrix based on the first vector set and the second vector set; decomposing the covariance matrix; determining a first rotation matrix for rotating the first facial feature point set based on the decomposed covariance matrix; determining a first translation vector for translating the first facial feature point set according to the first geometric center point, the second geometric center point and the first rotation matrix; determining a first rigid transformation matrix for coordinate transformation of the first facial feature point set based on the first rotation matrix and the first translation vector.
3. The method of spatial registration of head images according to claim 2, wherein, The method comprises: determining a product of transposition of a left singular vector matrix and a right singular vector matrix in the decomposed covariance matrix as an initial rotation matrix; if a determinant of the initial rotation matrix is a preset value, determining the initial rotation matrix as the first rotation matrix; if the determinant of the initial rotation matrix is not the preset value, inverting a preset column of the right singular vector matrix, and determining a product of transposition of the left singular vector matrix and the right singular vector matrix corresponding to the inverted preset column as the first rotation matrix.
4. The method of spatial registration of head images of claim 1, wherein, The second rigid transformation matrix for re-coordinate transformation of the head three-dimensional model after the coordinate transformation is determined based on the second face feature point set and the first face feature point set. A preset region corresponding to each second face feature point is divided in the three-dimensional point cloud data, with each second face feature point in the second face feature point set as a center; For each scanning point in each preset region, a surface point with a minimum distance to the scanning point is determined from the head three-dimensional model after the coordinate transformation; Based on all scanning points and surface points corresponding to the scanning points, a spatial distance error and a Jacobian matrix of all scanning points and corresponding surface points are determined; Based on the spatial distance error and the Jacobian matrix of all scanning points and surface points corresponding to the scanning points, a second rotation matrix for rotating the head three-dimensional model after the coordinate transformation and a second translation vector for translating the head three-dimensional model after the coordinate transformation are determined; The second rotation matrix and the second translation vector are iteratively updated until the spatial distance error is not greater than a preset error value or the number of iterations reaches a preset number, and the updating of the second rotation matrix and the second translation vector is stopped; Based on the second rotation matrix and the second translation vector after the last update, a second rigid transformation matrix for re-coordinate transformation of the head three-dimensional model after the coordinate transformation is determined.
5. The method of spatial registration of head images of claim 1, wherein, The spatial transformation matrix for coordinate transformation of the image data is determined according to the first rigid transformation matrix and the second rigid transformation matrix. The product of the first rigid transformation matrix and the second rigid transformation matrix is determined as the spatial transformation matrix for coordinate transformation of the image data.
6. An apparatus for spatial registration of head images, characterized by The device comprises: A construction module configured to construct a head three-dimensional model of a target object based on head key data extracted from image data of a head of the target object; A first extraction module configured to extract first face feature points from the head three-dimensional model to obtain a first face feature point set of the target object; A second extraction module configured to extract second face feature points from three-dimensional point cloud data of a face of the target object to obtain a second face feature point set of the target object; A first rigid transformation matrix determination module configured to determine a first rigid transformation matrix for coordinate transformation of the head three-dimensional model based on the first face feature point set and the second face feature point set; A first transformation module configured to perform coordinate transformation on the head three-dimensional model by using the first rigid transformation matrix; A second rigid transformation matrix determination module configured to determine a second rigid transformation matrix for re-coordinate transformation of the head three-dimensional model after the coordinate transformation based on the head three-dimensional model after the coordinate transformation; A spatial transformation matrix determination module configured to determine a spatial transformation matrix for coordinate transformation of the image data according to the first rigid transformation matrix and the second rigid transformation matrix; A second transformation module configured to transform the image data from an image coordinate system to a physical coordinate system in which the head of the target object is located based on the spatial transformation matrix.
7. Apparatus for spatial registration of head images according to claim 6, characterized in that The first rigid transformation matrix determination module is specifically used for: determining a first geometric center point corresponding to the first facial feature point set and a second geometric center point corresponding to the second facial feature point set, respectively; determining a vector of each of the first facial feature points relative to the first geometric center point according to the first geometric center point to obtain a first vector set, and determining a vector of each of the second facial feature points relative to the second geometric center point according to the second geometric center point to obtain a second vector set; constructing a covariance matrix based on the first vector set and the second vector set; decomposing the covariance matrix; determining a first rotation matrix for rotating the first facial feature point set based on the decomposed covariance matrix; determining a first translation vector for translating the first facial feature point set according to the first geometric center point, the second geometric center point and the first rotation matrix; determining a first rigid transformation matrix for coordinate transformation of the first facial feature point set based on the first rotation matrix and the first translation vector.
8. Apparatus for spatial registration of head images according to claim 7, characterized in that When the first rotation matrix for rotating the first facial feature point set is determined based on the decomposed covariance matrix, the method further includes: determining a product of a left singular vector matrix and a transpose of a right singular vector matrix in the decomposed covariance matrix as an initial rotation matrix; if a determinant of the initial rotation matrix is a preset value, determining the initial rotation matrix as the first rotation matrix; if the determinant of the initial rotation matrix is not the preset value, taking a preset column of the right singular vector matrix in reverse; determining a product of the left singular vector matrix and a transpose of the right singular vector matrix corresponding to the reversed preset column as the first rotation matrix.
9. An electronic device, comprising: The method includes: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the head image spatial registration method according to any one of claims 1 to 5.
10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the head image spatial registration method according to any one of claims 1 to 5.