Mixed reality optical localization arbitrary sequence four-point registration navigation method and mixed reality optical localization arbitrary sequence four-point registration navigation system

Abstract

The invention discloses a mixed reality optical localization arbitrary sequence four-point registration navigation method and system, and belongs to the technical field of image processing and space registration. The method comprises the following steps: reconstructing a three-dimensional model of a target part according to MRI or CT image data; arranging at least four physical mark points which can be identified by an optical positioning system on the surface of the target area, and acquiring three-dimensional coordinate data of each mark point through an optical tracking system; selecting at least four physical mark points in any sequence to form a registration point set; identifying a corresponding relationship between the physical mark points selected in any sequence and the corresponding mark points in the three-dimensional model; and calculating a coordinate transformation matrix between a real coordinate system and a virtual model coordinate system according to the rigid body transformation parameters. According to the technical scheme, the flexibility and precision of the registration process are improved, the operation difficulty is reduced, the system robustness is enhanced, and it is ensured that accurate mapping between the real space and the virtual space is efficiently achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and spatial registration, and particularly relates to a mixed reality optical positioning arbitrary sequence four-point registration navigation method and system. BACKGROUND

[0002] The existing navigation system is widely applied to positioning and visual operation of a target region, and the core thereof is to realize spatial correspondence between a virtual model and an actual scene through key technologies such as image registration, spatial tracking and visual superposition. Among them, the mixed reality (MR) technology has become an important means for enhancing spatial positioning and accurate navigation because of the ability to superimpose a three-dimensional virtual model on a real scene in real time and to interact synchronously. In cooperation with an optical positioning tracking system, high-precision three-dimensional coordinate information can be obtained, thereby supporting spatial mapping between the model and the actual object. The current spatial registration method mostly adopts preset marker points, and the coordinate transformation relationship is calculated by sequentially selecting a plurality of points in a specific order, and then the spatial registration between the virtual model and the actual target is completed.

[0003] However, the traditional multi-point sequential registration strategy has significant limitations. The user must strictly select the marker points according to the system specified order, which restricts the flexibility of actual operation and increases the memory burden. In the actual operation process, some marker points may be preferentially visible due to factors such as occlusion and body position adjustment, thereby interfering with the registration process. If the point selection order is wrong, the entire registration process must be performed again, which affects the work efficiency and system usability. How to properly solve the above problems has become an urgent task in the industry. SUMMARY

[0004] The present application provides a mixed reality optical positioning arbitrary sequence four-point registration navigation method and system to improve the flexibility and precision of the registration process, reduce the operation difficulty, enhance the system robustness, and ensure efficient and accurate mapping of real and virtual spaces.

[0005] According to a first aspect of the present application, a mixed reality optical positioning arbitrary sequence four-point registration navigation method is provided, which comprises: reconstructing a three-dimensional model of a target site according to MRI or CT image data; setting at least four physical marker points capable of being recognized by an optical positioning system on the surface of the target region, and collecting three-dimensional coordinate data of each of the marker points through an optical tracking system; selecting at least four physical marker points in any order to form a registration point set; An algorithm of non-corresponding point set registration based on singular value decomposition is used to identify the correspondence between the arbitrarily ordered physical marker points and the corresponding marker points in the three-dimensional model, and to calculate the rigid transformation parameters; A coordinate transformation matrix between the real coordinate system and the virtual model coordinate system is calculated according to the rigid transformation parameters, so as to realize coordinate space mapping.

[0006] In one embodiment, further comprising: The rigid transformation is applied to the coordinates of the corresponding marker points in the virtual model coordinate system to obtain corresponding converted coordinates. The registration error is calculated by calculating the deviation between the converted coordinates and the coordinates of the corresponding actual physical marker points, and the registration accuracy is evaluated according to the registration error.

[0007] In one embodiment, further comprising: Based on the preset identification information or spatial distribution characteristics of each physical marker point, the correspondence between the physical marker points selected in arbitrary order by the user and the corresponding marker points in the three-dimensional model is automatically identified. According to the identified correspondence, the coordinate data of the physical marker points is sorted to match the order of the corresponding marker points in the virtual model, so as to establish consistent point sets for registration calculation.

[0008] In one embodiment, further comprising: The registration error is compared with a preset error threshold value. When the registration error exceeds the error threshold value, a predetermined fault-tolerant processing is performed, including reselecting the physical marker points and repeating the arbitrary order point set registration step until the registration error is reduced to within the error threshold value.

[0009] In one embodiment, the calculation of the coordinate transformation matrix comprises: A rotation matrix and a translation vector are extracted from the rigid transformation obtained by singular value decomposition. A homogeneous coordinate transformation matrix is constructed by the rotation matrix and the translation vector to convert the coordinates in the real coordinate system to the virtual model coordinate system.

[0010] In one embodiment, further comprising: No less than four marker points are arbitrarily selected from the physical marker points for registration calculation. When the number of selected physical marker points exceeds four, a least square algorithm based on singular value decomposition is used to calculate the best rigid transformation between the physical marker point set and the corresponding point set in the virtual model, so as to minimize the overall registration error of all selected points.

[0011] According to a second aspect of the present application, there is provided a mixed reality optical positioning arbitrary sequence four-point registration navigation system, comprising: a reconstruction module configured to reconstruct a three-dimensional model of a target region based on MRI or CT image data; a collection module configured to set at least four physical marker points on a surface of the target region, the physical marker points being identifiable by an optical positioning system, and to collect three-dimensional coordinate data of each of the physical marker points by an optical tracking system; a formation module configured to select at least four of the physical marker points in an arbitrary sequence to form a registration point set; a calculation module configured to identify a correspondence between the physical marker points selected in the arbitrary sequence and corresponding marker points in the three-dimensional model by a singular value decomposition non-correspondence point set registration algorithm, and to calculate rigid transformation parameters; a transformation module configured to calculate a coordinate transformation matrix between a real coordinate system and a virtual model coordinate system based on the rigid transformation parameters, so as to realize coordinate space mapping.

[0012] In one embodiment, the reconstruction module, the collection module, the formation module, the calculation module and the transformation module are controlled to implement any of the above-mentioned mixed reality optical positioning arbitrary sequence four-point registration navigation methods.

[0013] According to a third aspect of the present application, there is provided an electronic device, comprising: a communication interface, a processor, a memory; wherein the memory is configured to store program instructions, the program instructions, when executed by the processor in communication connection with the memory through the communication interface, implement any of the above-mentioned mixed reality optical positioning arbitrary sequence four-point registration navigation methods.

[0014] According to a fourth aspect of the present application, there is provided a computer readable storage medium, the computer readable storage medium storing computer program instructions, the computer program instructions, when executed by a computer (e.g., a processor in the computer), implement any of the above-mentioned mixed reality optical positioning arbitrary sequence four-point registration navigation methods.

[0015] In summary, the application provides a mixed reality optical positioning arbitrary sequence four-point registration navigation method and system, which comprises the following steps: reconstructing a three-dimensional model of a target part according to MRI or CT image data; arranging at least four physical marker points capable of being recognized by an optical positioning system on the surface of a target region, and collecting three-dimensional coordinate data of each of the marker points through an optical tracking system; selecting at least four of the physical marker points in an arbitrary sequence to form a registration point set; identifying the correspondence between the physical marker points selected in the arbitrary sequence and corresponding marker points in the three-dimensional model through a singular value decomposition non-corresponding point set registration algorithm, and calculating rigid transformation parameters; and calculating a coordinate transformation matrix between a real coordinate system and a virtual model coordinate system according to the rigid transformation parameters to realize coordinate space mapping.

[0016] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and attained by the structure particularly pointed out in the written description and claims hereof.

[0017] The technical solutions of the present application will be further described in detail below with the help of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 A flow chart of a mixed reality optical positioning arbitrary sequence four-point registration navigation method provided for an embodiment of the present application; Figure 2 A flow chart of another mixed reality optical positioning arbitrary sequence four-point registration navigation method provided for an embodiment of the present application; Figure 3 A flow chart of still another mixed reality optical positioning arbitrary sequence four-point registration navigation method provided for an embodiment of the present application; Figure 4 A flow chart of still another mixed reality optical positioning arbitrary sequence four-point registration navigation method provided for an embodiment of the present application; Figure 5 A flow chart of still another mixed reality optical positioning arbitrary sequence four-point registration navigation method provided for an embodiment of the present application; Figure 6A flowchart of another mixed reality optical positioning arbitrary order four-point registration navigation method provided for embodiments of the present invention; Figure 7 A structural diagram of a mixed reality optical positioning arbitrary order four-point registration navigation system provided for embodiments of the present invention; Figure 8 This is a structural diagram of an electronic device provided as an embodiment of the present invention. Detailed Implementation

[0020] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0021] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0022] like Figure 1 As shown, this invention provides a mixed reality optical positioning arbitrary order four-point registration and navigation method, which includes: In step S11, a three-dimensional model of the target area is reconstructed based on MRI or CT image data; In step S12, at least four physical markers that can be identified by the optical positioning system are set on the surface of the target area, and the three-dimensional coordinate data of each marker is collected by the optical tracking system. In step S13, at least four of the physical marker points are selected in any order to form a registration point set; In step S14, the correspondence between the arbitrarily selected physical marker points and the corresponding marker points in the three-dimensional model is identified by a non-correspondence point set registration algorithm based on singular value decomposition, and the rigid transformation parameters are calculated. In step S15, a coordinate transformation matrix between the real coordinate system and the virtual model coordinate system is calculated according to the rigid transformation parameters, so as to realize the coordinate space mapping.

[0023] In one embodiment, the image three-dimensional modeling, optical coordinate acquisition, point set registration and rigid transformation matrix construction are aimed at improving the flexibility and accuracy of spatial registration. The fixed sequence requirement in the traditional multi-point registration process is overcome, and the synergistic advantages of mixed reality and high-precision optical tracking are fully utilized. Through the automatic identification of the dynamic and arbitrary sequence correspondence between the physical marker points and the virtual model marker points, efficient mapping and fusion between the spatial coordinate systems are realized, which lays a technical foundation for precise positioning and visual interaction in the fields of intelligent navigation and virtual reality.

[0024] According to the image data such as MRI or CT, a three-dimensional model of the target part is reconstructed. The three-dimensional model not only reflects the real spatial structure of the target region, but also provides the necessary data basis for subsequent spatial coordinate mapping and accurate registration. For example, in the craniocerebral scene, the three-dimensional model automatically reconstructed by the image software can fully reflect the various anatomical features and spatial distribution, laying a foundation for the setting of physical marker points and the mapping relationship between virtual and actual.

[0025] At least four physical marker points that can be accurately recognized by the optical positioning system need to be set on the surface of the target region. These marker points, as the anchoring reference for spatial registration, must be reasonably distributed and easily perceived by the acquisition system. For example, in actual scenarios, retroreflective spheres or special coded points are often used as physical marker points, and their spatial positions are captured in real time by infrared optical cameras. The optical tracking system can continuously acquire three-dimensional coordinate information of the marker points at a high frame rate and low delay, and transmit it to the registration module in real time. The arrangement of the marker points must ensure that they are not collinear and have good geometric stability, so as to avoid the decrease of the accuracy of the registration algorithm.

[0026] At least four physical marker points are selected in an arbitrary order to form a registration point set, which improves the flexibility and fault tolerance of the registration operation. According to the visibility, occlusion or operation habit on site, the points are randomly selected for acquisition. For example, in complex environments, some marker points may be temporarily invisible due to occlusion or contamination, so visible points can be preferentially selected for registration, greatly improving the adaptability and practicality of the system.

[0027] The automatic corresponding relationship between the selected points and the model points is recognized and the rigid transformation parameters are calculated by a non-corresponding point set registration algorithm based on singular value decomposition (SVD). As a basic tool in the field of modern computational geometry and registration, the SVD algorithm can efficiently solve the optimal rotation matrix and translation vector between two sets of points, thereby completing the rigid mapping of the spatial coordinate system. Unlike the traditional one-to-one point set, the technical solution in the embodiment automatically exhausts or intelligently matches all possible point set corresponding relationships, calculates the registration error one by one, and finally selects the point set arrangement with the smallest error as the best solution, aiming at the situation that the order of selected points is not fixed and the numbering may be chaotic in actual operation. For example, if the four points are numbered as A, B, C, and D, the system will automatically analyze the point set in any input order, such as C, A, D, and B, compare it with the numbering in the model, and automatically establish the optimal one-to-one mapping relationship based on the coordinate data.

[0028] The rigid transformation parameters obtained according to the above steps, i.e. the rotation matrix and the translation vector, are further used to calculate the coordinate transformation matrix between the real coordinate system and the virtual model coordinate system, thereby realizing efficient mapping of the coordinate space. For example, the target model reconstructed in three dimensions is accurately superimposed on the augmented reality terminal or navigation platform according to the actual target spatial attitude, ensuring that the virtual information is highly consistent with the actual space, and facilitating the data flow and scene seamless switching between multiple coordinate systems.

[0029] The mixed reality optical positioning arbitrary sequence four-point registration navigation method includes the following steps: (1) Three-dimensional modeling: according to MRI or CT image data, a three-dimensional model of the target region is reconstructed to provide a structured data basis for subsequent spatial registration.

[0030] (2) Mark point setting: At least four physical mark points that can be recognized by an optical positioning system are set on the surface of the target region to ensure that the mark points are reasonably distributed and easy to track.

[0031] (3) Optical acquisition: The three-dimensional spatial coordinates of each physical mark point are obtained by using an infrared optical tracking system, and the collected data are transmitted to the registration system in real time.

[0032] (4) Point set selection and registration: The user is allowed to select no less than four physical mark points in any order, and the system automatically recognizes the corresponding relationship between the selected points and the three-dimensional model mark points based on the non-corresponding point set registration algorithm based on singular value decomposition (SVD), and calculates the rigid transformation parameters accordingly. This mechanism ensures that the spatial point mapping relationship can be accurately established regardless of the order of point set selection.

[0033] (5) Coordinate space transformation: According to the obtained rigid transformation parameters, the coordinate transformation matrix between the real coordinate system and the virtual model coordinate system is calculated to realize high-precision spatial registration.

[0034] (6) Mixed reality visualization: superimpose the three-dimensional reconstruction model to the actual scene through the mixed reality device to realize the spatial visualization and navigation under the augmented reality.

[0035] The method fuses high-precision three-dimensional modeling, flexible point set registration and real-time spatial visualization, and is suitable for various application scenarios requiring high-precision spatial mapping and virtual-reality fusion.

[0036] The spatial registration system based on mixed reality and optical positioning and tracking technology is composed of a mixed reality terminal, an optical registration module and a workstation. The optical registration module includes a binocular optical camera, a positioning reference frame and physical marker points, and can accurately obtain three-dimensional spatial information of a target region. The workstation is used to run software modules related to spatial registration and visualization, mainly including an image processing module and a spatial registration module. The image processing module has functions of preoperative image import, data management, multi-modal image fusion, automatic generation of fiber tract tracking, preoperative planning and three-dimensional model reconstruction, and supports export of processing results to the spatial registration module through a standard data format (such as v3d). The spatial registration module realizes data interaction with the optical positioning module, and presents three-dimensional model data of the target region through the mixed reality terminal and the display screen to realize spatial superposition and visualization.

[0037] The spatial registration method needs to collect and fuse image data of the target region, complete three-dimensional model reconstruction by using the image processing module, and export in a standard format. The three-dimensional model file is imported into the spatial registration module. During the registration operation, at least four physical marker points are selected in any order, a non-corresponding point set registration algorithm based on singular value decomposition (SVD) is used to automatically establish a mapping relationship between the physical space coordinate system and the virtual model coordinate system, solve the rigid body transformation parameters and complete the spatial registration. The three-dimensional model is displayed through the mixed reality terminal to realize the augmented visualization and navigation under spatial mapping.

[0038] The purpose of registration is to unify two sets of points in different coordinate systems to the same coordinate system, which can be achieved by a set of mapping transformation H:

[0039] Wherein, R represents a rotation matrix, T represents a translation vector, V represents a perspective change vector, and S represents a scale factor.

[0040] Since the transformation in the embodiment is a rigid body transformation, there is no deformation, only rotation and translation transformation, and therefore V is a zero vector and S = 1. At this time:

[0041] Wherein, the rotation matrix R is a 3×3 matrix, and the translation vector T is a 3×1 matrix.

[0042] If known is a point in the optical positioning tracker coordinate system, is a point in the MR module coordinate system, then the transformation relationship of three-dimensional coordinate points and is:

[0043] Further, the point set in one coordinate system is converted to the point set in another coordinate system, and the best rotation matrix R and translation vector T between the two corresponding point sets need to be found, and at least three known points are required for each point set, and the three points cannot be coplanar.

[0044] If f represents the error between the source point set P and the target point set Q under the action of the transformation matrix H, then solving the optimal conversion matrix is to solve the transformation matrix (R, T) when the error is minimized, that is:

[0045] Where, Pi represents a point in the source point set P, and Qi represents a point in the target point set Q.

[0046] Further, obtaining the best transformation matrix can be divided into the following steps: Step 1, find the center of mass of the two point sets:

[0047]

[0048] Where, and are 3x1 vectors, and represent the center of mass of the source point set P and the target point set Q, respectively, and N represents the number of points in the source point set P or the target point set Q (the points in the point sets P and Q are in one-to-one correspondence).

[0049] Step 2, remove the translation components of the two point sets,

[0050]

[0051] Process the non-corresponding relationship (exhaustive method), generate all possible corresponding relationships arrangements) For each arrangement , based on each arrangement corresponding point set: .

[0052] Based on the non-corresponding point set, if represents the source point set under the transformation matrix Under the action of the target point set The optimal transformation matrix is solved when the error is minimized , According to the following formula.

[0053]

[0054] Calculate the covariance matrix: Based on singular value decomposition (SVD), find the best rotation matrix R.

[0055] Singular value decomposition is to decompose the matrix Into three matrices, namely:

[0056] Where, is a diagonal matrix arranged in descending order, is the first orthogonal matrix, is the second orthogonal matrix. Where T represents the matrix transpose, and the final rotation matrix can be calculated by the first orthogonal matrix and the second orthogonal matrix , that is:

[0057] If the number of points in the point set is greater than three, the least squares solution is obtained.

[0058] Step 3, solve the best translation vector

[0059] Target point set can be obtained by the following formula: + =

[0060] From the above formula combined with the above step (1) can be obtained: -

[0061] By the above formula to solve the best rotation matrix and translation vector , that is, the transformation matrix can be reconstructed.

[0062] By calculating the transformation matrix each time, we can record the error of the registration calculation:

[0063] Find the arrangement corresponding to the minimum error by comparing the error:

[0064] Optimal rotation:

[0065] Optimal translation:

[0066] Further, in the present embodiment, if Representing the conversion of the source point set P in the optical positioning tracker coordinate system {Tr} to the target point set Q in the MR module coordinate system {h}, it can be represented as: Q=HP The entire system in the present embodiment includes three coordinate systems, namely: the optical positioning tracker coordinate system {Tr}, the virtual model coordinate system {V}, and the MR module coordinate system {h}; The ultimate goal of spatial registration is to register the virtual model by coordinating the conversion relationship between the three coordinate systems, and to solve That is, the conversion matrix between the virtual model coordinate system {V} and the optical positioning tracker coordinate system {Tr}. Solve That is, the conversion matrix between the optical positioning tracker coordinate system {Tr} and the MR module coordinate system {h}. Solve That is, the conversion matrix between the virtual model coordinate system {V} and the MR module coordinate system {h}. Finally, complete the conversion from the virtual model coordinate system to the MR module coordinate system: It can be seen that through the non-corresponding point set registration algorithm based on singular value decomposition, the real-time dynamic fusion of the preoperative three-dimensional modeling image and the navigation target visual image is realized, the efficiency of selecting points during operation is improved, and the accuracy of navigation positioning during operation is improved.

[0067] The technical solution in the present embodiment realizes a spatial registration process with marker points selected in any order, significantly improving the operation flexibility and system fault tolerance. Through the non-corresponding point set registration algorithm based on singular value decomposition, the registration process is efficient and accurate, effectively reducing the operation complexity, improving the practicality and robustness of the navigation system, and meeting the high-precision spatial mapping requirements.

[0068] In one embodiment, as Figure 2 shown, further comprising the following steps S21-S22: In step S21, the rigid body transformation is applied to the coordinates of the corresponding marker points in the virtual model coordinate system to obtain the corresponding conversion coordinates; In step S22, the deviation between the conversion coordinates and the coordinates of the corresponding actual physical marker points is calculated to obtain the registration error, and the registration accuracy is evaluated according to the registration error.

[0069] In one embodiment, based on the accuracy evaluation and feedback mechanism of the spatial registration result, the rigid body transformation is applied to the marker points in the virtual model coordinate system, the registration error is quantitatively analyzed by comparing the spatial positions of the actual physical marker points, and the accuracy of the entire registration process is evaluated accordingly. The consistency of the spatial mapping of the virtual model and the actual object can be guaranteed, and the effectiveness of the registration algorithm can be verified. In actual application, there are multiple coordinate system conversions between the virtual model and the physical target, and any small registration error can affect the accuracy of subsequent navigation, display or guidance.

[0070] The rigid body transformation obtained by the algorithm (including the rotation matrix and the translation vector) is applied to the known marker point coordinates in the virtual model, and the coordinates are converted to the reference coordinate system in the real world. By comparing the converted coordinates with the physical marker point coordinates collected by optical positioning, the Euclidean distance difference between each pair of corresponding points is calculated, and finally the registration error index is obtained. For example, if the calculated coordinates of a certain marker point in the virtual model after rigid body transformation deviate slightly from the actual physical point coordinates, it indicates that the registration accuracy is high; otherwise, the point set selection or algorithm adjustment needs to be performed again. This method not only provides an objective evaluation standard for the registration process, but also helps the operator to find and correct abnormal points in time, significantly improving the stability and practicality of the system in complex environments.

[0071] In one embodiment, as shown in Figure 3 the steps S31-S32 are further included: In step S31, based on the preset identification information or spatial distribution characteristics of each physical marker point, the correspondence between the physical marker points selected in any order by the user and the corresponding marker points in the three-dimensional model is automatically identified; In step S32, according to the identified correspondence, the coordinate data of the physical marker points is sorted to match the order of the corresponding marker points in the virtual model, so as to establish a consistent point set for registration calculation.

[0072] In one embodiment, the mechanism of automatically corresponding the physical marker points and the virtual model marker points in the spatial registration process is realized, and through intelligent identification and data sorting, the strict dependence on the selection order in the traditional registration process is broken. No matter what order the physical marker points are selected by the operator, automatic mapping and matching can be realized based on the preset identification information (such as unique number, optical code) or the spatial distribution characteristics (such as geometric shape, relative distance) of the points. The flexibility and user-friendliness of spatial registration are significantly improved, and the registration failure caused by incorrect operation order is effectively reduced.

[0073] After a user collects multiple physical marker points in an arbitrary order in a real-world scenario, the system analyzes the identification information or spatial distribution of each point. For example, if each marker point has a unique code, a one-to-one match can be achieved directly based on the code; if some marker points do not have obvious codes, they can be automatically matched based on their relative spatial positions by calculating geometric features such as distances and angles between points. After matching, the system intelligently sorts the collected physical point coordinate data to match the order of the marker points in the virtual model, thus forming a set of "correspondingly ordered" points for subsequent registration algorithms to solve for rigid body transformation parameters.

[0074] Suppose there are four marker points on the virtual model, numbered M1, M2, M3, and M4, while the field acquisition sequence is M3, M1, M4, and M2. The system analyzes the identifiers or spatial characteristics of the acquisition points to determine that M3 corresponds to the virtual point M3, and so on, sorting the acquisition points in the correct order to M1, M2, M3, and M4, corresponding one-to-one with the marker points on the virtual model. Then, registration calculations can be performed on this sorted set of points to obtain high-precision spatial transformation parameters. Through this mechanism, even if the user's acquisition order changes, the spatial registration process can still automatically correct itself, ensuring algorithm robustness and final registration accuracy.

[0075] In one embodiment, such as Figure 4 As shown, it also includes the following steps S41-S42: In step S41, the registration error is compared with a preset error threshold; In step S42, when the registration error exceeds the error threshold, a predetermined fault tolerance process is performed, including reselecting physical marker points and repeating the arbitrary sequence point set registration step until the registration error drops below the error threshold.

[0076] In one embodiment, by setting an error threshold and providing dynamic feedback, the accuracy of the registration results and the overall robustness of the system are effectively guaranteed. After completing the rigid body transformation and point set registration, the registration error is automatically calculated and compared with the preset error threshold. The registration error, as an objective accuracy evaluation indicator, can reflect the magnitude of the deviation between the current registration result and the ideal target in real time. When the registration error is below the threshold, it indicates that the registration result has reached the expected accuracy and can be safely used for subsequent spatial mapping and information overlay; conversely, it indicates that there is an anomaly in the current point set or data, and fault tolerance processing needs to be initiated.

[0077] The fault-tolerant processing usually includes reselecting part or all of the physical marker points and repeating the whole process of point set registration calculation. According to the actual situation on site, the order of point selection or the point is replaced until the registration error is reduced to the predetermined threshold range, thereby ensuring that the spatial transformation result output by the system has high accuracy and usability. For example, in a certain round of registration, if part of the points cause the measurement error to be too large due to reasons such as occlusion, reflection, positioning abnormality, etc., the problem will be automatically detected and the points will be reacquired or replaced until an optimal point set is selected, and high-quality spatial registration is achieved. The feedback and adaptive mechanism based on the error threshold value constructs a closed-loop quality control system for the whole registration process, which can effectively prevent the reliability of spatial mapping from being affected by accidental errors.

[0078] In one embodiment, as shown in Figure 5 Further comprising the following steps S51-S52: In step S51, a rotation matrix and a translation vector are extracted from the rigid body transformation obtained by singular value decomposition; In step S52, a homogeneous coordinate transformation matrix is constructed by the rotation matrix and the translation vector to convert the coordinates in the real coordinate system to the virtual model coordinate system.

[0079] In one embodiment, after point set registration, high-precision conversion between different spatial coordinate systems is achieved. After completing the spatial registration of the non-corresponding point set, the rigid body transformation parameters, i.e., the rotation matrix and the translation vector, are obtained by the singular value decomposition (SVD) algorithm, and then a homogeneous coordinate transformation matrix is constructed based on this to realize the unified representation of spatial points between different reference systems. The SVD algorithm calculates the rotation matrix and the translation vector that can optimally map a set of points from one coordinate system to another coordinate system by minimizing the error between the point sets. For example, in the image navigation scene, if the coordinates of the physically collected marker points and the corresponding points in the virtual model are known, the system can obtain the optimal rotation and translation in three-dimensional space by SVD solution to achieve rigid body registration. Thereafter, the two parameters are integrated into a homogeneous coordinate transformation matrix, which can simply and effectively convert the real space coordinates of any point into virtual space coordinates and support efficient conversion between multiple coordinate systems. For example, assuming that the rotation matrix R and the translation vector T describe the rigid body change of the space, the corresponding homogeneous transformation matrix can be represented as a 4×4 matrix, which can be directly used for batch transformation of point coordinates and matrix chain operation in subsequent spatial mapping, data fusion and visualization processes.

[0080] In one embodiment, as shown in Figure 6 Further comprising the following steps S61-S62: In step S61, at least four marker points are selected from the physical marker points to participate in the registration calculation; In step S62, when the number of selected physical marker points exceeds four, a least square algorithm based on singular value decomposition is used to calculate the best rigid transformation between the physical marker point set and the virtual model corresponding point set to minimize the overall registration error of all selected points.

[0081] In one embodiment, in a multi-point registration scenario, the registration accuracy and system robustness are improved by mathematical optimization. In actual operation, according to the field conditions or specific requirements, no less than four physical marker points are selected from all physical marker points to participate in registration calculation. When the number of marker points participating in registration exceeds four, a least square algorithm based on singular value decomposition (SVD) is automatically introduced to optimize all selected points as a whole, and finally the optimal rigid transformation parameters are obtained, thereby significantly improving the flexibility and anti-interference ability of the registration process.

[0082] Specifically, the least square algorithm can make full use of the information of redundant points, automatically balance and correct the influence of single-point abnormality or measurement error, and minimize the overall registration error. For example, in an actual scenario, a physical marker point may have an abnormal value due to local occlusion or insufficient positioning accuracy. The traditional four-point unique solution method is easily disturbed by abnormal points, resulting in a sharp increase in spatial registration error. In contrast, by using five, six or more marker points to participate in registration, the SVD least square algorithm can automatically adjust the weights of each point, reduce the influence of abnormal points, and make the final transformation parameters closer to the true spatial relationship, thereby improving the overall accuracy and reliability of the system.

[0083] For example, assuming that six reasonably distributed physical marker points are set, and the data deviates due to reflection of a certain point during the collection process, after least square optimization, the registration solution does not rely too much on the abnormal point, but takes the global optimum. This mechanism not only improves the accuracy of spatial registration, but also greatly enhances the robustness and user experience of the system in actual complex environments, ensuring that the spatial mapping of the virtual model and the real target is always highly consistent.

[0084] In one embodiment, Figure 7 is a block diagram of a mixed reality optical positioning arbitrary sequence four-point registration navigation system according to an exemplary embodiment. As shown in Figure 7 The mixed reality optical positioning arbitrary sequence four-point registration navigation system includes a reconstruction module 71, an acquisition module 72, a formation module 73, a calculation module 74, and a transformation module 75.

[0085] The reconstruction module 71 is configured to reconstruct a three-dimensional model of a target site according to MRI or CT image data; The acquisition module 72 is configured to set at least four physical marker points on the surface of the target region, wherein the physical marker points can be recognized by an optical positioning system, and to acquire three-dimensional coordinate data of each of the physical marker points by using an optical tracking system; The forming module 73 is configured to select at least four physical marker points in any order to form a registration point set; The computing module 74 is configured to identify the correspondence between the physical marker points selected in any order and the corresponding marker points in the three-dimensional model by using a singular value decomposition-based non-correspondence point set registration algorithm, and calculate rigid transformation parameters. The transformation module 75 is configured to calculate a coordinate transformation matrix between a real coordinate system and a virtual model coordinate system according to the rigid transformation parameters, so as to realize coordinate space mapping.

[0086] The reconstruction module 71, the acquisition module 72, the forming module 73, the computing module 74 and the transformation module 75 included in the mixed reality optical positioning arbitrary sequence four-point registration navigation system block diagram are controlled to perform the mixed reality optical positioning arbitrary sequence four-point registration navigation method described in any of the above embodiments.

[0087] As shown in Figure 8 The present application provides an electronic device 800, which comprises a communication interface, a processor 801, and a memory 802. The memory 802 is configured to store program instructions, which, when executed by the processor 801 in communication connection with the memory 802 through the communication interface, reconstruct a three-dimensional model of a target site according to MRI or CT image data; set at least four physical marker points capable of being recognized by an optical positioning system on a surface of a target region, and acquire three-dimensional coordinate data of each of the marker points through an optical tracking system; select at least four physical marker points in any order to form a registration point set; identify the correspondence between the physical marker points selected in any order and the corresponding marker points in the three-dimensional model by using a singular value decomposition-based non-correspondence point set registration algorithm, and calculate rigid transformation parameters; and calculate a coordinate transformation matrix between a real coordinate system and a virtual model coordinate system according to the rigid transformation parameters, so as to realize coordinate space mapping.

[0088] The present application provides a computer readable storage medium, which stores computer program instructions, which, when executed by a processor, reconstruct a three-dimensional model of a target site according to MRI or CT image data; set at least four physical marker points capable of being recognized by an optical positioning system on a surface of a target region, and acquire three-dimensional coordinate data of each of the marker points through an optical tracking system; select at least four physical marker points in any order to form a registration point set; identify the correspondence between the physical marker points selected in any order and the corresponding marker points in the three-dimensional model by using a singular value decomposition-based non-correspondence point set registration algorithm, and calculate rigid transformation parameters; and calculate a coordinate transformation matrix between a real coordinate system and a virtual model coordinate system according to the rigid transformation parameters, so as to realize coordinate space mapping.

[0089] It should be understood that the specific features, operations and details described herein in relation to the method of the present application can be applied analogously to the device and system of the present application, or vice versa. In addition, each step of the method of the present application described above can be performed by a corresponding component or unit of the device or system of the present application.

[0090] It should be understood that each module / unit of the device of the present application can be implemented in whole or in part by software, hardware, firmware, or a combination thereof. Each module / unit can be embedded in a processor of a computer device in hardware or firmware form, or independent of the processor, or stored in a memory of the computer device in software form to be invoked by the processor to perform the operations of each module / unit. Each module / unit can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.

[0091] In one embodiment, a computer device is provided, which includes a memory and a processor, the memory having stored thereon computer instructions executable by the processor, the computer instructions instructing the processor to perform each step of the method of an embodiment of the present application when executed by the processor. The computer device can be a server, a terminal, or any other electronic device with necessary computing and / or processing capability in a broad sense. In one embodiment, the computer device can include a processor, a memory, a network interface, a communication interface, and the like connected by a system bus. The processor of the computer device can be configured to provide necessary computing, processing and / or control capability. The memory of the computer device can include a non-volatile storage medium and an internal memory. The non-volatile storage medium can have stored therein or thereon an operating system, a computer program, and the like. The internal memory can provide an environment for running of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the computer device can be configured to connect and communicate with external devices through a network. The computer program, when executed by the processor, performs the steps of the method of the present application.

[0092] The present application can be implemented as a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, causes the steps of the method of an embodiment of the present application to be performed. In one embodiment, the computer program is distributed over a network of network-coupled computer devices or processors such that the computer program is stored, accessed and executed by one or more computer devices or processors in a distributed manner. A single method step / operation, or two or more method steps / operations, can be performed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations can be performed by one or more computer devices or processors and one or more other method steps / operations can be performed by one or more other computer devices or processors. One or more computer devices or processors can perform a single method step / operation, or perform two or more method steps / operations.

[0093] It will be appreciated by those skilled in the art that the method steps of the present application can be instructed by a computer program to relevant hardware such as a computer device or processor, which can be stored in a non-transitory computer-readable storage medium, which when executed causes the steps of the present application to be performed. Any reference herein to a memory, storage, database or other medium can include non-volatile and / or volatile memory as the case can be. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state disk, and the like. Examples of volatile memory include random access memory (RAM), external cache memory, and the like.

[0094] The various technical features described above can be combined in any manner. Although not all possible combinations thereof are described, any combination of the technical features should be considered to be within the scope of the present specification, as long as such a combination does not result in a contradiction.

[0095] Finally, it should be noted that the above-described embodiments are merely intended to illustrate the technical solutions of the present application, not to limit the technical solutions of the present application; even though the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some or all of the technical features; and such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A mixed reality optical positioning arbitrary order four-point registration navigation method, characterized in that, include: Reconstruct a three-dimensional model of the target area based on MRI or CT image data; At least four physical markers that can be identified by an optical positioning system are set on the surface of the target area, and the three-dimensional coordinate data of each marker is collected by an optical tracking system; At least four of the physical marker points are selected in any order to form a registration point set; The non-corresponding point set registration algorithm of singular value decomposition is used to identify the correspondence between the physical marker points selected in arbitrary order and the corresponding marker points in the three-dimensional model, and to calculate the rigid body transformation parameters. The coordinate transformation matrix between the real coordinate system and the virtual model coordinate system is calculated based on the rigid body transformation parameters to achieve coordinate space mapping.

2. The mixed reality optical positioning arbitrary order four-point registration navigation method as described in claim 1, characterized in that, Also includes: The rigid body transformation is applied to the coordinates of the corresponding marked points in the virtual model coordinate system to obtain the corresponding transformed coordinates; The deviation between the transformed coordinates and the corresponding actual physical marker coordinates is calculated to obtain the registration error, and the registration accuracy is evaluated based on the registration error.

3. The mixed reality optical positioning arbitrary order four-point registration navigation method as described in claim 1, characterized in that, Also includes: Based on the preset identification information or spatial distribution characteristics of each physical marker point, the correspondence between the physical marker points selected by the user in any order and the corresponding marker points in the three-dimensional model is automatically identified. Based on the identified correspondence, the coordinate data of the physical marker points are sorted to match the order of the corresponding marker points in the virtual model, thereby establishing a consistent point set for registration calculation.

4. The mixed reality optical positioning arbitrary order four-point registration navigation method as described in claim 2, characterized in that, Also includes: The registration error is compared with a preset error threshold; When the registration error exceeds the error threshold, a predetermined fault-tolerance process is performed, including reselecting physical marker points and repeating the arbitrary sequence point set registration step until the registration error drops below the error threshold.

5. The mixed reality optical positioning arbitrary order four-point registration navigation method as described in claim 1, characterized in that, The calculation of the coordinate transformation matrix includes: Extract the rotation matrix and translation vector from the rigid body transformation obtained using singular value decomposition; A homogeneous coordinate transformation matrix is ​​constructed using the rotation matrix and translation vector to transform the coordinates in the real coordinate system to the virtual model coordinate system.

6. The mixed reality optical positioning arbitrary order four-point registration navigation method as described in claim 1, characterized in that, Also includes: Select at least four physical markers to participate in the registration calculation; When the number of selected physical markers exceeds four, the least squares algorithm based on singular value decomposition is used to calculate the optimal rigid body transformation between the set of physical markers and the corresponding set of points in the virtual model, so as to minimize the overall registration error of all selected points.

7. A mixed reality optical positioning arbitrary order four-point registration navigation system, characterized in that, include: The reconstruction module is used to reconstruct a three-dimensional model of the target area based on MRI or CT image data. The acquisition module is used to set at least four physical markers that can be identified by the optical positioning system on the surface of the target area, and to acquire the three-dimensional coordinate data of each marker through the optical tracking system; A forming module is used to select at least four of the physical marker points in any order to form a registration point set; The calculation module is used to identify the correspondence between the physical marker points selected in any order and the corresponding marker points in the three-dimensional model through the non-corresponding point set registration algorithm of singular value decomposition, and to calculate the rigid body transformation parameters. The transformation module is used to calculate the coordinate transformation matrix between the real coordinate system and the virtual model coordinate system based on the rigid body transformation parameters, so as to realize coordinate space mapping.

8. The mixed reality optical positioning arbitrary order four-point registration navigation system as described in claim 7, characterized in that: The reconstruction module, the acquisition module, the forming module, the calculation module, and the transformation module are controlled to execute the mixed reality optical positioning arbitrary order four-point registration navigation method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, include: Communication interface, processor, memory; The memory is used to store program instructions, which, when executed by the processor connected to the memory via the communication interface, enable the electronic device to implement the mixed reality optical positioning arbitrary order four-point registration navigation method according to any one of claims 1 to 6.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the computer, the computer implements the mixed reality optical positioning arbitrary order four-point registration navigation method as described in any one of claims 1 to 6.

Images