Mixed reality deep fusion spatial positioning viewing, registration and path planning system

Abstract

The present invention discloses a mixed reality deep fusion spatial positioning viewing, alignment and path planning system. By introducing a positioning point planning module, a mixed reality calibration module and a mixed reality positioning module, a tracking device and a positioning device, it is possible to simply and quickly realize spatial deep fusion, that is, to achieve a complete overlapping effect between the three-dimensional model and the entity in space.

Description

Technical Field

[0001] The present invention relates to the technical field of mixed reality deep fusion spatial positioning, and in particular to a mixed reality deep fusion spatial positioning viewing, registration and path planning system. Background Art

[0002] Existing mixed reality systems require manual movement and rotation of the 3D model to make the 3D model and the entity reach a consistent position in space. This process is cumbersome, time-consuming and requires high operating skills. Summary of the invention

[0003] The purpose of the present invention is to overcome the defects of the prior art and provide a mixed reality deep fusion spatial positioning viewing, alignment and path planning system that can integrate virtual models and real space, move, rotate and scale the virtual model, change the color of the virtual model in the real space, and can be applied in different fields.

[0004] The technical solution to achieve the purpose of the present invention is: a mixed reality deep fusion spatial positioning viewing, registration and path planning system, comprising the following steps:

[0005] S1. View: Share the field of view between virtual mixed reality devices. First, make the mixed reality device scan the physical space. After the scan is completed, start the mixed reality device, start the tablet computer to scan the space and find a plane. After the plane is found, three logos appear on the tablet computer screen. The mixed reality device turns on the camera and starts to recognize the three logos. It calculates the position and angle of the camera based on the three logos, records this position and sets the spatial anchor point. After completion, the space fusion of the two devices is completed.

[0006] S2. Registration: Import the 3D model into the positioning point planning module, and combine the morphology of the positioning device to obtain the three model positioning points P1, P2 and P3 corresponding to the 3D model; obtain the calibration coordinate conversion matrix of the mixed reality device and the tracking device through the hybrid tracker Import the same 3D model and the planned three positioning point information into the mixed reality positioning module to achieve deep integration of the entity and the 3D model in the mixed reality scene;

[0007] S3. Path planning.

[0008] The S1 described in the above technical solution includes the following steps:

[0009] (1) Start the first mixed reality device as the server, first scan the real scene server to see if there is a spatial anchor point, and if so, import and store the anchor point information of the current space;

[0010] (2) Start the other secondary mixed reality device as a client. The client downloads the spatial anchor point information from the server, finds the spatial point in the current space, and sets the spatial anchor point of the device to be consistent with the anchor point information of the current space in the first secondary mixed reality device, thus completing the spatial fusion of the two corresponding devices.

[0011] When the number of mixed reality devices is greater than 2, repeat the above (2); start importing the virtual model on the first mixed reality device, and other virtual reality devices will also import the same virtual model.

[0012] The calibration coordinate conversion matrix in S2 of the above technical solution is The position matrix obtained by the mixed reality device and the tracking device in the mixed reality calibration module identifying the hybrid tracker at the same time and Calculated,

[0013]

[0014] The obtaining of the three model positioning points P1, P2 and P3 corresponding to the three-dimensional model in S2 of the above technical solution is specifically to calculate the positions of the points P1, P2 and P3 on the model, and the steps are as follows:

[0015] A. Add a bounding box to define the exposed area of ​​the 3D model and ensure that all positioning points are within the bounding box;

[0016] B. Define the desired direction V of the positioning device;

[0017] C. Select a landmark point on the model surface within the bounding box as the point where P1 is located;

[0018] D. Generate a sphere with P1 as the center and d12 as the radius, and calculate the intersection line between the sphere surface and the 3D model in the bounding box. The point on the intersection line is the candidate point of P2. P2};

[0019] E. For {C P2}Calculate the surface curvature of the midpoint and sort them from large to small; try to generate a sphere with the candidate point as the center and d23 as the radius in the sorted candidate points, calculate the intersection line between the surface of the sphere and the three-dimensional model in the bounding box, and the point on the intersection line is the candidate point of P3{C P3};

[0020] F. For {C P3}Calculate the surface curvature of the midpoint and sort it from large to small; try to calculate the P3 candidate point C in turn among the sorted candidate points P3Is the distance to P1 d13? If so, calculate whether the angle between the normal vector of the plane where P1, alternative P2, and alternative P3 are located and the expected direction V of the positioning device is less than 20 degrees. If so, further determine whether the positioning device collides with the three-dimensional model except the needle tip in the current posture. If there is no collision, find three needle tip points that meet the requirements, otherwise continue to try the next P3 alternative point.

[0021] G. Get {P1, P2, P3} PL .

[0022] In S2 of the above technical solution, the position matrix of the tracker on the positioning device is tracked by the tracking device By calibrating the coordinate transformation matrix Calculate the position matrix of the tracker in the mixed reality coordinate system Since the tracker on the positioning device has a rigid fixed relationship with the three pinpoints below it, the three transformation matrices and Combination Calculate the positions of the three needle tips in the mixed reality coordinate system {P1, P2, P3} ms , based on the information of the three model positioning points {P1, P2, P3} on the imported 3D model PL , real-time calculation of the transformation matrix of the 3D model from the initial position to the three needle points of the follower locator That is, the three model positioning points of the three-dimensional model can be bound one by one to the three pinpoints of the entity's positioning device in the mixed reality coordinate system, so that the positioning device and the three-dimensional model can maintain a relative posture relationship when the positioning points are planned in real time in the mixed reality device; then, by moving the three pinpoints of the positioning device to the three entity positioning points on the corresponding entities in the scene, the deep fusion of the entity and the three-dimensional model in the mixed reality scene, that is, complete overlap, can be achieved.

[0023] The S1 described in the above technical solution specifically includes:

[0024] a. The mixed reality device starts image recognition, identifies three logo images, and records the positions of the logo images in the mixed reality device;

[0025] b. Calculate the current position of the tablet computer camera in the virtual mixed reality device based on the positions provided by the three identification images, and record the position;

[0026] c. Synchronize the space between the two devices according to the currently recorded position and set the anchor point;

[0027] dAfter the setup is completed, import the virtual model in the mixed reality device, and the same virtual model will also be imported on the tablet;

[0028] e. Changing the shape and position of the model in the hybrid display device will also result in corresponding changes on the tablet;

[0029] The tablet computer can view the virtual model in multiple directions.

[0030] The tracker described in the above technical solution is a photosensitive ball or a positioning QR code.

[0031] A mixed reality deep fusion spatial positioning viewing, registration and path planning system for application in medical puncture.

[0032] After adopting the above technical solution, the present invention has the following positive effects:

[0033] (1) The present invention can integrate the virtual model with the real space, move, rotate, and scale the virtual model, and change the color of the virtual model in the real space. It can be applied in different fields, and the real world and the virtual scene of the hybrid display device are overlapped to form a real and complete display scene visually. The changes in images, sounds, touch, etc. in the virtual scene are synchronized to the real world.

[0034] (2) The present invention can simply and quickly realize spatial depth fusion by introducing a positioning point planning module, a mixed reality calibration module and a mixed reality positioning module, a tracking device and a positioning device, so as to achieve a complete overlapping effect between the three-dimensional model and the entity in space.

[0035] (3) The present invention can support multiple devices at the same time, with multiple three-party perspectives for viewing. The delay between multiple devices is low, the operations in the first device can be synchronized to other devices, the transmission between devices is smooth and clear, and multiple people can view the same model for discussion and research.

[0036] (4) The present invention can also record the entire operation steps to facilitate subsequent viewing, and can also use a tablet computer to observe individuals without being limited by the number of people. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments and in conjunction with the accompanying drawings, wherein

[0038] Figure 1 This is a principle block diagram of the positioning point planning module of the present invention;

[0039] Figure 2 A schematic diagram of a bounding box of a positioning point planning module of the present invention;

[0040] Figure 3A schematic diagram of the desired direction of the positioning device of the positioning point planning module of the present invention;

[0041] Figure 4 Schematic diagram of the spatial position relationship of three needle tips of the present invention;

[0042] Figure 5 It is a schematic diagram of point P1 of the present invention;

[0043] Figure 6 It is a schematic diagram of point P2 of the present invention;

[0044] Figure 7 This is a flow chart of the positioning point planning module of the present invention;

[0045] Figure 8 It is a schematic diagram of a mixed reality calibration module of the present invention;

[0046] Fig. 9 It is a schematic diagram of a mixed reality positioning module of the present invention;

[0047] Fig.10 It is a schematic diagram of the following mode of the present invention;

[0048] Fig.11 It is a schematic diagram after registration of the present invention. DETAILED DESCRIPTION

[0049] (Example 1)

[0050] The present invention comprises the following steps:

[0051] S1. View: Share the field of view between virtual mixed reality devices. First, make the mixed reality device scan the physical space. After the scan is completed, start the mixed reality device, start the tablet computer to scan the space and find a plane. After the plane is found, three logos appear on the tablet computer screen. The mixed reality device turns on the camera and starts to recognize the three logos. It calculates the position and angle of the camera based on the three logos, records this position and sets the spatial anchor point. After completion, the space fusion of the two devices is completed.

[0052] S2. Registration: Import the 3D model into the positioning point planning module, and combine the morphology of the positioning device to obtain the three model positioning points P1, P2 and P3 corresponding to the 3D model; obtain the calibration coordinate conversion matrix of the mixed reality device and the tracking device through the hybrid tracker Import the same 3D model and the planned three positioning point information into the mixed reality positioning module to achieve deep integration of the entity and the 3D model in the mixed reality scene; see Figure 1 .

[0053] S3. Path planning.

[0054] See Figure 7 , said S1 comprises the following steps:

[0055] (1) Start the first mixed reality device as the server, first scan the real scene server to see if there is a spatial anchor point, and if so, import and store the anchor point information of the current space;

[0056] (2) Start the other secondary mixed reality device as a client. The client downloads the spatial anchor point information from the server, finds the spatial point in the current space, and sets the spatial anchor point of the device to be consistent with the anchor point information of the current space in the first secondary mixed reality device, thus completing the spatial fusion of the two corresponding devices.

[0057] When the number of mixed reality devices is greater than 2, repeat the above (2); start importing the virtual model on the first mixed reality device, and other virtual reality devices will also import the same virtual model.

[0058] The S1 specifically includes:

[0059] a. The mixed reality device starts image recognition, identifies three logo images, and records the positions of the logo images in the mixed reality device;

[0060] b. Calculate the current position of the tablet computer camera in the virtual mixed reality device based on the positions provided by the three identification images, and record the position;

[0061] c. Synchronize the space between the two devices according to the currently recorded position and set the anchor point;

[0062] dAfter the setup is completed, import the virtual model in the mixed reality device, and the same virtual model will also be imported on the tablet;

[0063] e. Changing the shape and position of the model in the hybrid display device will also result in corresponding changes on the tablet;

[0064] The tablet computer can view the virtual model in multiple directions.

[0065] In S1, the second mixed reality device has a screen recording function, and when the screen recording is turned on, the screen is recorded and saved.

[0066] See Figure 8-11 , the calibration coordinate transformation matrix in S2 The position matrix obtained by the mixed reality device and the tracking device in the mixed reality calibration module identifying the hybrid tracker at the same time and Calculated,

[0067] The coordinate origin of the mixed reality device is determined when the device is started. The movement of the device will not change the origin. The mixed reality device has SLAM spatial positioning capability, so it is determined The value will not change afterwards.

[0068] The obtaining of the three model positioning points P1, P2 and P3 corresponding to the three-dimensional model in S2 is specifically to calculate the positions of the points P1, P2 and P3 on the model, and the steps are as follows:

[0069] A. Add a bounding box to define the exposed area of ​​the 3D model, and ensure that all positioning points are within the bounding box. Figure 2 ;

[0070] B. Define the desired direction V of the positioning device, see Figure 3 ;

[0071] C. Select a landmark point on the model surface within the bounding box as the point where P1 is located. Figure 5 ;

[0072] D. Generate a sphere with P1 as the center and d12 as the radius, and calculate the intersection line between the sphere surface and the 3D model in the bounding box. The point on the intersection line is the candidate point of P2. P2},See Figure 4 ;

[0073] E. For {C P2}Calculate the surface curvature of the midpoint and sort them from large to small; try to generate a sphere with the candidate point as the center and d23 as the radius in the sorted candidate points, calculate the intersection line between the surface of the sphere and the three-dimensional model in the bounding box, and the point on the intersection line is the candidate point of P3{C P3},See Figure 6 ;

[0074] F. For {C P3}Calculate the surface curvature of the midpoint and sort it from large to small; try to calculate the P3 candidate point C in turn among the sorted candidate points P3 Is the distance to P1 d13? If so, calculate whether the angle between the normal vector of the plane where P1, alternative P2, and alternative P3 are located and the expected direction V of the positioning device is less than 20 degrees. If so, further determine whether the positioning device collides with the three-dimensional model except the needle tip in the current posture. If there is no collision, find three needle tip points that meet the requirements, otherwise continue to try the next P3 alternative point.

[0075] G. Get {P1, P2, P3} PL .

[0076] In S2, the position matrix of the tracker on the positioning device is tracked by the tracking device. By calibrating the coordinate transformation matrix Calculate the position matrix of the tracker in the mixed reality coordinate system Since the tracker on the positioning device has a rigid fixed relationship with the three pinpoints below it, the three transformation matrices and Combination Calculate the positions of the three needle tips in the mixed reality coordinate system {P1, P2, P3} ms , based on the information of the three model positioning points {P1, P2, P3} on the imported 3D model PL , real-time calculation of the transformation matrix of the 3D model from the initial position to the three needle points of the follower locator That is, the three model positioning points of the 3D model can be bound one by one with the three pinpoints of the entity's positioning device in the mixed reality coordinate system, so that the positioning device and the 3D model can maintain the relative posture relationship when the positioning points are planned in real time in the mixed reality device; then, by moving the three pinpoints of the positioning device to the three entity positioning points on the corresponding entity in the scene, the entity and the 3D model can be deeply integrated in the mixed reality scene, that is, completely overlapped. When the coincidence of the 3D model and the corresponding entity is observed, the positioning and registration is completed, the positioning and registration operation is exited, and the final transformation matrix is ​​recorded To maintain the registration result.

[0077] The positioning device is mainly used to position the spatial positions of the three needle tips at the end of the positioning device.

[0078] Path planning includes the following steps:

[0079] 1) Open the mixed virtual reality device and import the mixed virtual model into the mixed reality device;

[0080] 2) Using gestures to move and rotate objects in a virtual mixed display device, the virtual model is integrated with the objects in the real world, which is only a preliminary integration;

[0081] 3) To perform precise fusion, first turn on the image recognition function in the virtual hybrid display device;

[0082] 4) Recognize the three prepared images, and calculate the position and angle of the point required by the current tool according to the position and angle of the recognized images, that is, the point in the real space in the virtual mixed display device;

[0083] 5) Check whether there are four small balls on the image, three in the center of the image and the fourth at the location of the selected point. If so, start selecting feature points. If not, repeat the above steps.

[0084] 6) Moving tool: When the touch position of the image tool reaches the position of the entity feature point, the current position information is recorded;

[0085] Repeat step 6 to select multiple points. The number of points is determined by the user (usually 3-7 points are selected);

[0086] 7) After the feature points are selected, confirm the fitting;

[0087] 8) After fitting is completed, check the position and angle relationship between the 3D model and the entity to see if they are completely aligned. If there is a deviation, repeat steps 3-6;

[0088] 9) Get the position of this point, select 3 to 7 different points in total,

[0089] 10) After the feature points are selected, matching begins;

[0090] 11) Matching the virtual model with objects in the real world according to the algorithm;

[0091] 12) Use the perspective ball to see through the virtual model, and you can see the model inside through the model outside the virtual model

[0092] 13) Subsequent operations such as changing color.

[0093] (Example 2)

[0094] A mixed reality deep fusion spatial positioning viewing, alignment and path planning system is applied in medical puncture, which is suitable for neurosurgery puncture, thoracic puncture, etc.

[0095] The working principle of the invention is: using spatial mapping, objects can be placed on a real surface, and occluding holograms based on other holograms and real-world objects helps convince users that these holograms are actually located in their space; when the mixed reality device collects new data about the environment, if changes are made to the environment, the spatial surface will appear, disappear and change.

[0096] In the present invention, the number of photosensitive balls is 4, and there is a specific spatial geometric design. This is mainly because multiple trackers are used in the present invention, so the photosensitive balls only need to satisfy that they are not in a straight line and the area of ​​the quadrilateral formed is not less than about 60 square centimeters.

[0097] The third person of the present invention can see the operations and virtual models in the operator's field of view, and the second device can also achieve its own current field of view. The perception of space between the two devices does not affect each other, and the second device is viewed as a third-party field of view. It can support multiple devices at the same time, and there are multiple three-party perspectives for viewing. The delay between multiple devices is low. The operations in the first device can be synchronized to other devices. The transmission between devices is smooth and clear, and multiple people view the same model for discussion and research.

[0098] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A mixed reality deep fusion spatial positioning viewing, registration and path planning system, characterized in that: The following steps are involved: S1. View: Share the field of view between virtual mixed reality devices. First, make the mixed reality device scan the physical space. After the scan is completed, start the mixed reality device, start the tablet computer to scan the space and find a plane. After the plane is found, three logos appear on the tablet computer screen. The mixed reality device turns on the camera and starts to recognize the three logos. It calculates the position and angle of the camera based on the three logos, records this position and sets the spatial anchor point. After completion, the space fusion of the two devices is completed. S2. Registration: Import the 3D model into the positioning point planning module, and combine the morphology of the positioning device to obtain the three model positioning points P1, P2 and P3 corresponding to the 3D model; obtain the calibration coordinate conversion matrix of the mixed reality device and the tracking device through the hybrid tracker , import the same 3D model and the planned three positioning point information into the mixed reality positioning module to achieve deep integration of the entity and the 3D model in the mixed reality scene; The obtaining of the three model positioning points P1, P2 and P3 corresponding to the three-dimensional model in S2 is specifically to calculate the positions of the points P1, P2 and P3 on the model, and the steps are as follows: A. Add a bounding box to define the exposed area of ​​the 3D model and ensure that all positioning points are within the bounding box; B. Define the desired direction V of the positioning device; C. Select a landmark point on the model surface within the bounding box as the point where P1 is located; D. Generate a sphere with P1 as the center and d12 as the radius, and calculate the intersection line between the sphere surface and the 3D model in the bounding box. The point on the intersection line is the candidate point of P2. P2 }; E. For {C P2 }Calculate the surface curvature of the midpoint and sort them from large to small; try to generate a sphere with the candidate point as the center and d23 as the radius in the sorted candidate points, calculate the intersection line between the surface of the sphere and the three-dimensional model in the bounding box, and the point on the intersection line is the candidate point of P3{C P3 }; F. For {C P3 }Calculate the surface curvature of the midpoint and sort it from large to small; try to calculate the P3 candidate point C in turn among the sorted candidate points P3 Is the distance to P1 equal to d13? If so, calculate whether the angle between the normal vector of the plane where P1, P2, and P3 are located and the expected direction V of the positioning device is less than 20 degrees. If so, further determine whether the positioning device collides with the three-dimensional model except the needle tip in the current posture. If there is no collision, find three needle tip points that meet the requirements, otherwise continue to try the next P3 alternative point. G. Get {P1, P2, P3} PL ; S3, path planning; S3 path planning includes the following steps: 1) Open the mixed virtual reality device and import the mixed virtual model into the mixed reality device; 2) Using gestures to move and rotate objects in a virtual mixed display device, the virtual model is integrated with the objects in the real world, which is only a preliminary integration; 3) To perform precise fusion, first turn on the image recognition function in the virtual hybrid display device; 4) Recognize the three prepared images, and calculate the position and angle of the point required by the current tool according to the position and angle of the recognized images, that is, the point in the real space in the virtual mixed display device; 5) Check whether there are four small balls on the image, three in the center of the image and the fourth at the location of the selected point. If so, start selecting feature points. If not, repeat the above steps. 6) Move the tool. When the touch position of the image tool reaches the position of the entity feature point, record the current position information; repeat step 6 to select multiple points, and the number of points is determined by the user; 7) After the feature points are selected, confirm the fitting; 8) After fitting is completed, check the position and angle relationship between the 3D model and the entity to see if they are completely aligned. If there is a deviation, repeat steps 3 to 6; 9) Get the position of this point; 10) After the feature points are selected, matching begins; 11) Matching virtual models with objects in the real world according to algorithms; 12) Use the perspective ball to see through the virtual model, and view the model inside through the model outside the virtual model; 13) Subsequent operations.

2. The mixed reality deep fusion spatial positioning viewing, registration and path planning system according to claim 1, characterized in that: The S1 comprises the following steps: (1) Start the first mixed reality device as the server, first scan the real scene server to see if there is a spatial anchor point, and if so, import and store the anchor point information of the current space; (2) Start other mixed reality devices as clients. The clients download the spatial anchor point information from the server, find this spatial point in the current space, and set the spatial anchor point of the device to be consistent with the anchor point information of the current space in the first mixed reality device, thus completing the spatial fusion of the two devices. When the number of mixed reality devices is greater than 2, repeat the above (2). Start importing the virtual model on the first mixed reality device, and other virtual reality devices will also import the same virtual model.

3. The mixed reality deep fusion spatial positioning viewing, registration and path planning system according to claim 1, characterized in that: The calibration coordinate transformation matrix in S2 The position matrix obtained by the mixed reality device and the tracking device in the mixed reality calibration module identifying the hybrid tracker at the same time and Calculated, -1 ， In the formula, is the transformation matrix of the anchor point p on the model relative to the tracking device w; -1 It is the inverse matrix of the transformation matrix of the positioning point p on the model relative to the mixed reality system ms, that is, the transformation matrix of the mixed reality system ms relative to the positioning point p on the model.

4. The mixed reality deep fusion spatial positioning viewing, registration and path planning system according to claim 1, characterized in that: In S2, the position matrix of the tracker on the positioning device is tracked by the tracking device. , by calibrating the coordinate transformation matrix Calculate the position matrix of the tracker in the mixed reality coordinate system , -1 , where -1 is the inverse matrix of the transformation matrix of the mixed reality system ms relative to the tracking device w, that is, the transformation matrix of the tracking device w relative to the mixed reality system ms; Since the tracker on the positioning device has a rigid fixed relationship with the three pinpoints below it, the three transformation matrices , and Combination Calculate the positions of the three needle tips in the mixed reality coordinate system {P1, P2, P3} ms ,in, is the transformation matrix of the tip P1 relative to the tracker d; is the transformation matrix of the tip P2 relative to the tracker d; is the transformation matrix of the tip P3 relative to the tracker d; According to the information of three model positioning points {P1, P2, P3} on the imported 3D model PL , real-time calculation of the transformation matrix of the 3D model from the initial position to the three needle points of the follower locator , that is, the three model positioning points of the three-dimensional model can be bound one by one with the three pinpoints of the entity's positioning device in the mixed reality coordinate system, so that the positioning device and the three-dimensional model can maintain the relative posture relationship when the positioning points are planned in real time in the mixed reality device; then, by moving the three pinpoints of the positioning device to the three entity positioning points on the corresponding entities in the scene, the deep fusion of the entity and the three-dimensional model in the mixed reality scene can be achieved, that is, complete overlap.

5. The mixed reality deep fusion spatial positioning viewing, registration and path planning system according to claim 2, characterized in that: The S1 specifically includes: a. The mixed reality device starts image recognition, identifies three logos, and records the positions of the logos in the mixed reality device; b. Calculate the current position of the tablet computer camera in the virtual mixed reality device based on the positions provided by the three identification images, and record the position; c. Synchronize the space between the two devices according to the currently recorded position and set the anchor point; dAfter the setup is completed, import the virtual model in the mixed reality device, and the same virtual model will also be imported on the tablet; e. Changing the shape and position of the model in the hybrid display device will also result in corresponding changes on the tablet; The tablet computer can view the virtual model in multiple directions.

6. The mixed reality deep fusion spatial positioning viewing, registration and path planning system according to claim 5, characterized in that: In S1, the second mixed reality device has a screen recording function, and when the screen recording is turned on, the screen is recorded and saved.

7. The mixed reality deep fusion spatial positioning viewing, registration and path planning system according to claim 1, characterized in that: The hybrid tracker is a photosensitive ball or a positioning QR code.

8. Application of the mixed reality deep fusion spatial positioning viewing, registration and path planning system as described in any one of claims 1 to 7 in medical puncture.

Images