Brain surgery path guidance system

Abstract

The application relates to the field of medical instruments, and provides a brain operation path guiding system, which comprises an operation instrument, a face scanning three-dimensional camera, a positioning three-dimensional camera and a processor. The application determines a first pose conversion relationship between an operation instrument positioning code and the operation instrument, a second pose conversion relationship between the face scanning three-dimensional camera and a camera positioning code, a third pose conversion relationship between the positioning three-dimensional camera and the camera positioning code, and a fourth pose conversion relationship between the positioning three-dimensional camera and the operation instrument positioning code, carries out three-dimensional reconstruction based on an MRI image of a patient to obtain an MRI reconstruction model comprising a three-dimensional operation path, and determines a position and an angle offset between the operation instrument and the three-dimensional operation path according to the first, second, third and fourth pose conversion relationships, the MRI reconstruction model and a three-dimensional point cloud model of a patient face transmitted by the face scanning three-dimensional camera, so that the accuracy of an operation position and an operation angle is improved, and the operation risk is reduced.

Description

Technical Field

[0001] This application relates to the field of medical devices, specifically to a brain surgery pathway guidance system. Background Technology

[0002] In current neurosurgical procedures, surgeons rely on clinical experience to perform procedures such as drilling holes in the skull and inserting electrodes into the target surgical area. However, if a surgeon lacks extensive clinical experience, the placement and angle of the tools used during these procedures may be inaccurate, leading to higher safety risks during the surgical process. Summary of the Invention

[0003] This application provides a brain surgery path guidance system to address the issue that in neurosurgery, doctors often need to rely on clinical experience to perform brain surgery operations such as drilling into the skull or inserting electrodes into the target surgical area, which can easily lead to inaccurate tool approach positions and angles, thereby increasing the risk of brain surgery.

[0004] In a first aspect, embodiments of this application provide a brain surgery path guidance system, including surgical instruments, a facial scanning 3D camera, a positioning 3D camera, and a processor:

[0005] The surgical instruments are equipped with surgical instrument positioning codes;

[0006] The facial scanning 3D camera is equipped with a camera positioning code. The facial scanning 3D camera is used to scan the patient's face and transmit the 3D point cloud model established based on the scan results to the processor.

[0007] The positioning 3D camera is used to capture images of the surgical instrument positioning code and the camera positioning code, and transmit them to the processor;

[0008] The processor is used for:

[0009] Obtain the first pose transformation relationship between the surgical instrument positioning code and the surgical instrument, and the second pose transformation relationship between the camera positioning code and the facial scanning 3D camera;

[0010] The images of the surgical instrument positioning code and the camera positioning code are identified to obtain the third pose transformation relationship between the positioning 3D camera and the camera positioning code, and the fourth pose transformation relationship between the positioning 3D camera and the surgical instrument positioning code.

[0011] Three-dimensional reconstruction is performed based on the patient's MRI images to obtain an MRI reconstruction model including a three-dimensional surgical path; wherein, the MRI images include a planar surgical path;

[0012] Based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, the fourth pose transformation relationship, the three-dimensional point cloud model, and the MRI reconstruction model, the positional offset and angular offset between the surgical instrument and the three-dimensional surgical path are determined.

[0013] In one embodiment, the processor is specifically used for:

[0014] The planar surgical path in the patient's MRI image is transformed into a three-dimensional coordinate system to obtain the three-dimensional surgical path;

[0015] Based on the three-dimensional surgical path, the MRI image is pixel-reduced to obtain the target image;

[0016] The target image is reconstructed in three dimensions to obtain an MRI reconstruction model.

[0017] In one embodiment, the processor is specifically used for:

[0018] The target image is reconstructed in three dimensions using the marching cube algorithm to obtain an MRI reconstruction model.

[0019] In one embodiment, the processor is specifically used for:

[0020] Based on the three-dimensional point cloud model and the MRI reconstruction model, the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera is determined;

[0021] Based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, and the fourth pose transformation relationship, the pose information of the surgical instrument under the facial scanning 3D camera is determined;

[0022] Based on the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera and the pose information of the surgical instrument under the facial scanning three-dimensional camera, the positional offset and angular offset between the surgical instrument and the three-dimensional surgical path are determined.

[0023] In one embodiment, the processor is further configured to:

[0024] Based on the three-dimensional point cloud model and the MRI reconstruction model, the fifth pose transformation relationship between the patient's face under the three-dimensional facial scanning camera and the patient's face in the MRI reconstruction model is determined.

[0025] Based on the fifth pose transformation relationship and the three-dimensional surgical path, the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera is determined.

[0026] In one embodiment, the processor is further configured to:

[0027] The three-dimensional point cloud model and the MRI reconstruction model are registered to obtain the fifth pose transformation relationship between the patient's face in the three-dimensional facial scanning camera and the patient's face in the MRI reconstruction model.

[0028] In one embodiment, the processor is further configured to:

[0029] The Lepard algorithm is used to register the 3D point cloud model with the MRI reconstruction model to obtain the fifth pose transformation relationship between the patient's face in the 3D facial scanning camera and the patient's face in the MRI reconstruction model.

[0030] In one embodiment, the processor is further configured to:

[0031] The first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, and the fourth pose transformation relationship are input into the first formula for calculation to obtain the pose information of the surgical instrument under the facial scanning 3D camera.

[0032] In one embodiment, the first formula is as follows:

[0033]

[0034] Wherein, Pose is the pose information of the surgical instrument under the facial scanning 3D camera; for The inverse matrix; This is the first pose transformation relationship; for The inverse matrix; This is the fourth pose transformation relationship; for The inverse matrix; This is the second pose transformation relationship; This is the third pose transformation relationship.

[0035] In one embodiment, the processor is further configured to:

[0036] Based on the three-dimensional surgical path, the second pose transformation relationship, the third pose transformation relationship, and the fifth pose transformation relationship, the pose information of the three-dimensional surgical path under the positioning three-dimensional camera is determined.

[0037] In one embodiment, the brain surgery path guidance system further includes a marker projector, a projector robotic arm, and a facial scanning 3D camera robotic arm;

[0038] The marker projector is used to project the three-dimensional surgical path.

[0039] The projector robotic arm is connected to the marker projector and is used to control the pose of the marker projector to adjust the projection position of the marker projector.

[0040] The robotic arm for facial scanning 3D camera is connected to the facial scanning 3D camera;

[0041] The processor is further configured to acquire a first kinematic relationship of the projector robotic arm and a second kinematic relationship of the facial scanning 3D camera robotic arm; determine a sixth pose transformation relationship between the marker projector and the facial scanning 3D camera based on the first kinematic relationship and the second kinematic relationship; determine the position information of the 3D surgical path under the marker projector based on the sixth pose transformation relationship; and control the projector robotic arm to adjust the current pose of the marker projector to the pose corresponding to the position information.

[0042] The brain surgery path guidance system provided in this application embodiment allows the processor to determine the first pose transformation relationship between the surgical instrument positioning code and the surgical instrument, the second pose transformation relationship between the facial scanning 3D camera and the camera positioning code, the third pose transformation relationship between the positioning 3D camera and the camera positioning code, and the fourth pose transformation relationship between the positioning 3D camera and the surgical instrument positioning code. Based on the patient's MRI images, a 3D reconstruction is performed to obtain an MRI reconstruction model including the 3D surgical path. Then, based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, the fourth pose transformation relationship, the MRI reconstruction model, and the 3D point cloud model of the patient's face transmitted by the facial scanning 3D camera, the positional and angular offsets between the surgical instruments and the 3D surgical path can be determined. This can provide drilling assistance during the surgical process, improve the accuracy of the surgical position and angle, and reduce the safety risks of the surgical process. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of a brain surgery path guidance system provided in an embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] The brain surgery path guidance system provided by the present invention will be described in detail below with reference to embodiments.

[0047] Figure 1 This is a schematic diagram of a brain surgery path guidance system provided in an embodiment of this application. (Refer to...) Figure 1 This application provides a brain surgery path guidance system, which may include a surgical instrument 1, a facial scanning 3D camera 3, a positioning 3D camera 5, a facial scanning 3D camera robotic arm 9, a marker projector 8, a projector robotic arm 10, and a processor 7 (not shown in the figure).

[0048] It should be noted that the brain surgery path guidance system of this application may also include an operating table 6 (not shown in the figure), wherein the marker projector 8 can be connected to the operating table 6 through the projector robotic arm 10.

[0049] The facial scanning 3D camera 3 can be connected to the operating table 6 via the facial scanning 3D camera robotic arm 9.

[0050] In some embodiments, the facial scanning 3D camera robotic arm 9 and the projector robotic arm 10 may also be part of the operating table 6.

[0051] Among them, surgical instrument 1 is equipped with surgical instrument positioning code 2 according to a known relationship, and facial scanning 3D camera 3 is equipped with camera positioning code 4 according to a known relationship.

[0052] Therefore, processor 7 can obtain the first pose transformation relationship between surgical instrument positioning code 2 and surgical instrument 1, and the second pose transformation relationship between camera positioning code 4 and facial scanning 3D camera 3, based on the aforementioned known relationships. Here, pose includes position and orientation (angle). That is, the aforementioned known relationships can be stored in processor 7, and therefore processor 7 can directly read them.

[0053] In this application, the processor 7 can control the projector robotic arm 10. Since the projector robotic arm 10 is integrally connected to the marker projector 8, the pose of the marker projector 8 can be adjusted by adjusting the pose of the projector robotic arm 10. Furthermore, it can control the facial scanning 3D camera robotic arm 9. Since the facial scanning 3D camera robotic arm 9 is integrally connected to the facial scanning 3D camera 3, the pose of the facial scanning 3D camera 3 can be adjusted by adjusting the pose of the facial scanning 3D camera robotic arm 9.

[0054] The marker projector is used to project the three-dimensional surgical path.

[0055] The pose transformation relationship between surgical instrument positioning code 2 and surgical instrument 1 can represent the relative positions of surgical instrument positioning code 2 and surgical instrument 1. Through the pose transformation relationship between surgical instrument positioning code 2 and surgical instrument 1, the specific position of surgical instrument 1 can be determined.

[0056] In this application, surgical instrument 1 can be a surgical robot actuator, a surgical drilling instrument, etc.

[0057] The facial scanning 3D camera 3 and camera positioning code 4 are installed above the patient's face, and the positioning 3D camera 5 is installed on one side of the operating table. The doctor can control the surgical instrument 1 to move closer to the patient. At this time, the positioning 3D camera 5 can simultaneously capture images of the surgical instrument positioning code 2 and the camera positioning code 4.

[0058] It should be noted that the positioning 3D camera 5 can communicate with the processor 7, thereby transmitting the images of the surgical instrument positioning code 2 and the camera positioning code 4 captured to the processor 7.

[0059] The facial scanning 3D camera 3 can scan the patient's face and create a 3D point cloud model. After obtaining the 3D point cloud model, it can be transmitted to the processor 7 for further processing.

[0060] The processor 7 can recognize the images of the surgical instrument positioning code 2 and the camera positioning code 4 to obtain the third pose transformation relationship between the positioning 3D camera 5 and the camera positioning code 4, and the fourth pose transformation relationship between the positioning 3D camera 5 and the surgical instrument positioning code 2. It should be noted that the technique of obtaining the pose transformation relationship based on the positioning code can be implemented using existing technologies, such as the recognition of Aruco markers.

[0061] Three-dimensional reconstruction is performed based on the patient's magnetic resonance imaging (MRI) images to obtain an MRI reconstruction model that includes a three-dimensional surgical path (which can be defined as Tra); wherein, the MRI image contains a planar surgical path, which can be specified by the doctor.

[0062] The MRI reconstruction model includes the patient's face, skull, and ventricles.

[0063] Among them, the patient's magnetic resonance imaging can be an image after the doctor uses 3D Slicer software to calibrate the planar surgical path of the skull on the patient's initial MRI image.

[0064] Furthermore, MRI images contain the three-dimensional coordinate information of each pixel.

[0065] For example, a doctor can mark a point at the beginning and end of the drilling, and the two points can form a planar surgical path.

[0066] Based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, the fourth pose transformation relationship, the three-dimensional point cloud model, and the MRI reconstruction model, the positional offset and angular offset between surgical instrument 1 and the three-dimensional surgical path are determined.

[0067] By determining the positional and angular offsets required to align surgical instrument 1 with the surgical path, data guidance can be provided to the surgeon. This method can also be applied to robot-assisted cranial drilling or brain surgery.

[0068] The brain surgery path guidance system provided in this application embodiment allows the processor 7 to determine the first pose transformation relationship between the surgical instrument positioning code 2 and the surgical instrument 1, the second pose transformation relationship between the facial scanning 3D camera 3 and the camera positioning code 4, the third pose transformation relationship between the positioning 3D camera 5 and the camera positioning code 4, and the fourth pose transformation relationship between the positioning 3D camera 5 and the surgical instrument positioning code 2. Based on the patient's MRI images, a 3D reconstruction is performed to obtain an MRI reconstruction model including the 3D surgical path. Then, based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, the fourth pose transformation relationship, the MRI reconstruction model, and the 3D point cloud model of the patient's face transmitted by the facial scanning 3D camera 3, the positional and angular offsets between the surgical instrument 1 and the 3D surgical path can be determined. This can provide drilling assistance during the surgical process, improve the accuracy of the surgical position and angle, and reduce the safety risks of the surgical process.

[0069] Specifically, when performing three-dimensional reconstruction based on the patient's MRI images to obtain an MRI reconstruction model including the three-dimensional surgical path, the processor 7 can achieve this by executing the following steps:

[0070] Since MRI images include the three-dimensional coordinates of each pixel, the pixels corresponding to the planar surgical path in the patient's MRI image can be converted to a three-dimensional coordinate system, and a three-dimensional surgical path can be obtained after the conversion.

[0071] After obtaining the three-dimensional surgical path, the drilling length and radius can be determined. Based on the drilling length and radius, the pixels along the surgical path in the MRI image are reduced, and the MRI image with reduced pixels is determined as the target image.

[0072] The target image is reconstructed in three dimensions to obtain an MRI reconstruction model.

[0073] Furthermore, when performing three-dimensional reconstruction of the target image to obtain an MRI reconstruction model, the processor 7 can specifically achieve this by executing the following steps:

[0074] The marching cube algorithm is used to reconstruct the target image in three dimensions, resulting in an MRI reconstruction model.

[0075] Among them, the moving cube algorithm is a classic algorithm in surface rendering. It is a voxel-level reconstruction algorithm proposed by W. Lorensen et al. in 1987, and is also known as the "isosurface extraction" algorithm. The main idea of ​​the moving cube algorithm is to approximate the isosurface in a three-dimensional discrete data field through linear interpolation.

[0076] The marching cube algorithm is a mature existing algorithm. This application does not specify the process of using the marching cube algorithm to perform three-dimensional reconstruction of the target image and obtain the MRI reconstruction model.

[0077] To achieve visualization and real-time analysis of the surgical path in skull surgery, this application presents a three-dimensional reconstruction algorithm for the face, skull, and brain based on preoperative MRI images. The algorithm can also accept the start and end coordinates of the surgical path in a 3DSlicer, automatically drilling holes in the reconstructed model according to the input path and displaying the results. It can provide real-time rendering of the relationship between the path and the brain parenchyma, improving the efficiency and accuracy of skull drilling.

[0078] Furthermore, when determining the positional and angular offsets between surgical instrument 1 and the three-dimensional surgical path based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, the fourth pose transformation relationship, the three-dimensional point cloud model, and the MRI reconstruction model, processor 7 can specifically achieve this by executing the following steps:

[0079] Based on the 3D point cloud model and MRI reconstruction model, the pose information of the 3D surgical path under the 3D facial scanning camera 3 was determined.

[0080] Based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, and the fourth pose transformation relationship, the pose information of the surgical instrument 1 under the facial scanning 3D camera 3 is determined;

[0081] Based on the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera 3 and the pose information of the surgical instrument 1 under the facial scanning three-dimensional camera 3, the positional offset and angular offset between the surgical instrument 1 and the three-dimensional surgical path are determined.

[0082] After obtaining the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera 3 and the pose information of the surgical instrument 1 under the facial scanning three-dimensional camera 3, the processor 7 can calculate the positional offset between the surgical instrument 1 and the three-dimensional surgical path based on the position information of the surgical instrument 1 and the position information of the three-dimensional surgical path. Furthermore, based on the angle information of the surgical instrument 1 and the angle information of the three-dimensional surgical path, the processor 7 can calculate the angular offset between the surgical instrument 1 and the three-dimensional surgical path.

[0083] Furthermore, when determining the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera 3 based on the three-dimensional point cloud model and the MRI reconstruction model, the processor 7 can specifically achieve this by executing the following steps:

[0084] Based on the 3D point cloud model and the MRI reconstruction model, the fifth pose transformation relationship between the patient's face under the 3D facial scanning camera and the patient's face in the MRI reconstruction model was determined.

[0085] Based on the fifth pose transformation relationship and the three-dimensional surgical path, the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera 3 is determined.

[0086] After obtaining the fifth pose transformation relationship After the 3D surgical path Tra is obtained, the pose information of the 3D surgical path under the facial scanning 3D camera can be calculated according to the fifth pose transformation relationship.

[0087] Furthermore, when determining the fifth pose transformation relationship between the patient's face as seen in the 3D facial scanning camera 3 and the patient's face in the MRI reconstruction model based on the 3D point cloud model and the MRI reconstruction model, the processor 7 can specifically achieve this by executing the following steps:

[0088] The three-dimensional point cloud model and the MRI reconstruction model were registered to obtain the fifth pose transformation relationship between the patient's face under the three-dimensional facial scanning camera and the patient's face in the MRI reconstruction model.

[0089] Specifically, when registering the 3D point cloud model with the MRI reconstruction model to obtain the fifth pose transformation relationship between the patient's face under the 3D facial scanning camera 3 and the patient's face in the MRI reconstruction model, the processor 7 can achieve this by executing the following steps:

[0090] Using the Lepard algorithm, the 3D point cloud model and the MRI reconstruction model were registered to obtain the fifth pose transformation relationship between the patient's face under the 3D facial scanning camera and the patient's face in the MRI reconstruction model.

[0091] Lepard is a learning-based method used for partial point cloud matching in rigid and deformable scenes.

[0092] Lepard is an existing mature algorithm. This application does not specifically limit the process of registering the three-dimensional point cloud model and the MRI reconstruction model using the Lepard algorithm to obtain the fifth pose transformation relationship between the patient's face under the three-dimensional facial scanning camera and the patient's face in the MRI reconstruction model.

[0093] This application uses a facial scanning 3D camera 3 to perform real-time scanning and 3D reconstruction of the patient's face. The real-time patient facial reconstruction results are aligned and matched with the MRI facial reconstruction results. The set drilling path can be registered to the 3D camera coordinate system, and the surgical path and position of the skull can be displayed in real time on the camera image.

[0094] Furthermore, when determining the pose information of surgical instrument 1 under the facial scanning 3D camera 3 based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, and the fourth pose transformation relationship, the processor 7 can specifically achieve this by executing the following steps:

[0095] The first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, and the fourth pose transformation relationship are input into the first formula for calculation to obtain the pose information of the surgical instrument 1 under the facial scanning 3D camera 3;

[0096] The first formula is shown below:

[0097]

[0098] Wherein, Pose is the pose information of the surgical instrument under the facial scanning 3D camera; for The inverse matrix; This is the first pose transformation relationship; for The inverse matrix; This is the fourth pose transformation relationship; for The inverse matrix; This is the second pose transformation relationship; This is the third pose transformation relationship.

[0099] Furthermore, processor 7 can also perform the following steps:

[0100] Based on the three-dimensional surgical path, the second pose transformation relationship, the third pose transformation relationship, and the fifth pose transformation relationship, the pose information of the three-dimensional surgical path under the positioning three-dimensional camera 5 is determined.

[0101] Specifically, the processor 7 can utilize the pose conversion relationship between the camera positioning code 4 and the facial scanning 3D camera 3. And the pose transformation relationship between the positioning 3D camera 5 and the camera positioning code 4. The calculated position of the 3D surgical path Tra in the image of the 3D positioning camera 5 is as follows:

[0102]

[0103] It can also display the surgical path in real time under the image of the positioning 3D camera 5, providing doctors with data guidance and improving the accuracy of neurosurgery or other surgeries that require skull drilling.

[0104] Corresponding positioning codes are installed on surgical instrument 1 and facial scanning 3D camera 3. The pose transformation relationship between the positioning codes and surgical instrument 1 and facial scanning 3D camera 3 can be calculated. The doctor controls surgical instrument 1 to move closer to the patient's head. The positioning 3D camera 5 can simultaneously capture surgical instrument positioning code 2 and camera positioning code 4. The pose transformation relationship between the two positioning codes can be obtained by processor 7. The relative pose of surgical instrument 1 in the facial scanning camera coordinate system can be obtained. The surgical path is marked on the positioning camera image, which can assist the doctor in drilling the skull.

[0105] Furthermore, the calculated pose of surgical instrument 1 and the surgical path are both in the facial camera coordinate system, allowing us to obtain the rotation angle and offset required to make surgical instrument 1 coincide with the surgical path, providing data guidance for the surgeon. The calculated specific offset angle and offset can also be applied to robot-assisted craniotomy or brain surgery.

[0106] In some embodiments, the processor 7 can obtain a first kinematic relationship A of the projector robotic arm 10. RP Second kinematic relationship A with facial scanning 3D camera robotic arm 9 RFC .

[0107] The first kinematic relationship is the pose relationship between the base of the projector robotic arm 10 and the end of the projector robotic arm 10, i.e., the marker projector 8. The second kinematic relationship is the pose relationship between the base of the 3D camera robotic arm 9 and the end of the 3D camera robotic arm 9, i.e., the face scanning 3D camera 3.

[0108] Furthermore, the processor 7 can obtain the pre-stored seventh pose transformation relationship based on the mechanical mounting positions of the projector robotic arm 10 and the facial scanning 3D camera robotic arm 10. Furthermore, according to the first kinematic relation A RP Second kinematic relationship A RFC Relationship with the seventh pose The sixth pose transformation relationship between the facial scanning 3D camera 3 and the marker projector 8 was determined.

[0109] in, For the original matrix A RFC The inverse matrix.

[0110] Furthermore, based on the sixth pose transformation relationship and the position of the three-dimensional surgical path Tra in the image of the positioning three-dimensional camera 5, the position of the three-dimensional surgical path Tra in the coordinate system of the marker projector 8 is calculated as follows:

[0111]

[0112] Furthermore, the movement of the projector robotic arm 10 can be controlled to adjust the current pose of the marker projector 8 to the pose corresponding to the determined position information, so that the marker projector 8 can project the three-dimensional surgical path onto the patient's face at the position corresponding to the aforementioned position information.

[0113] The brain surgery path guidance system provided in this application embodiment can provide doctors with real-time visual information guidance in addition to their clinical experience, and provides path guidance directly to the lesion location. Furthermore, the data-driven guidance information also makes robot-assisted operation possible.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A brain surgery path guidance system, characterized in that, Includes surgical instruments, a facial scanning 3D camera, a positioning 3D camera, and a processor: The surgical instruments are equipped with surgical instrument positioning codes; The facial scanning 3D camera is equipped with a camera positioning code. The facial scanning 3D camera is used to scan the patient's face and transmit the 3D point cloud model established based on the scan results to the processor. The positioning 3D camera is used to capture images of the surgical instrument positioning code and the camera positioning code, and transmit them to the processor; The processor is used for: Obtain the first pose transformation relationship between the surgical instrument positioning code and the surgical instrument, and the second pose transformation relationship between the camera positioning code and the facial scanning 3D camera; The images of the surgical instrument positioning code and the camera positioning code are identified to obtain the third pose transformation relationship between the positioning 3D camera and the camera positioning code, and the fourth pose transformation relationship between the positioning 3D camera and the surgical instrument positioning code. Three-dimensional reconstruction is performed based on the patient's MRI images to obtain an MRI reconstruction model including a three-dimensional surgical path; wherein, the MRI images include a planar surgical path; Based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, the fourth pose transformation relationship, the three-dimensional point cloud model, and the MRI reconstruction model, determine the positional offset and angular offset between the surgical instrument and the three-dimensional surgical path; The processor is specifically used for: Based on the three-dimensional point cloud model and the MRI reconstruction model, the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera is determined; Based on the first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, and the fourth pose transformation relationship, the pose information of the surgical instrument under the facial scanning 3D camera is determined; Based on the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera and the pose information of the surgical instrument under the facial scanning three-dimensional camera, the positional offset and angular offset between the surgical instrument and the three-dimensional surgical path are determined. The processor is further specifically used for: Based on the three-dimensional point cloud model and the MRI reconstruction model, the fifth pose transformation relationship between the patient's face under the three-dimensional facial scanning camera and the patient's face in the MRI reconstruction model is determined. Based on the fifth pose transformation relationship and the three-dimensional surgical path, the pose information of the three-dimensional surgical path under the facial scanning three-dimensional camera is determined.

2. The brain surgery path guidance system according to claim 1, characterized in that, The processor is specifically used for: The planar surgical path in the patient's MRI image is transformed into a three-dimensional coordinate system to obtain the three-dimensional surgical path; Based on the three-dimensional surgical path, the MRI image is pixel-reduced to obtain the target image; The target image is reconstructed in three dimensions to obtain an MRI reconstruction model.

3. The brain surgery path guidance system according to claim 2, characterized in that, The processor is specifically used for: The target image is reconstructed in three dimensions using the marching cube algorithm to obtain an MRI reconstruction model.

4. The brain surgery path guidance system according to claim 1, characterized in that, The processor is further specifically used for: The three-dimensional point cloud model and the MRI reconstruction model are registered to obtain the fifth pose transformation relationship between the patient's face in the three-dimensional facial scanning camera and the patient's face in the MRI reconstruction model.

5. The brain surgery path guidance system according to claim 4, characterized in that, The processor is further specifically used for: The Lepard algorithm is used to register the 3D point cloud model with the MRI reconstruction model to obtain the fifth pose transformation relationship between the patient's face in the 3D facial scanning camera and the patient's face in the MRI reconstruction model.

6. The brain surgery path guidance system according to claim 1, characterized in that, The processor is further specifically used for: The first pose transformation relationship, the second pose transformation relationship, the third pose transformation relationship, and the fourth pose transformation relationship are input into the first formula for calculation to obtain the pose information of the surgical instrument under the facial scanning 3D camera.

7. The brain surgery path guidance system according to claim 6, characterized in that, The first formula is shown below: ； in, The pose information of the surgical instruments under the facial scanning 3D camera; for The inverse matrix; This is the first pose transformation relationship; for The inverse matrix; This is the fourth pose transformation relationship; for The inverse matrix; This is the second pose transformation relationship; This is the third pose transformation relationship.

8. The brain surgery path guidance system according to claim 1, characterized in that, The processor is further specifically used for: Based on the three-dimensional surgical path, the second pose transformation relationship, the third pose transformation relationship, and the fifth pose transformation relationship, the pose information of the three-dimensional surgical path under the positioning three-dimensional camera is determined.

9. The brain surgery path guidance system according to claim 1, characterized in that, It also includes a marker projector, a projector robotic arm, and a facial scanning 3D camera robotic arm; The marker projector is used to project the three-dimensional surgical path. The projector robotic arm is connected to the marker projector and is used to control the pose of the marker projector to adjust the projection position of the marker projector. The robotic arm for facial scanning 3D camera is connected to the facial scanning 3D camera; The processor is further configured to acquire a first kinematic relationship of the projector robotic arm and a second kinematic relationship of the facial scanning 3D camera robotic arm; determine a sixth pose transformation relationship between the marker projector and the facial scanning 3D camera based on the first kinematic relationship and the second kinematic relationship; determine the position information of the 3D surgical path under the marker projector based on the sixth pose transformation relationship; and control the projector robotic arm to adjust the current pose of the marker projector to the pose corresponding to the position information.

Images