Processing method for surgical navigation system, software system, and surgical navigation system

Abstract

A processing method for a surgical navigation system, a software system, and a surgical navigation system. The processing method comprises: acquiring coordinate positions of a plurality of sets of positioning balls on the basis of an optical positioning instrument, and calculating centroid coordinates of each set of positioning balls and a unit normal vector of a plane determined by the plurality of sets of positioning balls (S102); determiningthat a three-dimensional coordinate system with a centroid coordinate position of a set of positioning balls on the skull side of a patient as an origin is a world coordinate system, acquiring spatial coordinates of Raman points in the world coordinate system, and calculating positions of pixel points on an image that correspond to the Raman points (S104); and acquiring a preoperative medical image, translating the world coordinate system to enable the world coordinate system and a coordinate system of the preoperative medical image to overlap at a coordinate origin, and displaying the positions of the pixel points on the image that correspond to the Raman points in the preoperative medical image (S106). The method can provide data support for subsequently determining positions of Raman spectrum measurement points in a preoperative three-dimensional reconstruction image, and determining two-dimensional projection positions of the Raman spectrum measurement points under the microscope field of view.

Description

Processing methods, software systems, and surgical navigation systems

[0001] Cross-references to related applications

[0002] This disclosure claims priority to Chinese Patent Application No. 2024104880381, filed on April 23, 2024, entitled "Processing Method, Software System and Surgical Navigation System for Surgical Navigation System", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of medical technology, and in particular to a processing method, software system, and surgical navigation system for a surgical navigation system. Background Technology

[0004] The high recurrence rate after glioma surgery leads to high patient mortality and poor prognosis, one of the main reasons being the microinvasive growth of gliomas. Glioma growth exhibits significant heterogeneity, with different molecular subtypes displaying markedly different invasive and microinvasive growth patterns. However, due to the lack of imaging techniques for precisely locating the microinvasive extent of gliomas, the relationship between glioma molecular subtypes and their microinvasive growth patterns remains unclear. Therefore, accurately determining the microinvasive extent of gliomas is of significant clinical importance for precise resection and reducing postoperative recurrence.

[0005] Fluorescence-guided imaging (FRI) technology, through preoperative injection of fluorescent agents, can rapidly and in real-time visualize the location and extent of gliomas during surgery. However, due to the lack of targeted fluorescent agents for gliomas and the high heterogeneity of gliomas themselves, FRI's accuracy and specificity in assessing glioma microinvasiveness are limited. Raman spectroscopy, by identifying molecular fingerprints, can highly specifically determine whether brain tissue is cancerous and even differentiate between different molecular subtypes, compensating for the lack of specificity and accuracy in microinvasiveness identification of FRI. However, gliomas are complex in composition, and their high heterogeneity results in extremely complex Raman spectra, making it difficult to distinguish tumors from normal tissue and determine their molecular subtypes by directly observing peak differences. Therefore, utilizing artificial intelligence technology to rapidly and efficiently analyze the Raman spectra of glioma microinvasive foci and extract the unique "Raman spectra" of different molecular subtypes of gliomas is crucial for accurately determining the extent of microinvasiveness and their molecular subtype during surgery.

[0006] Glioma surgery typically requires an surgical microscope. The surgeon continuously adjusts the microscope's angle and distance around the surgical area. The location of Raman spectroscopy measurements needs to be memorized in a three-dimensional coordinate system and reflected in the current surgical view in real time. Establishing and transforming this coordinate system is a crucial technical challenge. Integrating fluorescence image navigation, Raman spectroscopy, and artificial intelligence technologies into a single surgical microscope to create a unified navigation and diagnostic system is a pressing technical problem that needs to be solved. Summary of the Invention

[0007] In view of this, the purpose of this disclosure is to provide a processing method, software system and surgical navigation system for a surgical navigation system. The optical positioning system monitors the spatial position of a positioning ball fixed on a fluorescence surgical microscope, the side of the patient's head and the Raman spectroscopy probe in real time. The software system of this disclosure then calculates the relative position and spatial angle between the three, providing data support for determining the position of the Raman spectroscopy measurement point in the preoperative three-dimensional reconstructed image and the two-dimensional projection position of the Raman spectroscopy measurement point under the microscope view.

[0008] This disclosure provides a processing method for a surgical navigation system, applied to the software system of the surgical navigation system. The method includes: acquiring the coordinate positions of multiple sets of positioning spheres based on an optical locator, calculating the centroid coordinates of each set of positioning spheres and the unit normal vector of the plane determined by the multiple sets of positioning spheres; wherein the positioning spheres are set on a fluorescence surgical microscope, the patient's head side, and a Raman spectroscopy probe; determining a three-dimensional coordinate system with the centroid coordinate position of a set of positioning spheres on the patient's head side as the origin as the world coordinate system, acquiring the spatial coordinates of each Raman point in the world coordinate system, calculating the position of the corresponding pixel point of each Raman point on the image based on the spatial coordinates of each Raman point in the world coordinate system; acquiring preoperative medical images, translating the world coordinate system so that the world coordinate system overlaps with the coordinate system of the preoperative medical images at the coordinate origin, and displaying the position of the corresponding pixel point of each Raman point on the image in the preoperative medical images.

[0009] In an optional embodiment of this disclosure, the steps of obtaining the coordinate positions of multiple sets of positioning spheres based on the optical locator, calculating the centroid coordinates of each set of positioning spheres, and the unit normal vector of the plane determined by the multiple sets of positioning spheres include: measuring the distance and angle parameters between the multiple sets of positioning spheres; adjusting the default reference frame of the optical locator to have the vertical line of the head pointing upwards as the positive z-axis, the sagittal axis pointing forward as the positive x-axis, and the frontal axis pointing left as the positive y-axis, and setting the centroid position of a set of positioning spheres on the side of the patient's head as the origin; determining the coordinates of a set of positioning spheres based on the optical locator, and determining the centroid coordinates and unit normal vector of the set of positioning spheres based on the coordinates of the set of positioning spheres; and calculating the vector connecting the centroid position of the set of positioning spheres to the center point, the coordinates of the center point, and the direction of the central axis based on the centroid coordinates, the unit normal vector, the distance parameters, and the angle parameters.

[0010] In optional embodiments of this disclosure, the aforementioned distance parameters and angle parameters include at least one of the following: the distance between the center of gravity of the positioning ball and the corresponding microscope objective exit center, the center point of the patient's skull top, and the end of the Raman probe; the spatial angle between the line connecting these distances and the normal direction of the corresponding positioning ball; the angle between the projection line of the line connecting these distances on the plane where the positioning ball is located and the line connecting the first ball and the second ball in each group of positioning balls; and the angle between the line connecting the first ball and the second ball in each group of positioning balls and the line connecting the first ball and the third ball.

[0011] In an optional embodiment of this disclosure, each of the above-mentioned positioning balls is installed such that its normal direction is parallel to the corresponding microscope field of view central axis, the vertical axis of the patient's skull, and the central axis of the Raman probe.

[0012] In an optional embodiment of this disclosure, the step of calculating the position of each Raman point in the corresponding pixel on the image based on the spatial coordinates of each Raman point in the world coordinate system includes: transforming the spatial coordinates of each Raman point in the world coordinate system to a camera coordinate system with the equivalent optical center of the microscope lens as the origin; calculating the two-dimensional coordinates of the spatial coordinates of each Raman point in the camera coordinate system on the camera imaging plane; wherein the camera imaging plane is set behind the equivalent back focus of the equivalent optical center; and translating the coordinate system containing the two-dimensional coordinates to obtain the coordinates of each Raman point in the pixel coordinate system as the position of the corresponding pixel on the image.

[0013] In an optional embodiment of this disclosure, the step of transforming the spatial coordinates of each Raman point in the world coordinate system to the camera coordinate system with the equivalent optical center of the microscope lens as the origin includes:

[0014] The spatial coordinates of each Raman point in the world coordinate system are transformed to the camera coordinate system with the equivalent optical center of the microscope lens as the origin using the following formula:

[0015] Where R represents an orthogonal matrix, (x i ,y i ,z i (x′) represents the spatial coordinates in the world coordinate system. i ,y′ i ,z′ i ) represents the spatial coordinates in the camera coordinate system, t represents the three-dimensional translation vector, the orthogonal matrix is ​​the product of the rotation matrices corresponding to each three-dimensional direction, and the three-dimensional translation vector is the difference between the equivalent optical center of the microscope lens and the centroid position of a set of positioning spheres on the side of the patient's head.

[0016] In an optional embodiment of this disclosure, the step of calculating the spatial coordinates of each Raman point in the camera coordinate system and the two-dimensional coordinates on the camera imaging plane includes:

[0017] The spatial coordinates of each Raman point in the camera coordinate system and its two-dimensional coordinates on the camera imaging plane are calculated using the following formula:

[0018] Among them, (x n ,y n (x′) represents two-dimensional coordinates. i ,y′ i ,z′ i ) represents the spatial coordinates in the camera coordinate system, and f represents the equivalent back focus.

[0019] In an optional embodiment of this disclosure, the step of translating the coordinate system containing the two-dimensional coordinates to obtain the coordinates of each Raman point in the pixel coordinate system as the position of the corresponding pixel point on the image includes:

[0020] The coordinate system containing the two-dimensional coordinates is translated according to the following formula:

[0021] Among them, (w i ,h i (x) represents the coordinates of the Raman point in the pixel coordinate system. n ,y n () represents the two-dimensional coordinates, (-w0,-h0) represents the translation vector, and dx and dy represent the actual size of each pixel in the w and h directions, respectively.

[0022] In an optional embodiment of this disclosure, after the step of translating the world coordinate system to overlap the world coordinate system with the coordinate system of the preoperative medical image at the origin, the method further includes one of the following: identifying the location of the lesion based on the preoperative medical image and displaying the location of the lesion in the preoperative medical image; calculating the field of view of the microscope and displaying the field cone of the microscope in the preoperative medical image; displaying the position of the Raman probe in the preoperative medical image and projecting the Raman probe according to a given viewing angle.

[0023] In optional embodiments of this disclosure, the method further includes: inputting Raman spectral data acquired by the Raman probe into a pre-established deep learning model based on the Raman spectroscopy diagnostic system, and outputting lesion classification results; displaying the lesion classification results at the positions of the corresponding pixels of each Raman point in the preoperative medical image.

[0024] In an optional embodiment of this disclosure, the step of displaying the lesion classification result at the position of the corresponding pixel of each Raman point in the preoperative medical image includes: determining the currently displayed Raman point in the preoperative medical image when the camera lens moves; and displaying the lesion classification result corresponding to the currently displayed Raman point at the position of the corresponding pixel of the currently displayed Raman point in the preoperative medical image.

[0025] This disclosure also provides a software system configured to execute the above-described surgical navigation system processing method.

[0026] This disclosure also provides a surgical navigation system, which includes: a fluorescence surgical microscope, a Raman spectroscopy measurement system, a Raman spectroscopy diagnostic system, an optical positioning system, and the aforementioned software system.

[0027] This disclosure also provides an electronic device, including a memory and a processor, the memory being configured to store computer instructions, and the processor being configured to execute the computer instructions to implement the processing method of the surgical navigation system according to any one of claims 1-11.

[0028] The embodiments disclosed herein bring the following beneficial effects:

[0029] This disclosure provides a processing method, software system, and surgical navigation system for a surgical navigation system. The optical positioning system monitors the spatial position of a positioning ball fixed on a fluorescence surgical microscope, the side of the patient's head, and a Raman spectroscopy probe in real time. The software system of this disclosure then calculates the relative position and spatial angle between the three, providing data support for determining the position of the Raman spectroscopy measurement point in the preoperative three-dimensional reconstructed image and the two-dimensional projection position of the Raman spectroscopy measurement point from the microscope's perspective.

[0030] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0031] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 is a flowchart of a processing method for a surgical navigation system provided in an embodiment of this disclosure;

[0034] Figure 2 is a schematic diagram of a three-sphere coordinate system and its centroid coordinate system provided in an embodiment of this disclosure;

[0035] Figure 3 is a schematic diagram of the positional relationship between three balls and a target positioning point according to an embodiment of this disclosure;

[0036] Figure 4 is a schematic diagram of calculating the coordinate vector of a target positioning point according to an embodiment of this disclosure;

[0037] Figure 5 is a schematic diagram of coordinate calculation in a camera imaging model provided in an embodiment of this disclosure;

[0038] Figure 6 is a schematic diagram of a navigation system and a fluorescence imaging display interface provided in an embodiment of this disclosure;

[0039] Figure 7 is a flowchart of another processing method of a surgical navigation system provided in an embodiment of this disclosure;

[0040] Figure 8 is a schematic diagram of a surgical navigation system provided in an embodiment of this disclosure;

[0041] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0042] Icons: 1-Fluorescence surgical microscope; 2-Optical positioning system; 3-Raman spectroscopy probe; 4-First positioning ball group; 5-Second positioning ball group; 6-Third positioning ball group; 201-Near-infrared positioning laser emission position; 202-Visible light camera; 203 Near-infrared laser positioning camera combination; 100-Memory; 101-Processor; 102-Bus; 103-Communication interface. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0044] Currently, glioma surgery is typically performed under an surgical microscope. The surgeon continuously adjusts the angle and distance of the microscope around the surgical area. The location measured by Raman spectroscopy needs to be memorized in a three-dimensional coordinate system and reflected in the current surgical view in real time. Establishing and transforming this coordinate system is a significant technical challenge. Integrating fluorescence image navigation technology, Raman spectroscopy, and artificial intelligence into a single surgical microscope to ultimately achieve a unified navigation and diagnostic system is a pressing technical problem that needs to be solved.

[0045] Based on this, the present disclosure provides a processing method, software system, and surgical navigation system for a surgical navigation system. The optical positioning system monitors the spatial position of a positioning ball fixed on a fluorescence surgical microscope, the side of the patient's head, and a Raman spectroscopy probe in real time. The software system of the present disclosure then calculates the relative position and spatial angle between the three, providing data support for determining the position of the Raman spectroscopy measurement point in the preoperative three-dimensional reconstructed image and the two-dimensional projection position of the Raman spectroscopy measurement point under the microscope view.

[0046] To facilitate understanding of this embodiment, a processing method for a surgical navigation system disclosed in this disclosure will first be described in detail.

[0047] This disclosure provides a processing method for a surgical navigation system, which is applied to the software system of the surgical navigation system. The software system in this embodiment includes a spatial three-dimensional coordinate algorithm, a two-dimensional projection algorithm for Raman measurement points under the microscope view, a real-time superposition algorithm for preoperative three-dimensional reconstructed medical images with microscope view and Raman probe position, an image fusion algorithm for fluorescence images and Raman measurement results, and an overall software display framework.

[0048] Based on the above description, referring to the flowchart of a processing method for a surgical navigation system shown in Figure 1, the processing method of the surgical navigation system includes the following steps:

[0049] Step S102: Based on the optical positioning instrument, obtain the coordinate positions of multiple sets of positioning spheres, calculate the centroid coordinates of each set of positioning spheres and the unit normal vector of the plane determined by the multiple sets of positioning spheres; wherein, the positioning spheres are set on the fluorescence surgical microscope, the side of the patient's head and the Raman spectroscopy probe.

[0050] Referring to Figure 2, a schematic diagram of a three-sphere coordinate system and its centroid coordinates can be provided. This embodiment offers a spatial three-dimensional coordinate algorithm (the coordinate unit in this algorithm is uniformly millimeters). Based on the coordinate positions of the three sets of positioning spheres given by the optical positioning instrument, the centroid coordinates of each set of positioning spheres and the unit normal vector of the plane defined by the three spheres are calculated. For example, for the coordinates of the three spheres... Its centroid coordinates are Its unit normal vector is The centroid coordinates of a set of positioning spheres on the microscope are: The unit normal vector is The centroid coordinates of a set of positioning balls on the side of the patient's head are: The unit normal vector is The coordinates of the center of gravity of a set of positioning spheres on the Raman probe are: The unit normal vector is

[0051] In some embodiments, distance and angle parameters between multiple sets of positioning spheres can be measured; the default reference frame of the optical positioning device is adjusted to have the positive z-axis pointing upwards along the vertical line of the head, the positive x-axis pointing forward along the sagittal axis, and the positive y-axis pointing left along the frontal axis; the origin is set at the centroid position of a set of positioning spheres on the side of the patient's head; the coordinates of a set of positioning spheres are determined based on the optical positioning device; the centroid coordinates and unit normal vector of the set of positioning spheres are determined based on the coordinates of the set of positioning spheres; the vector connecting the centroid position of the set of positioning spheres to the center point, the coordinates of the center point, and the direction of the central axis are calculated based on the centroid coordinates, the unit normal vector, the distance parameters, and the angle parameters.

[0052] The distance and angle parameters include at least one of the following: the distance between the center of gravity of the positioning ball and the corresponding center of the microscope objective lens exit, the center of the patient's skull, and the end of the Raman probe; the spatial angle between the line connecting these distances and the normal direction of the corresponding positioning ball; the angle between the projection line of this distance line on the plane where the positioning ball is located and the line connecting the first and second balls in each group of positioning balls; and the angle between the line connecting the first and second balls in each group of positioning balls and the line connecting the first and third balls.

[0053] Referring to Figure 3, a schematic diagram of the positional relationship between three spheres and the target positioning point can be provided. The distances d1, d2, and d3 between the centroid of each positioning sphere and the corresponding center point of the microscope objective lens exit, the center point of the patient's skull, and the tip of the Raman probe are measured beforehand. The spatial angle between the line connecting these distances (starting from the centroid of the positioning sphere) and the normal direction of the corresponding positioning sphere is also considered. The angles θ1, θ2, θ3 between the projection of this distance line onto the plane of the positioning ball and the line connecting the ab and ac balls (these projection lines are manually set between the ab and ac ball lines during ball installation) and the ab ball line defined in each group of balls. Connect with AC ball The included angles are α1, α2, and α3 (this included angle cannot be equal to 0 degrees or 180 degrees). Each set of positioning balls is installed so that its normal direction is parallel to the corresponding microscope field-of-view central axis, the patient's head vertical axis, and the Raman probe central axis.

[0054] As shown in Figure 3, the default reference frame of the optical positioning device is adjusted so that the positive z-axis is upward along the vertical line of the head, the positive x-axis is forward along the sagittal axis, and the positive y-axis is leftward along the frontal axis. The center of gravity of a set of positioning balls on the side of the patient's head is set as the origin (0,0,0).

[0055] The coordinates of a set of positioning spheres on the microscope are obtained using an optical positioning instrument. The centroid coordinates of the positioning sphere on the microscope can then be calculated. The unit normal vector of the positioning sphere on the microscope To ensure the unit normal vector of the positioning sphere on the microscope Let the positive x-axis be the unit vector, aligned with the direction of observation under a microscope. Algorithm determination required The sign of , if negative, then The direction is correct; otherwise, within the algorithm... Reassignment:

[0056] Referring to Figure 4, which shows a schematic diagram of calculating the coordinate vector of the target positioning point, the next step is to calculate the vector of the line connecting the centroid of the positioning sphere and the center point of the microscope objective lens exit. Let the unit vector of the projection line of the distance line onto the plane where the positioning ball is located be . Because during the installation of the positioning ball, the projection line was manually set between the line connecting spheres ab and spheres ac, i.e. lie in and Inside the included angle, It can be represented as and Satisfying the relation therefore,

[0057] but Therefore, we can solve for:

[0058] The other solution to this quadratic equation is negative, which does not satisfy the presupposition that "the projection line lies between the line connecting spheres ab and spheres ac". Therefore, λ1 has only one solution, as described above. Based on this solution, we can calculate... The vector value.

[0059] So Therefore, the coordinates of the center point of the microscope objective lens exit can be obtained as follows: The microscope viewing angle is as previously preset.

[0060] By analogy, the parameters of the patient's head side can be obtained:

[0061] Among these, to ensure the unit normal vector of the positioning ball on the side of the patient's head. Let the z-axis be a positive unit vector aligned with the vertical axis of the patient's skull. Algorithm determination required The sign, if positive, then The direction is correct; otherwise, within the algorithm... Reassignment:

[0062] The coordinates of the center point of the patient's skull are: The patient's vertical axis direction is as previously preset.

[0063] The parameters of the Raman probe can be obtained:

[0064] Among these, to ensure the unit normal vector of the positioning sphere on the Raman probe Aligned with the direction of the central axis of the Raman probe (pointing towards the end), only need to ensure that the required direction is met. If the angle between the line connecting the center of gravity of the Raman probe positioning sphere and the center point of the patient's skull is acute, then an algorithm is needed to determine... The sign, if positive, then The direction is correct; otherwise, within the algorithm... Reassignment:

[0065] The coordinates of the Raman probe tip are: The direction of the central axis of the Raman probe (pointing towards the end) is as previously preset.

[0066] At this point, the positions and orientations of the microscope, patient's head, and Raman probe can be calculated in real time using the real-time positioning spherical coordinates provided by the positioning system, which can then be used for subsequent calculations of 3D image fusion and projection.

[0067] Step S104: Determine the three-dimensional coordinate system with the origin of a set of positioning spheres on the side of the patient's head as the world coordinate system, obtain the spatial coordinates of each Raman point in the world coordinate system, and calculate the position of each Raman point on the corresponding pixel in the image based on the spatial coordinates of each Raman point in the world coordinate system.

[0068] This embodiment also provides a two-dimensional projection algorithm for Raman measurement points from a microscope perspective. The above method establishes a three-dimensional coordinate system (also known as the world coordinate system) with the centroid of a set of positioning spheres on the side of the patient's head as the origin, and the spatial coordinates (x, y, z) of all Raman points can be obtained within this coordinate system. i ,y i ,z i ), i∈[1…n], where n is the number of Raman points; the spatial coordinates (x, y) of the microscope lens center can also be obtained in real time. c ,y c ,z c The purpose of this part of the algorithm is: when the microscope lens position is (x... c ,y c ,z c When calculating the Raman point (x), i ,y i ,z i The position of the corresponding pixel on the image (w) i ,h i ), w i It is the width value in pixel coordinates, h i This is the height value, measured in pixels, used to mark all Raman points in the microscope fluorescence image. To obtain (x... i ,y i ,z i ) corresponding to (w i ,h i To achieve this, a mapping relationship between the world coordinate system and the pixel coordinate system needs to be established.

[0069] In some embodiments, the spatial coordinates of each Raman point in the world coordinate system can be transformed to the camera coordinate system with the equivalent optical center of the microscope lens as the origin; the two-dimensional coordinates of each Raman point in the camera coordinate system on the camera imaging plane can be calculated; wherein, the camera imaging plane is set behind the equivalent back focus of the equivalent optical center; the coordinate system containing the two-dimensional coordinates is translated to obtain the coordinates of each Raman point in the pixel coordinate system as the position of the corresponding pixel point on the image.

[0070] Step 1: The world coordinate system's (x) needs to be changed. i ,y i ,z i (Convert to the equivalent optical center of the microscope lens) nCamera coordinate system with origin (x′) i ,y′ i ,z′ i ):

[0071] Where R is an orthogonal matrix; t represents the three-dimensional translation vector, which is represented by the equivalent optical center of the microscope lens. m The coordinates are obtained by subtracting the centroid positions of a set of positioning spheres on the side of the patient's head. The rotation matrices in the three directions can be obtained based on the rotation angles, and the orthogonal matrix is ​​their product: R = R x ×R y ×R z When it is necessary to rotate θ around x, y, and z respectively x θ y θ z hour:

[0072] Where θ x θ y θ z This involves transforming a real-world coordinate system with the center of gravity of a set of positioning spheres on the side of the patient's head as the origin to a coordinate system with the equivalent optical center of the microscope lens as the origin. m The rotation angle of a coordinate system with the origin (the z' axis is outward from the main optical axis, and the x' and y' axes are parallel to the length and width of the CMOS, respectively).

[0073] Step 2: Referring to Figure 5, a schematic diagram of coordinate calculation in a camera imaging model, at the equivalent optical center o m There is an imaging plane at a distance f. Calculate the point (x′) in 3D space. i ,y′ i ,z′ i Two-dimensional coordinates (x, y) on the camera's imaging plane CMOS n ,y n ), that is, simulation (x′ i ,y′ i ,z′ i (Through the camera's optical center) n The process of projecting onto the imaging plane, where f is the equivalent back focal length of the camera.

[0074] Since the direction of light rays passing through the equivalent optical center remains unchanged, we can conclude from the principle of similar triangles that:

[0075] The image coordinates on the CMOS here have been mirrored to account for the inverted nature of imaging, and therefore have the same coordinate symbols as those in the camera coordinate system.

[0076] Step 3: Digital cameras use a CMOS sensor as the imaging plane, imaging as discrete pixels to obtain a digital image. Therefore, the imaging plane coordinate system needs to be discretized into a pixel array. Furthermore, since digital images typically use the top-left corner as the origin, it is necessary to set (x... m ,y n If the origin of the coordinate system containing (x) is shifted (at the center of the image) by the vector (-w0, -h0) to the top left corner of the image (i.e., coordinate system translation), then (x) will be shifted to the center of the image. n ,y n Discretized into pixels on a digital image (w) i ,h i ):

[0077] Where dx and dy represent the actual size of each pixel in the w and h directions (unit: mm / pixel), respectively, which is determined by the size of each photosensitive element in the camera sensor.

[0078] These three steps will determine the midpoint (x) of the world coordinate system. i ,y i ,z i (Unit: millimeters) Convert to points in pixel coordinates (w) i ,h i (Unit: pixels). If w i or h i When the value exceeds the pixel range that the CMOS can acquire, it indicates that the currently calculated Raman point (x) is... i ,y i ,z i If a point is not in the image, it does not need to be marked on the image. At this point, the three-dimensional coordinates of the Raman measurement point can be projected onto the microscope image in real time.

[0079] Step S106: Acquire preoperative medical images, translate the world coordinate system so that the world coordinate system overlaps with the coordinate system of the preoperative medical images at the origin, and display the position of each Raman point on the corresponding pixel in the image in the preoperative medical images.

[0080] In some embodiments, it may also include one of the following: identifying the location of the lesion based on preoperative medical images and displaying the location of the lesion in the preoperative medical images; calculating the field of view of the microscope and displaying the field cone of the microscope in the preoperative medical images; displaying the position of the Raman probe in the preoperative medical images and projecting the Raman probe in a given viewing angle direction.

[0081] Referring to Figure 6, which shows a schematic diagram of a navigation system and a fluorescence imaging display interface, this embodiment also provides a real-time overlay algorithm for preoperative three-dimensional reconstructed medical images with microscope viewpoints and Raman probe positions. Furthermore, Figure 6 also shows the overall display framework of the software.

[0082] Using preoperative medical imaging data such as CT (Computed Tomography), PET-CT (Positron Emission Tomography-Computed Tomography), or MRI (Magnetic Resonance Imaging), preoperative three-dimensional reconstructed medical images can be obtained using existing algorithms, translating the real-world three-dimensional coordinate system (i.e., the world coordinate system). This allows the origin to be fixed at the patient's head. Since the unit is standardized to millimeters, the translated real-world coordinate system and the 3D reconstructed medical image coordinate system can be superimposed at the origin. Then, it is projected onto the screen at any given viewpoint. Preoperative medical images, using existing mature image segmentation and recognition algorithms, can roughly indicate the location and extent of the lesion, as shown in the white area in the left image of Figure 6.

[0083] The coordinates of the microscope lens center and its viewing angle vector can be obtained using a three-dimensional coordinate algorithm. The relationship between the microscope's field of view ω and the object focus u, ω = F(u), can be measured beforehand. Therefore, given any object focus u, the microscope's field of view ω can be calculated. Combining this information, the microscope's field of view cone can be calculated in real time and projected onto the screen according to any given viewing angle, as shown in the light cone projection in the left image of Figure 6.

[0084] By using the coordinates of the Raman probe measurement point and the direction the Raman probe is pointing using a spatial three-dimensional coordinate algorithm, a line segment can be used to simulate the position of the Raman probe in the three-dimensional image. Then, it can be projected onto the screen according to any given viewing angle, as shown by the line segment projection in the left figure of Figure 6.

[0085] After each Raman spectrum measurement, the coordinates of the probe measurement point are recorded and marked on the three-dimensional reconstructed medical image, as shown by the marked points in the left image of Figure 6.

[0086] This disclosure provides a processing method for a surgical navigation system. The optical positioning system monitors the spatial position of a positioning ball fixed on a fluorescence surgical microscope, the side of the patient's head, and a Raman spectroscopy probe in real time. The software system of this disclosure then calculates the relative position and spatial angle between the three, providing data support for determining the position of the Raman spectroscopy measurement point in the preoperative three-dimensional reconstructed image and the two-dimensional projection position of the Raman spectroscopy measurement point from the microscope's perspective.

[0087] This embodiment provides another processing method for a surgical navigation system, which is implemented based on the above embodiment. Referring to the flowchart of another processing method for a surgical navigation system shown in Figure 7, the processing method of the surgical navigation system includes the following steps:

[0088] Step S702: Based on the optical positioning instrument, obtain the coordinate positions of multiple sets of positioning spheres, calculate the centroid coordinates of each set of positioning spheres and the unit normal vector of the plane determined by the multiple sets of positioning spheres; wherein, the positioning spheres are set on the fluorescence surgical microscope, the side of the patient's head and the Raman spectroscopy probe.

[0089] Step S704: Determine the three-dimensional coordinate system with the origin of a set of positioning spheres on the side of the patient's head as the world coordinate system, obtain the spatial coordinates of each Raman point in the world coordinate system, and calculate the position of each Raman point on the corresponding pixel in the image based on the spatial coordinates of each Raman point in the world coordinate system.

[0090] Step S706: Acquire preoperative medical images, translate the world coordinate system so that the world coordinate system overlaps with the coordinate system of the preoperative medical images at the origin, and display the position of each Raman point on the corresponding pixel in the image in the preoperative medical images.

[0091] Step S708: Based on the Raman spectroscopy diagnostic system, the Raman spectral data acquired by the Raman probe is input into a pre-established deep learning model, and the lesion classification results are output; the lesion classification results are displayed at the position of the corresponding pixel point of each Raman point in the preoperative medical image.

[0092] This embodiment also provides an image fusion algorithm for fluorescence images and Raman measurement results. The Raman spectroscopy diagnostic system inputs the Raman spectral data acquired by the Raman probe into a pre-established deep learning model. The model infers the lesion classification results, including cancer-negative / positive and their molecular subtypes. Then, according to different classification results, the Raman measurement point locations obtained by the two-dimensional projection algorithm of the Raman measurement points under the microscope are marked with different colors.

[0093] In some embodiments, when the camera lens moves, the Raman point currently displayed in the preoperative medical image can be determined; the lesion classification result corresponding to the currently displayed Raman point can be displayed at the position of the pixel corresponding to the currently displayed Raman point in the preoperative medical image.

[0094] A key feature of this fusion algorithm is its ability to update the Raman measurement points in the right image of Figure 6 in real time. When the image content changes due to camera lens movement, the stored Raman points are first iterated through one by one in real time. Then, using the aforementioned "two-dimensional projection algorithm for Raman measurement points under microscope view," the spatial coordinates of each Raman point are mapped to pixel coordinates within the current image content. If the pixel coordinates are within the current image size, the Raman point is located in the right image of Figure 6 and needs to be marked. Otherwise, the point is not located in the right image of Figure 6 and no marking is required. Therefore, the position of the Raman measurement points is updated due to changes in the relative position of the microscope and the patient's brain, or due to the addition of new measurement points. Clicking on any measurement point in the 3D or 2D image will display its detailed Raman spectrum in the left image of Figure 6.

[0095] The method provided in this disclosure specifically includes the following:

[0096] (1) Three-dimensional coordinate algorithm, two-dimensional projection algorithm of Raman measurement points under microscope view, real-time superposition algorithm of preoperative three-dimensional reconstruction medical images with microscope view and Raman probe position, and image fusion algorithm of fluorescence images and Raman measurement results.

[0097] (2) The relationship between the microscope imaging field of view and the Raman measurement point in two dimensions is analyzed by three-dimensional coordinate calculation and projection.

[0098] (3) After normalization and standardization, the Raman spectra are entered into the model in the form of two-dimensional images for training, in order to overcome the influence of hardware differences between different Raman spectroscopy devices and improve the universality of the model.

[0099] (4) By connecting the fluorescence surgical microscope, Raman diagnostic system and preoperative three-dimensional reconstruction system in series through the optical positioning system, the problem of tissue penetration depth can be solved, and the problems of Raman measurement point position memory tracking and Raman fluorescence image precise fusion can also be solved.

[0100] This embodiment utilizes an optical positioning device to detect and determine the relative positions of the surgical microscope, Raman spectroscopy probe, and positioning ball fixed to the side of the patient's head, thereby establishing a three-dimensional coordinate system. Combined with preoperative CT / MRI / PET-CT three-dimensional reconstructions of the brain, the coordinates of the fluorescence imaging area, Raman detection points, and the brain can be determined intraoperatively, resolving the issue of tissue penetration depth. This provides a coordinate calculation basis for the fusion of fluorescence and Raman images and the comparison of Raman measurement points with preoperative image diagnosis of the tumor area. Real-time large-area scanning using fluorescence imaging quickly determines the approximate extent of the tumor. Then, guided by three-dimensional visualization and artificial intelligence Raman measurement, precise scanning diagnosis is performed on areas where fluorescence cannot determine the extent. Simultaneously, Raman measurement points can be registered in the preoperative image three-dimensional reconstruction space, allowing for real-time monitoring of the surgical progress. The surgeon can refer to the relationship between the preoperative three-dimensional reconstruction image and the Raman measurement points, and the relationship between the real-time two-dimensional fluorescence image and the Raman measurement points, to make a clear judgment on whether the tumor has been completely removed. Meanwhile, the artificial intelligence algorithm is based on deep learning. With a large number of glioma and normal brain tissue samples trained in the early stage, it can not only determine whether the measured point is a tumor, but also make a rapid pre-diagnosis of the molecular subtype of the tumor, allowing doctors to make timely adjustments to the aggressiveness of the surgery based on the molecular subtype of the tumor.

[0101] This disclosure also provides a software system that implements the method based on the above embodiments, wherein the software system is configured as the processing method of the surgical navigation system provided in the foregoing embodiments.

[0102] The software system in this embodiment may include the spatial three-dimensional coordinate algorithm, the two-dimensional projection algorithm of Raman measurement points under the microscope view provided in the foregoing embodiment, the real-time superposition algorithm of preoperative three-dimensional reconstructed medical images with microscope view and Raman probe position, the image fusion algorithm of fluorescence images and Raman measurement results, and the overall software display framework.

[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the software system described above can be referred to the corresponding process in the embodiments of the aforementioned surgical navigation system processing method, and will not be repeated here.

[0104] This disclosure also provides a surgical navigation system, which is implemented based on the above embodiments. The surgical navigation system includes: a fluorescence surgical microscope, a Raman spectroscopy measurement system, a Raman spectroscopy diagnostic system, an optical positioning system, and the software system provided in the foregoing embodiments.

[0105] Referring to Figure 8, a schematic diagram of a surgical navigation system is shown, which includes: a fluorescence surgical microscope 1, an optical positioning system 2, a Raman spectroscopy probe 3, a first positioning ball group 4, a second positioning ball group 5, a third positioning ball group 6, a near-infrared positioning laser emission position 201, a visible light camera 202, and a near-infrared laser positioning camera combination 203.

[0106] The fluorescence surgical microscope system comprises a surgical microscope optical path system, a laser, optical fiber, a filter assembly, and a camera. The laser is preferably a 785nm laser, which is guided into the filter assembly via optical fiber. The filter assembly includes a notch dichroic mirror and a notch filter, with the notch center wavelength preferably at 785nm. The filter assembly is mounted directly below the microscope objective. The laser is incident at a 90° angle, reflected by the notch dichroic mirror (45°), and becomes coaxial with the microscope's field of view. When a fluorescent agent is present in the microscope's field of view, it will be excited by the excitation light to produce fluorescence. Additionally, white light illumination is guided to the front of the filter assembly via optical fiber and a ring illumination beam guide, providing white light illumination without interfering with fluorescence imaging. The fluorescence and reflected white light pass directly through the notch dichroic mirror and notch filter, filtering out stray excitation and reflected light, before entering the surgical microscope optical system and finally the camera. The camera contains a beam splitter and bandpass coating to separate the white light and fluorescence and focus them onto a CMOS chip, forming a white light and fluorescence image.

[0107] The Raman spectroscopy measurement system consists of a fiber optic lens Raman spectroscopy probe, a Raman spectrometer, and its built-in laser. It can excite and read Raman spectral signals in biological tissues, process them into digital signals, and send them to the Raman spectroscopy diagnostic system.

[0108] The Raman spectroscopy diagnostic system inputs the acquired Raman spectral digital signals into a pre-established deep learning model. The model then infers and generates classification results, including whether the tumor is malignant and its molecular subtype. The deep learning model is based on a one-dimensional convolutional neural network, or YOLO, VGG, or ResNet models. The Raman spectra undergo normalization and standardization preprocessing and are input into the model as two-dimensional images for training. This overcomes the influence of hardware differences between different Raman spectroscopy devices (such as differences in wavenumber step size between data points due to different slit widths, and differences in digital signal intensity due to different detector sensitivities), thereby improving the model's versatility.

[0109] The optical positioning system monitors the spatial position of a positioning ball fixed on a fluorescence surgical microscope, the side of the patient's head, and a Raman spectroscopy probe in real time. The software system disclosed herein then calculates the relative position and spatial angle between the three, providing data support for determining the position of the Raman spectroscopy measurement point in the preoperative three-dimensional reconstructed image and the two-dimensional projection position of the Raman spectroscopy measurement point from the microscope's perspective.

[0110] The software system includes a spatial three-dimensional coordinate algorithm, a two-dimensional projection algorithm for Raman measurement points under the microscope view, a real-time overlay algorithm for preoperative three-dimensional reconstructed medical images with microscope view and Raman probe position, an image fusion algorithm for fluorescence images and Raman measurement results, and an overall software display framework.

[0111] Furthermore, the Raman probe positioning method in this embodiment can be replaced by CT guidance or MRI guidance, but due to the large size of the equipment and the difficulty in positioning the microscope's position and angle, an optical positioning device is preferred.

[0112] The excitation wavelength of the fluorescence surgical microscope can be replaced with the excitation wavelength of other clinically commonly used fluorescent agents, such as 405±20nm for 5-ALA, 660±10nm for methylene blue, and 460-488nm for sodium fluorescein. Correspondingly, the selected notch filter combination is also for the selected excitation wavelength.

[0113] In addition to fiber optic lens probes, Raman probes can also use "filter + lens" probes. However, given the narrow operating space in glioma surgery, the former has the advantages of small size and ease of operation, so fiber optic lens probes are preferred.

[0114] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the surgical navigation system described above can be referred to the corresponding process in the embodiments of the aforementioned surgical navigation system processing method, and will not be repeated here.

[0115] This disclosure also provides an electronic device configured to run the above-described surgical navigation system processing method. Referring to FIG9, a schematic diagram of an electronic device is shown. The electronic device includes a memory 100 and a processor 101. The memory 100 is configured to store one or more computer instructions, which are executed by the processor 101 to implement the above-described surgical navigation system processing method.

[0116] Optionally, the electronic device shown in FIG9 further includes a bus 102 and a communication interface 103, wherein the processor 101, the communication interface 103 and the memory 100 are connected via the bus 102.

[0117] The memory 100 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 103 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network. The bus 102 may be an ISA bus, PCI bus, or EISA bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only a single bidirectional arrow is used in Figure 9, but this does not indicate that there is only one bus or one type of bus.

[0118] Processor 101 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 101 or by instructions in software form. The processor 101 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a readily available storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 100, and processor 101 reads information from memory 100 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0119] This disclosure also provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are called and executed by a processor, they cause the processor to implement the processing method of the surgical navigation system described above. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0120] The processing method, software system, and computer program product of the surgical navigation system provided in this disclosure include a computer-readable storage medium storing program code. The instructions included in the program code can be configured to execute the methods in the preceding method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and / or device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0122] Furthermore, in the description of the embodiments of this disclosure, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure based on the specific circumstances.

[0123] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0124] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure. Furthermore, the terms "first," "second," and "third" are configured for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0125] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims. Industrial applicability:

[0126] This disclosure provides a processing method, software system, and surgical navigation system for a surgical navigation system, which can provide data support for subsequently determining the position of Raman spectroscopy measurement points in preoperative three-dimensional reconstructed images and determining the two-dimensional projection position of Raman spectroscopy measurement points from a microscope perspective.

Claims

1. A processing method for a surgical navigation system, characterized in that, The software system applied to a surgical navigation system, the method comprising: The coordinate positions of multiple sets of positioning spheres are obtained based on an optical positioning instrument, and the centroid coordinates of each set of positioning spheres and the unit normal vector of the plane determined by the multiple sets of positioning spheres are calculated; wherein, the positioning spheres are set on a fluorescence surgical microscope, the side of the patient's head, and a Raman spectroscopy probe; The three-dimensional coordinate system with the origin of a set of positioning spheres on the side of the patient's head is defined as the world coordinate system. The spatial coordinates of each Raman point in the world coordinate system are obtained. Based on the spatial coordinates of each Raman point in the world coordinate system, the position of each Raman point on the corresponding pixel point in the image is calculated. Acquire preoperative medical images, translate the world coordinate system so that it overlaps with the coordinate system of the preoperative medical images at the origin, and display the positions of the corresponding pixels of each Raman point in the image in the preoperative medical images.

2. The processing method of the surgical navigation system according to claim 1, characterized in that, The steps of obtaining the coordinate positions of multiple sets of positioning spheres based on an optical positioning device, calculating the centroid coordinates of each set of positioning spheres, and the unit normal vector of the plane defined by the multiple sets of positioning spheres include: Measure the distance and angle parameters between multiple sets of the positioning balls; The default reference frame of the optical positioning device is adjusted so that the positive z-axis is the vertical line of the head pointing upwards, the positive x-axis is the sagittal axis pointing forwards, and the positive y-axis is the frontal axis pointing to the left. The center of gravity of a set of positioning balls on the side of the patient's head is set as the origin. Based on the optical positioning instrument, the coordinates of a set of positioning spheres are determined, and based on the coordinates of the set of positioning spheres, the centroid coordinates and the unit normal vector of the set of positioning spheres are determined. Based on the centroid coordinates of the set of positioning spheres, the unit normal vector of the set of positioning spheres, the distance parameter, and the angle parameter, calculate the vector connecting the centroid position of the set of positioning spheres to the center point, the coordinates of the center point, and the direction of the central axis.

3. The processing method of the surgical navigation system according to claim 2, characterized in that, The distance parameter and the angle parameter include at least one of the following: The distance between the center of gravity of the positioning ball and the corresponding point between the center of the microscope objective lens exit, the center point of the patient's skull, and the end of the Raman probe; The spatial angles between the lines connecting these distances and the corresponding positioning ball's normal direction; The angles between the projection of the distance line onto the plane where the positioning ball is located and the line connecting the first ball and the second ball in each group of positioning balls; The angle between the line connecting the first and second balls in each set of positioning balls and the line connecting the first and third balls.

4. The processing method of the surgical navigation system according to claim 3, characterized in that, Each set of positioning balls is installed such that its normal direction is parallel to the corresponding microscope field of view center axis, the patient's head vertical axis, and the Raman probe center axis.

5. The processing method of the surgical navigation system according to claim 1, characterized in that, The steps for calculating the position of each Raman point in the corresponding pixel on the image based on the spatial coordinates of each Raman point in the world coordinate system include: Transform the spatial coordinates of each Raman point in the world coordinate system to the camera coordinate system with the equivalent optical center of the microscope lens as the origin; Calculate the two-dimensional coordinates of each Raman point in the camera coordinate system onto the camera imaging plane; wherein the camera imaging plane is located behind the equivalent back focus of the equivalent optical center; The coordinate system containing the two-dimensional coordinates is translated to obtain the coordinates of each Raman point in the pixel coordinate system, which are used as the positions of the corresponding pixels on the image.

6. The processing method of the surgical navigation system according to claim 5, characterized in that, The step of transforming the spatial coordinates of each Raman point in the world coordinate system to the camera coordinate system with the equivalent optical center of the microscope lens as the origin includes: The spatial coordinates of each Raman point in the world coordinate system are transformed to the camera coordinate system with the equivalent optical center of the microscope lens as the origin using the following formula: Where R represents an orthogonal matrix, (x i ,y i ,z i (x′) represents the spatial coordinates in the world coordinate system. i ,y′ i ,z′ i ) represents the spatial coordinates in the camera coordinate system, t represents the three-dimensional translation vector, the orthogonal matrix is ​​the product of the rotation matrices corresponding to each three-dimensional direction, and the three-dimensional translation vector is the difference between the equivalent optical center of the microscope lens and the centroid position of a set of positioning spheres on the side of the patient's head.

7. The processing method of the surgical navigation system according to claim 5, characterized in that, The step of calculating the spatial coordinates of each Raman point in the camera coordinate system and the two-dimensional coordinates on the camera imaging plane includes: The spatial coordinates of each Raman point in the camera coordinate system and its two-dimensional coordinates on the camera imaging plane are calculated using the following formula: Among them, (x n ,y n ) represents the two-dimensional coordinates, (x′ i ,y′ i ,z′ i ) represents the spatial coordinates in the camera coordinate system, and f represents the equivalent back focus.

8. The processing method of the surgical navigation system according to claim 5, characterized in that, The step of translating the coordinate system containing the two-dimensional coordinates to obtain the coordinates of each Raman point in the pixel coordinate system as the position of the corresponding pixel point on the image includes: The coordinate system containing the two-dimensional coordinates is translated according to the following formula: Among them, (w i ,h i (x) represents the coordinates of the Raman point in the pixel coordinate system. n ,y n () represents the two-dimensional coordinates, (-w0,-h0) represents the translation vector, and dx and dy represent the actual size of each pixel in the w and h directions, respectively.

9. The processing method of the surgical navigation system according to claim 1, characterized in that, After the step of translating the world coordinate system so that it overlaps with the coordinate system of the preoperative medical image at the origin, the method further includes one of the following: Based on the location of the lesion identified in the preoperative medical images, the location of the lesion is displayed in the preoperative medical images; Calculate the field of view of the microscope and display the field cone of the microscope in the preoperative medical image; The position of the Raman probe is displayed in the preoperative medical images, so that the Raman probe is projected in a given angular direction.

10. The processing method of the surgical navigation system according to any one of claims 1-9, characterized in that, The method further includes: The Raman spectroscopy diagnostic system inputs the Raman spectroscopy data acquired by the Raman probe into a pre-established deep learning model and outputs lesion classification results. The lesion classification results are displayed at the positions of the corresponding pixels of each Raman point in the preoperative medical image.

11. The processing method of the surgical navigation system according to claim 10, characterized in that, The step of displaying the lesion classification results at the positions of the corresponding pixels of each Raman point in the preoperative medical image includes: As the camera lens moves, the Raman point currently displayed in the preoperative medical image is determined; The lesion classification result corresponding to the currently displayed Raman point is displayed at the position of the pixel corresponding to the currently displayed Raman point in the preoperative medical image.

12. A software system, characterized in that, The software system is configured to execute the processing method of the surgical navigation system according to any one of claims 1-11.

13. A surgical navigation system, characterized in that, The surgical navigation system includes: a fluorescence surgical microscope, a Raman spectroscopy measurement system, a Raman spectroscopy diagnostic system, an optical positioning system, and the software system described in claim 12.

14. An electronic device, characterized in that, The system includes a memory and a processor, the memory being configured to store computer instructions and the processor being configured to execute the computer instructions to implement the processing method of the surgical navigation system according to any one of claims 1-11.

Images