An image registration method and system

Abstract

This invention provides an image registration method and system. The image registration method includes: focusing a scanning beam emitted by an OCT system onto the surface of a chromogenic sample and displaying it as a microscopic image by a microscopic imaging unit; changing the galvanometer voltage of the galvanometer unit in the OCT system to change the scanning path of the scanning beam on the chromogenic sample surface; obtaining the coordinates of feature points in the OCT scanning path in the microscopic image, and the galvanometer voltage corresponding to each feature point to construct a mapping matrix; and performing matrix inverse operation using the mapping matrix and the pixel coordinates of the center point of the target OCT scanning area in the microscopic image to obtain the galvanometer voltage corresponding to the center point of the target OCT scanning area. This invention addresses the problems of insufficient registration accuracy and high calibration difficulty in existing image registration methods, and can meet the clinical needs for real-time high-precision display of the OCT scanning area and its corresponding B-scan image during surgery.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, and in particular to an image registration method and system. Background Technology

[0002] Optical coherence tomography (OCT) is a non-invasive medical imaging technique primarily used to acquire high-resolution cross-sectional images of biological tissues. When combined with microsurgical techniques that enable precise observation of lesions, OCT provides depth information about tissue structures, thus offering further precision support for the surgery.

[0003] Current methods for registering microscopic and OCT images suffer from insufficient registration accuracy and significant calibration challenges, making it difficult to meet the real-time requirements of using microscopic imaging to determine the scanning range and guide the OCT scanning path. More specifically, interference factors such as optical distortion, noise, and spatial mapping errors can all lead to poor registration accuracy during the registration process. Accurate mapping between OCT scanning imaging and microscopic imaging requires a complex calibration process. Furthermore, in dynamic surgical scenarios, existing registration methods struggle to achieve precise image alignment within a short timeframe, resulting in insufficient image registration accuracy.

[0004] In view of this, it is necessary to improve the existing image registration methods to solve the above problems. It should be noted that the above description of the background technology is only for the purpose of clearly and completely explaining the technical solutions of this application and facilitating understanding by those skilled in the art. It should not be assumed that these technical solutions are known to those skilled in the art simply because they have been described in the background technology section of this application. Summary of the Invention

[0005] The purpose of this invention is to disclose an image registration method and system to solve the problems of insufficient registration accuracy and high calibration difficulty in existing image registration methods, and to meet the clinical needs of real-time high-precision display of OCT scan area and its corresponding B-scan image during surgery.

[0006] To achieve the above objectives, the present invention provides an image registration method, comprising: focusing a scanning beam emitted by an OCT system onto the surface of a colorimetric sample and displaying it as a microscopic image by a microscopic imaging unit; and changing the mirror voltage of the mirror unit of the OCT system to change the scanning path of the scanning beam on the surface of the colorimetric sample.

[0007] The coordinates of feature points in the OCT scanning path in the microscopic image and the galvanometer voltage corresponding to each feature point are obtained to construct a mapping matrix;

[0008] By performing matrix inverse operation on the mapping matrix and the pixel coordinates of the center point of the target OCT scanning area in the microscopic image, the galvanometer voltage corresponding to the center point of the target OCT scanning area can be obtained.

[0009] As a further improvement of the present invention, the galvanometer unit includes an X-mirror and a Y-mirror. By controlling the voltage of the X-mirror and the Y-mirror, the deflection angle of the X-mirror and the Y-mirror is controlled, so that the scanning beam emitted from the OCT system hits the surface of the colorimetric sample in a preset regular path to form a number of the aforementioned feature points.

[0010] Obtain the center pixel coordinates of several feature points formed by the scanning beam on the surface of the colorimetric sample, as well as the mirror voltage coordinates of the X-mirror and Y-mirror corresponding to the center pixel coordinates of the feature points.

[0011] As a further improvement of the present invention, obtaining the center pixel coordinates of a plurality of feature points formed on the surface of the colorimetric sample by the scanning beam along a preset regular path includes:

[0012] The scanning beam emitted by the OCT system forms a light spot of a specific size and approximately circular shape on the surface of the colorimetric sample;

[0013] Extract the edge points of each light spot corresponding to several feature points in the microscopic image;

[0014] A least-squares circle fit is performed on the edge points of each extracted light spot to obtain the center pixel coordinates and radius of the light spot.

[0015] As a further improvement of the present invention, the two-dimensional affine transformation matrix T from the pixel coordinate space to the galvanometer voltage space is constructed by taking the galvanometer voltage corresponding to several feature points and the center pixel coordinates of the corresponding feature points in the microscopic image as input.

[0016] The coordinates are calculated by solving the mapping matrix and outputting the corresponding center pixel from the galvanometer voltage coordinates.

[0017] The coordinates are calculated by solving the mapping matrix and the center pixel coordinates to output the corresponding mirror voltage.

[0018] As a further improvement of the present invention, the formula for solving the mapping matrix is:

[0019]

[0020] Where x and y are the voltage coordinates of the galvanometer, x ' and y ' The coordinates of the center pixel;

[0021] The formula for the two-dimensional affine transformation matrix T is:

[0022]

[0023] Based on the solution of the mapping matrix, the parameters in the formula for calculating the two-dimensional affine transformation matrix are calculated, where a, b, c, and d are the parameters of the affine transformation matrix, and t x and t y These are translation parameters;

[0024] As a further improvement of the present invention, the coordinates are calculated by solving the mapping matrix and outputting the corresponding center pixel of the galvanometer voltage coordinates. Center pixel coordinates calculation The calculation verification formula is as follows:

[0025]

[0026] The coordinates are calculated by solving the mapping matrix and the center pixel coordinates to output the corresponding mirror voltage. Calculation coordinates of galvanometer voltage The calculation verification formula is as follows:

[0027]

[0028] As a further improvement of the present invention, the coordinates of the center pixel of the current scanning beam in the microscope image are obtained and matrix inverse operation is performed to calculate the current galvanometer voltage coordinates.

[0029] Based on the same inventive concept, this invention also discloses an image registration system, characterized in that it includes: a microscope imaging unit, an OCT system, and a data acquisition, registration, and calculation unit. The OCT system includes an OCT light source collimation unit, an OCT scanning unit, an OCT optical path deflection unit, and an OCT focusing unit. The OCT scanning unit has a galvanometer unit.

[0030] The scanning beam emitted by the OCT light source collimation unit is reflected by the galvanometer unit and the optical path deflection unit, and then focused by the OCT focusing unit onto the surface of the colorimetric sample and displayed as a microscopic image by the microscopic imaging unit;

[0031] The data acquisition and registration calculation unit obtains the feature points of the scanning path in the microscopic image and the corresponding galvanometer voltages, and calibrates them with the scanning path displayed in the microscopic image.

[0032] As a further improvement of the present invention, the microscope imaging unit has a dichroic mirror, an objective lens, a microscopic imaging module and a stage. The microscopic imaging module is located on the side of the dichroic mirror away from the objective lens. The lens of the microscopic imaging module and the objective lens are arranged along the same central optical axis. The dichroic mirror is at a 45° angle to the central optical axis. The OCT optical path deflection unit and the OCT focusing unit are the dichroic mirror and objective lens shared with the microscope imaging unit.

[0033] The stage is positioned on the side of the objective lens away from the dichroic mirror. The height of the stage is adjusted to adjust the relative distance between the colorimetric sample placed on the stage and the objective lens.

[0034] As a further improvement of the present invention, the galvanometer unit includes an X-mirror and a Y-mirror. The scanning beam emitted by the OCT light source collimation unit is emitted to the X-mirror and then reflected by the X-mirror to the Y-mirror. It is then emitted from the Y-mirror to the dichroic mirror. The path feature points of the scanning beam emitted by the OCT scanning unit are adjusted by adjusting the voltage of the X-mirror and the Y-mirror.

[0035] The microscopic image and the OCT scanning area are calibrated using the path feature points of the scanning beam and the corresponding voltage coordinates of the X and Y mirrors.

[0036] Compared with existing technologies, the beneficial effects of this invention are as follows: The scanning beam emitted by the OCT system is focused on the colorimetric sample to form a microscopic image. Furthermore, by changing the galvanometer voltage of the galvanometer unit in the OCT system, the path generated by the scanning beam focusing on the surface of the colorimetric sample is altered. This path is presented in the microscopic image. By determining the feature points of the path and their corresponding galvanometer voltages, a corresponding mapping matrix is ​​obtained. Then, the current galvanometer voltage is derived by inversely using the center pixel coordinates of the target OCT scanning area in the microscopic imaging and the mapping matrix. This achieves real-time synchronous presentation of the corresponding area between the microscopic image and the OCT system, as well as rapid response control. Compared with existing technologies, this invention effectively solves the display synchronization problem caused by system differences. Due to fundamental differences in hardware construction and imaging mechanisms between microscopic imaging systems and OCT scanning systems, there is an inherent systematic deviation between the coordinate systems used by the two systems when acquiring images. During surgery, the OCT scanning area is guided by the scanning path in the microscopic image. When it is necessary to adjust the OCT scanning path, the required scanning path in the microscopic image is specified. The galvanometer voltage of the galvanometer unit can be quickly obtained through the mapping matrix calculated by registration. After adjusting the galvanometer voltage to the calculated value, a precise OCT B-scan image can be presented for the specified scanning path in the microscopic image. This effectively simplifies the control difficulty caused by the different control modes between the microscope and the OCT system, and achieves the effect of aligning the microscopic image with the corresponding OCT-B-scan image in a short time, thus meeting the real-time requirements during surgery. Attached Figure Description

[0037] Figure 1 This is a flowchart of the image registration method of the present invention;

[0038] Figure 2 This is a flowchart illustrating the specific process of step S3 in the image registration method of the present invention.

[0039] Figure 3 This is a flowchart illustrating the specific process of step S31 in the image registration method of the present invention.

[0040] Figure 4 This is a schematic diagram illustrating how a cross-shaped scanning beam strikes a colorimetric sample to form five feature points in this invention.

[0041] Figure 5 This is a schematic diagram illustrating the microscopic image and OCT cross-sectional state of the feature points in the microscope imaging unit in this invention.

[0042] Figure 6 This is a schematic diagram of the center pixel coordinates of the microscopic image presented by the ophthalmic surgical microscopic imaging unit in this invention and its corresponding OCT cross-section.

[0043] Figure 7 This is a schematic diagram of the image registration system in this invention;

[0044] Figure 8 This is a schematic diagram of the optical path of the scanning beam emitted from the OCT light source collimation unit in the image registration system of the present invention, which passes through the galvanometer unit, dichroic mirror, and objective lens in sequence before being imaged onto the microscope imaging module. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. However, it should be noted that these embodiments are not intended to limit the present invention. Equivalent changes or substitutions in function, method, or structure made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0046] In brief, the image registration methods disclosed in the various embodiments shown in this application are as follows: Figures 1 to 3As shown, the scanning beam emitted by the OCT system is focused on the chromogenic sample to form a microscopic image. By changing the galvanometer voltage of the galvanometer unit of the OCT system to change the path of the scanning beam focused on the surface of the chromogenic sample, the path is presented in the microscopic image. The corresponding mapping matrix is obtained by determining the characteristic points of the path and their corresponding galvanometer voltages. Then, the current galvanometer voltage is deduced inversely from the central pixel coordinates of the target OCT scanning area in the microscopic imaging and the mapping matrix, so as to achieve the effect of real-time synchronization of the microscopic image and the OCT system in presenting the corresponding area and controlling fast response. Compared with the prior art, the present invention effectively solves the display synchronization problem caused by system differences. The microscopic imaging system and the OCT scanning system working in different wavelength bands have inherent deviations in their image coordinate systems and scanning coordinate systems due to different physical fields of view and imaging principles; during the operation, the OCT scanning range is guided by the scanning path in the microscopic image. When the OCT scanning path needs to be adjusted, the scanning path to be specified in the microscopic image is designated, and the galvanometer voltage of the galvanometer unit can be quickly obtained through the mapping matrix calculated by registration. After adjusting the galvanometer voltage to the calculated value, an accurate OCT-B-scan image of the scanning path specified in the microscopic image can be presented, so as to effectively simplify the control difficulty brought by different control modes between the microscope and the OCT system, and achieve the effect of aligning the microscopic image and the corresponding OCT-Bscan image within a short time to meet the real-time requirement during the operation.

[0047] Specifically, it should be noted that B-scan (cross-sectional scan) refers to a two-dimensional imaging mode in optical coherence tomography (OCT). It quickly moves the scanning beam by changing the rotation angle of the galvanometer, and combines a series of continuous A-scan (depth scan) signals; each A-scan signal reflects the depth structure information of biological tissue at a certain lateral point, and its backscattered light intensity is converted into image brightness, and finally a two-dimensional gray-scale image of the tissue cross-section is generated on the screen. This method is especially suitable for ophthalmology and can present the layered characteristics of structures such as the retina with high resolution.

[0048] Refer to Figure 1 As shown, a method for image registration provided in this embodiment includes steps S1 to S4:

[0049] S1. The scanning beam emitted by the OCT system is focused on the surface of the chromogenic sample and displayed as a microscopic image by the microscopic imaging unit 10.

[0050] S2. Change the galvanometer voltage of the galvanometer unit of the OCT system to change the scanning path of the scanning beam on the surface of the chromogenic sample. Further, the scanning path of the scanning beam on the surface of the chromogenic sample is presented in the microscopic image (such as Figure 4As shown in the figure, guided by the scanning line in the microscopic image, the scanning path corresponding to the corresponding OCT system is modified.

[0051] S3. Obtain the coordinates of the feature points in the microscopic image in the OCT scanning path, and the galvanometer voltages corresponding to each feature point to construct a mapping matrix.

[0052] S4. Perform matrix inverse operation through the mapping matrix and the pixel coordinates of the center point of the target OCT scanning area in the microscopic image to obtain the galvanometer voltage corresponding to the center point of the target OCT scanning area.

[0053] Specifically, referring to Figure 8 As shown, the galvanometer unit 201 includes an X galvanometer 201a and a Y galvanometer 201b. In step S2, by controlling the galvanometer voltages of the X galvanometer 201a and the Y galvanometer 201b, the deflection angles of the X galvanometer 201a and the Y galvanometer 201b are controlled, so that the scanning beam emitted by the OCT system hits the surface of the chromogenic sample along a preset regular path. It should be noted that in this embodiment, the regular scanning path (such as: a straight line, a cross, a star shape or other regular paths) presented by the scanning beam emitted by the OCT system 2 on the surface of the chromogenic sample will form several feature points. The feature points can specifically be several end points formed during the scanning path process (combined with Figure 3 As shown). Further, in this embodiment, the chromogenic sample is specifically a near-infrared chromogenic card.

[0054] Referring to Figure 2 As shown, step S3 includes steps S31 to S33:

[0055] S31. Obtain the central pixel coordinates of several feature points formed by the scanning beam on the surface of the chromogenic sample, and the galvanometer voltage coordinates of the X galvanometer and the Y galvanometer corresponding to the central pixel coordinates of the feature points.

[0056] Referring to Figure 3 As shown, step S31 specifically includes steps S311 and S312:

[0057] S311. Extract the edge points of each light spot corresponding to several feature points in the microscopic image respectively. Specifically, the edge points of the light spot are extracted by edge detection.

[0058] It should be noted that in this embodiment, the preset regular path of the scanning beam hitting the surface of the chromogenic sample is a cross and forms five feature points as shown in Figure 3 As shown, Figure 5 The galvanometer voltage coordinates corresponding to the five feature points in are A(0, 0), B(0, 1), C(0, -1), D(-1, 0), E(1, 0). It should be noted that in combination with Figure 4 and Figure 5As shown, due to factors such as the optical diffraction limit, the Gaussian distribution characteristics of the light source, and system aberrations, the microscopic image presented on the OCT scanning beam developed sample 5 is not a concentrated light spot, but rather a diffuse light spot that is nearly circular and has a specific size. This diffuse light spot suffers from significant noise and unclear edges. Therefore, it is necessary to determine the center pixel coordinates of the corresponding light spot in the microscopic image for the five feature points to reduce the impact of noise on the accuracy of image registration, thereby improving the accuracy of image registration.

[0059] S312. Perform a least-squares circle fitting on the edge points of each extracted light spot to obtain the center pixel coordinates and radius of the light spot. Specifically, the circle equation is obtained through least-squares fitting:

[0060] (xa) 2 +(yb) 2 =r 2

[0061] The center pixel coordinates (a, b) and radius r of the spot formed by the feature points in the microscopic image are calculated using the above equations. It should be noted that when performing edge detection on the spot, it is sufficient to obtain the coordinates of at least three edge points. Taking the obtained three edge points of the spot as (3, 1), (1, 3), and (5, 3) as an example, the center pixel coordinates of the spot can be obtained as (3, 3), and the radius is 2. Based on the circle equation and the extracted edge points, the center pixel coordinates of the five feature points can be calculated separately.

[0062] S32. Using the mirror voltages corresponding to several feature points and the center pixel coordinates of the corresponding feature points in the microscopic image obtained by fitting as input, construct a two-dimensional affine transformation matrix T to solve the transformation from pixel coordinate space to mirror voltage space.

[0063] S33. The data acquisition and registration calculation unit obtains the mirror voltage calculation coordinates of the feature point positions by solving the mapping matrix. And the coordinates of the center pixel

[0064] Specifically, the formula for solving the mapping matrix is ​​shown in Formula 1, where x and y are the mirror voltage coordinates, and x' and y' are the center pixel coordinates:

[0065]

[0066] The formula for the two-dimensional affine transformation matrix is ​​shown in Formula 2, where a, b, c, and d are the parameters of the affine transformation matrix, and t x and t y Translation parameters:

[0067]

[0068] The center pixel coordinates of the five feature points formed by the aforementioned scanning path in the microscopic image are obtained by changing the voltage coordinates of the x-mirror and y-mirror. Therefore, the mirror voltage coordinates are used as the original points, and the center pixel coordinates of the feature points are used as the transformed mapping points to solve the mapping matrix (Formula 1) to calculate the parameters in the two-dimensional spatial matrix, i.e.:

[0069] x'=a×x+b×y+t x (Formula 3)

[0070] y'=c×x+d×y+t y (Formula 4)

[0071] It should be noted that a, b, and t are calculated using Formula 3. x The three parameters, c, d, and t, are calculated using Formula 4. y Therefore, given the three parameters, the center pixel coordinates (x', y') of three of the aforementioned five feature points, along with the corresponding galvanometer voltage coordinates (x, y), can be substituted into the above calculation formula to calculate parameters a, b, c, d, and t respectively. x and t y Thus, Formula 2 is obtained. In subsequent steps, error verification is performed using the center coordinates of the other two feature points and their corresponding galvanometer voltage coordinates.

[0072] It should be noted that parameters a, b, c, and d are used to control different geometric transformations. Specifically, parameters a and d control geometric scaling, which means scaling the image by a factor of a in the x-direction and by a factor of d in the y-direction. The scaling transformation matrix is ​​as follows:

[0073]

[0074] Parameters a, b, c, and d jointly control the geometric rotation. When controlling the rotation transformation of the image, the image is rotated counterclockwise by an angle θ around the origin (or a specified point). Therefore, in the rotation transformation matrix, a = cosθ, b = -sinθ, c = sinθ, and d = cosθ. The specific rotation transformation matrix is ​​as follows:

[0075]

[0076] Parameters b and c control the geometric cropping of the image. The cropping transformation can tilt the image horizontally or vertically. When cropping the image horizontally along the x-axis, b = tanφ (φ is the cropping angle), a and d are both 1, and the cropping transformation matrix is ​​as follows:

[0077]

[0078] When performing vertical shearing of the image along the y-axis, c = tanφ (φ is the shearing angle), a and d are 1, and the shearing transformation matrix is ​​as follows:

[0079]

[0080] Translation parameter t x and t y Controlling image translation means shifting the image by t in the x-direction. x Times, translated t in the y direction y The translation transformation matrix is ​​as follows:

[0081]

[0082] For example, by inputting the mirror voltage coordinates and center pixel coordinates corresponding to three feature points, the above parameters can be calculated and output. Combining this with the previous steps, selecting three mirror voltage coordinates A(0,0), E(1,0), and B(0,1), and their corresponding center pixel coordinates A1(1,1), E1(5,3), and B1(3,4), the values ​​of each parameter can be calculated as a=4, b=2, c=2, d=3, and t=4. x =1,t y =1, that is:

[0083]

[0084] The coordinates are calculated by solving the mapping matrix and outputting the corresponding center pixel from the galvanometer voltage coordinates. Center pixel coordinates calculation The calculation verification formula is as follows:

[0085]

[0086] The coordinates are calculated by solving the mapping matrix and the center pixel coordinates to output the corresponding mirror voltage. Calculation coordinates of galvanometer voltage The calculation verification formula is as follows:

[0087]

[0088] Specifically, the values ​​of each parameter are calculated using the three galvanometer voltage coordinates A, E, and B. The center pixel coordinates of the other two feature points (C(0, -1) and D(-1, 0)) and their corresponding center pixel coordinates are then combined to verify the error of the center pixel calculation and the accuracy of the galvanometer voltage calculation, i.e., to verify the accuracy of the solution matrix. Further, in this embodiment, the center pixel coordinates corresponding to galvanometer voltage coordinate C are (-1, -2), and the center pixel coordinates corresponding to galvanometer voltage coordinate D are (-3, -1).

[0089] The center pixel coordinates corresponding to the galvanometer voltage coordinates C(0, -1) are calculated using the aforementioned formula (5):

[0090]

[0091] The calculated center pixel coordinates corresponding to the galvanometer voltage coordinates C(0, -1) are (-1, -2). Since the calculated center pixel coordinates are consistent with the actual center pixel coordinates, it shows that the corresponding center pixel coordinates can be accurately calculated by solving the mapping matrix and the galvanometer voltage coordinates.

[0092] Using the aforementioned formula (6) and the center pixel coordinates (-3, -1) corresponding to the galvanometer voltage coordinate D, the corresponding galvanometer voltage calculation coordinates can be deduced:

[0093]

[0094] The calculated coordinates of the galvanometer voltage are (-1, 0) obtained by calculating the center pixel coordinates corresponding to the galvanometer voltage coordinates D. Since the calculated coordinates of the galvanometer voltage are consistent with the actual galvanometer voltage coordinates D, it shows that the corresponding galvanometer voltage coordinates can be accurately calculated by solving the mapping matrix and the Zhongxiang pixel coordinates. It should be noted that the above formulas (5) and (6) are variations of solving the mapping matrix (i.e., formula 2). Based on the obtained solution mapping matrix, formulas (5) and (6) can be easily derived for the verification calculation of the center pixel coordinates and the galvanometer voltage coordinates. It should be noted that... Figure 5 Specifically, this is a schematic diagram of a feature point in a microscopic image, displayed as a light spot on a touchscreen. The displayed number (363, 270) represents the center pixel coordinates of that feature point. However, this number is not related to the five center pixel coordinates set above for explaining the calculation of the mapping matrix. Furthermore, the display area (i.e., center pixel coordinates) of the microscopic image can be personalized using the various arrow buttons on the touchscreen to suit the actual needs of the surgical procedure.

[0095] The coordinates of the center pixel of the current scanning beam in the microscope image are obtained and matrix inversion is performed to calculate the current galvanometer voltage coordinates. Specifically, the formula used for matrix inversion is formula (6). The parameters in the two-dimensional affine transformation matrix obtained in the previous steps have been verified to be accurate. During the operation, the microscopic images obtained by the operator are as follows: Figure 6 As shown, the center of the crosshairs displayed on the screen represents the center pixel coordinates of the current microscopic image. These center pixel coordinates correspond to a mirror voltage coordinate. Figure 6The image shown is specifically an image presented during eye surgery, and a cross-sectional OCT scan image cut at the center pixel coordinate point. Specifically, it obtains the OCT scan image corresponding to the center pixel coordinate point of the microscopic image displayed on the screen. When it is necessary to obtain OCT scan images at other points, the position of the center pixel coordinate point is moved (i.e.,...). Figure 6 The intersection of the crosshairs shown in the image is located in the microscope image, and the corresponding center pixel coordinates are displayed on the screen. Figure 6 The current center pixel coordinates are shown as (278, 222). Substituting these coordinates into the aforementioned formula (6), the corresponding galvanometer voltage coordinates are calculated. Then, the deflection angles of the X and Y galvanometers are adjusted according to the galvanometer voltage coordinates to ensure that the scanning beam emitted by the OCT system hits the specified center pixel coordinates, and to perform OCT cross-sectional scanning on the specified center pixel coordinates. It should be noted that the calculation of each parameter in formula (2) in the aforementioned content is intended to illustrate the calculation relationship between the center pixel coordinates and the galvanometer voltage coordinates, and may not necessarily be applicable to all cases. Figure 6 Calculation of the center pixel coordinates (278, 222) and its corresponding galvanometer voltage coordinates.

[0096] Using the image registration method described above, the scanning beam emitted by the OCT system projects five feature points onto the surface of the colorimetric sample, forming a scanning path. To address the noise interference caused by light spots formed by these feature points in the scanned image, the center pixel coordinates of the light spot are obtained using the least squares method. Furthermore, in this embodiment, three of the five feature points are used as input to construct a two-dimensional pixel matrix and solve the mapping matrix to calculate various parameters. Then, the center pixel coordinates and galvanometer voltage coordinates are calculated using the remaining two feature points combined with the solved mapping matrix. The accuracy of the solved mapping matrix is ​​ensured by comparing the calculated coordinates with the actual coordinates. The calculated coordinates are then compared with the actual coordinates. In practical surgical applications, by substituting the demapping matrix into the actual mapping matrix, the galvanometer voltage coordinates can be quickly calculated after specifying the center pixel coordinates. The scanning beam emitted by the OCT system can then be directed to the specified center pixel coordinate point to obtain the OCT cross-sectional scan image corresponding to the specified center pixel coordinate point. Compared to the problem in existing technologies where complex registration operations between the microscope system and the OCT system are required, which can affect the operation, the entire registration process only requires specifying the center pixel coordinates of the microscopic image to automatically calculate the galvanometer voltage coordinates, adjust the X and Y galvanometers, and obtain an accurate OCT cross-sectional scan image, thereby ensuring the smooth progress of the operation.

[0097] Based on the same inventive concept, this invention also discloses an image registration system, which... Figure 7 and Figure 8As shown, it includes: a microscope imaging unit 10, an OCT system 20, and a data acquisition, registration, and calculation unit 30. The OCT system 20 includes an OCT light source collimation unit 201, an OCT scanning unit 202, an OCT optical path deflection unit (not labeled), and an OCT focusing unit. The OCT scanning unit 202 has a galvanometer unit 200. The scanning beam emitted by the OCT light source collimation unit 201 is reflected by the galvanometer unit 200 and the optical path deflection unit, and then focused by the OCT focusing unit onto the surface of the chromogenic sample 5 and displayed as a microscopic image in the microscope imaging unit 10. The data acquisition, registration, and calculation unit 30 acquires the feature points of the scanning path in the microscopic image and the galvanometer voltage corresponding to the feature points, and performs microscopic image calibration.

[0098] The microscope imaging unit 10 includes a dichroic mirror 101, an objective lens 102, a microscopic imaging module 103, and a stage 104. The OCT focusing unit is located on the side of the dichroic mirror 101 away from the microscopic imaging module 103. Specifically, the OCT focusing unit is selected as a convex lens, and the OCT focusing unit is specifically the objective lens shared with the microscopic imaging unit 10. The optical path deflection unit is specifically the dichroic mirror 101 shared with the microscopic imaging unit 10. The lens of the microscopic imaging module 103 and the objective lens 102 are arranged along the same central optical axis O. The dichroic mirror 101 forms a 45° angle with the central optical axis. The stage 104 is located on the side of the objective lens 102 away from the dichroic mirror 101. The height of the stage 104 is adjusted to adjust the relative distance between the colorimetric sample 5 placed on the stage 104 and the objective lens 203.

[0099] The galvanometer unit 200 includes an X-mirror 200a and a Y-mirror 200b. The scanning beam emitted from the OCT light source collimation unit 201 is emitted to the X-mirror 200a, reflected by the X-mirror 200a, and then emitted from the Y-mirror 200b to the dichroic mirror 101. The path feature points of the scanning beam emitted from the OCT scanning unit to the surface of the colorimetric sample 5 are adjusted by regulating the voltages of the X-mirror 200a and Y-mirror 200b. Microscopic image calibration is performed using the path feature points of the scanning beam and the corresponding voltage coordinates of the X-mirror and Y-mirror. Specific calibration steps are described in the foregoing embodiments and will not be repeated here.

[0100] It should be noted that the light source emitted by the collimation unit 201 of the OCT light source is a near-infrared beam with a wavelength of 850nm or 1050nm to 1300nm. It has the advantage of less scattering in biological tissues (such as the retina and skin) and the ability to penetrate deeper structures. By selecting the color sample 5 as a color card, invisible light can be imaged in the microscopic imaging module 103 for observation.

[0101] Compared with the prior art, this embodiment effectively solves the problem that it is difficult to accurately synchronize the operation of microscopes and OCT scanning units operating in different wavelength bands to obtain consistent spatial information. By integrating the OCT scanning unit 202, OCT light source collimation unit 201, microscope imaging unit 10 and data registration calculation unit 30 into one unit, the scanning beam emitted by the OCT light source collimation unit 201 is reflected by the galvanometer unit 200, then reflected by the dichroic mirror 101 of the microscopic imaging unit 10, and then focused by the objective lens 102 onto the colorimetric sample 5 to form a feature point of the scanning path. The scanning beam reflected by the colorimetric sample 5 is then displayed as a microscopic image on the display screen of the microscopic imaging module 103 and the center pixel coordinates of the feature point are displayed (see figure). After the entire scan is completed, a total of five feature points are formed on the surface of the colorimetric sample 5. At the same time, the galvanometer voltage coordinates and center pixel coordinates of the five feature points are input to the registration calculation unit 30 to construct and solve the mapping matrix. The specific calculation steps are described in the aforementioned embodiment and will not be repeated here.

[0102] In this embodiment, by integrating the OCT system with the microscopic imaging unit, the scanning beam emitted by the OCT light source collimation unit 201 of the OCT system is emitted from the OCT scanning unit 202 and then focused on the chromogenic sample 5 through the optical path of the microscopic imaging unit 10 to finally form an image. This effectively solves the problem in the prior art where the inherent systematic deviation between the microscope and the OCT system due to the essential differences in hardware structure and imaging mechanism between the OCT scanning unit and the microscope imaging unit leads to a large registration error that affects the smooth progress of the surgery.

[0103] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

[0104] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0105] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An image registration method, characterized in that, include: The scanning beam emitted by the OCT system is focused on the surface of the colored sample and displayed as a microscopic image by the microscopic imaging unit. Changing the galvanometer voltage of the galvanometer unit in the OCT system changes the scanning path of the scanning beam on the surface of the colored sample. The coordinates of feature points in the OCT scanning path in the microscopic image and the galvanometer voltage corresponding to each feature point are obtained to construct a mapping matrix; By performing matrix inverse operation on the mapping matrix and the pixel coordinates of the center point of the target OCT scanning area in the microscopic image, the galvanometer voltage corresponding to the center point of the target OCT scanning area can be obtained.

2. The image registration method according to claim 1, characterized in that, The galvanometer unit includes an X-mirror and a Y-mirror. By controlling the voltage of the X-mirror and the Y-mirror, the deflection angle of the X-mirror and the Y-mirror is controlled, so that the scanning beam emitted from the OCT system hits the surface of the colorimetric sample in a preset regular path to form a number of the aforementioned feature points. Obtain the center pixel coordinates of several feature points formed by the scanning beam on the surface of the colorimetric sample, as well as the mirror voltage coordinates of the X-mirror and Y-mirror corresponding to the center pixel coordinates of the feature points.

3. The image registration method according to claim 2, characterized in that, Obtaining the center pixel coordinates of several feature points formed on the surface of the colorimetric sample by the scanning beam following a preset regular path includes: The scanning beam emitted by the OCT system forms a light spot of a specific size and approximately circular shape on the surface of the colorimetric sample; Extract the edge points of each light spot corresponding to several feature points in the microscopic image; A least-squares circle fit is performed on the edge points of each extracted light spot to obtain the center pixel coordinates and radius of the light spot.

4. The image registration method according to claim 3, characterized in that, Using the mirror voltages corresponding to several feature points and the center pixel coordinates of the corresponding feature points in the microscopic image obtained by fitting as input, a two-dimensional affine transformation matrix T is constructed to solve from the pixel coordinate space to the mirror voltage space. The coordinates are calculated by solving the mapping matrix and outputting the corresponding center pixel from the galvanometer voltage coordinates. The coordinates are calculated by solving the mapping matrix and the center pixel coordinates to output the corresponding mirror voltage.

5. The image registration method according to claim 4, characterized in that, The formula for solving the mapping matrix is: Where x and y are the mirror voltage coordinates, and x' and y' are the center pixel coordinates; The formula for the two-dimensional affine transformation matrix T is: Based on the solution of the mapping matrix, the parameters in the formula for calculating the two-dimensional affine transformation matrix are calculated, where a, b, c, and d are the parameters of the affine transformation matrix, and t x and t y These are the translation parameters of the galvanometer voltage coordinates along the x-axis and y-axis.

6. The image registration method according to claim 5, characterized in that, The coordinates are calculated by solving the mapping matrix and outputting the corresponding center pixel from the galvanometer voltage coordinates. Center pixel coordinates calculation The calculation verification formula is as follows: The coordinates are calculated by solving the mapping matrix and the center pixel coordinates to output the corresponding mirror voltage. Calculation coordinates of galvanometer voltage The calculation verification formula is as follows:

7. The image registration method according to claim 3, characterized in that, Obtain the center pixel coordinates of the current scanning beam in the microscope image and perform matrix inverse operation to calculate the current galvanometer voltage coordinates.

8. An image registration system, characterized in that, include: The system includes a microscope imaging unit, an OCT system, and a data acquisition, registration, and calculation unit. The OCT system comprises an OCT light source collimation unit, an OCT scanning unit, an OCT optical path deflection unit, and an OCT focusing unit. The OCT scanning unit has a galvanometer unit. The scanning beam emitted by the OCT light source collimation unit is reflected by the galvanometer unit and the optical path deflection unit, and then focused by the OCT focusing unit onto the surface of the colorimetric sample and displayed as a microscopic image by the microscopic imaging unit; The data acquisition and registration calculation unit obtains the feature points of the scanning path in the microscopic image and the corresponding galvanometer voltages, and calibrates them with the scanning path displayed in the microscopic image.

9. The image registration system according to claim 8, characterized in that, The microscope imaging unit includes a dichroic mirror, an objective lens, a microscopic imaging module, and a stage. The microscopic imaging module is located on the side of the dichroic mirror away from the objective lens. The lens of the microscopic imaging module and the objective lens are arranged along the same central optical axis. The dichroic mirror forms a 45° angle with the central optical axis. The OCT optical path deflection unit and the OCT focusing unit are the dichroic mirror and objective lens shared with the microscope imaging unit. The stage is positioned on the side of the objective lens away from the dichroic mirror. The height of the stage is adjusted to adjust the relative distance between the colorimetric sample placed on the stage and the objective lens.

10. The image registration system according to claim 9, characterized in that, The galvanometer unit includes an X-mirror and a Y-mirror. The scanning beam emitted by the OCT light source collimation unit is emitted to the X-mirror and then reflected by the X-mirror to the Y-mirror. It is then emitted from the Y-mirror to the dichroic mirror. The path feature points of the scanning beam emitted from the OCT scanning unit are adjusted by adjusting the voltage of the X-mirror and the Y-mirror. The microscopic image and the OCT scanning area are calibrated using the path feature points of the scanning beam and the corresponding voltage coordinates of the X and Y mirrors.

Images