Multi-modal information fusion method and multi-modal OCT imaging system

Abstract

The invention discloses a multi-modal information fusion method and a multi-modal OCT imaging system, and the method comprises the steps: obtaining an intracavity interference signal, and generating an OCT structure diagram according to the intensity information of the interference signal; generating an NIRS spectral image according to the spectral information of the interference signal; generating a depolarization coefficient graph and a phase delay graph according to the polarization information of the interference signal; the OCT structure diagram and the NIRS spectral image are fused to form a first image, the NIRS spectral image and the depolarization coefficient diagram are fused, the NIRS spectral image and the depolarization coefficient diagram are arranged on the periphery of the first image to be displayed to form a second image after fusion, the OCT structure diagram and the phase delay diagram are subjected to weighted fusion, the OCT structure diagram and the phase delay diagram are arranged in the second image after fusion, and a final image is formed. According to the method, intracavity OCT, PSOCT and NIRS multi-modal information can be obtained through one-time catheter withdrawing, multi-modal fusion visualization is carried out, and calcified plaque information contained in the fused structure OCT, fiber plaque information contained in the polarization phase delay image and lipid plaque information contained in the NIRS and a depolarization coefficient image are obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of optical coherence tomography, and in particular to a multimodal information fusion method and a multimodal OCT imaging system. Background Art

[0002] Optical coherence tomography (OCT) is an optical imaging technology that can obtain non-invasive high-resolution cross-sectional images in real time, and is known as "optical biopsy." In the field of intracavitary imaging, high-resolution endoscopic OCT imaging makes it possible to analyze plaque components and can provide more accurate assistance for clinical diagnosis. Traditional intracavitary OCT imaging only uses light intensity information for structural imaging, but light signals also carry multi-dimensional information such as spectrum, polarization, and phase. Mining the information carried by light signals enables multimodal imaging of tissue components, and can obtain the characteristics of different tissue components from multiple dimensions and distinguish them more intuitively.

[0003] There have been some studies on multimodal imaging. In the patent document with publication number CN115336969A, a multimodal intracavitary imaging system and a multimodal data fusion method are disclosed, which realize the fusion of OCT structural images and NIRS images. However, the system requires two light sources and two catheters for separate imaging, and the system is complex and costly. In the patent document with publication number CN117322839A, a lipid plaque detection method and system combining OCT polarization and spectral information are disclosed. The system uses the combination of polarization and NIRS to achieve the accuracy of lipid plaque identification, but only uses the two orthogonal channels of the polarization imaging system to obtain spectral information separately and fuse them, and does not effectively utilize the polarization information. Summary of the invention

[0004] Therefore, the object of the present invention is to provide a multimodal information fusion method and a multimodal OCT imaging system, which utilizes a set of systems, a light source, a catheter, and a single pullback to acquire intracavitary OCT, PSOCT, and NIRS multimodal information and perform multimodal fusion visualization.

[0005] In order to achieve the above object, the present invention provides a multimodal information fusion method, comprising the following steps: acquiring an intracavity interference signal, generating an OCT structure diagram according to the interference signal; generating a NIRS spectral image according to the spectral information of the interference signal;

[0006] Calculate the Stokes matrix based on the polarization information of the interference signal;

[0007] Calculate the depolarization coefficient matrix DI based on the Stokes matrix and generate a depolarization coefficient map;

[0008] Calculate the phase delay matrix DPPR based on the Stokes matrix and generate a phase delay graph;

[0009] The OCT structure image is fused with the NIRS spectral image to form a first image, and the NIRS spectral image is fused with the depolarization coefficient map. After fusion, the fused image is placed at the corresponding position of the NIRS spectral image in the first image to form a second image; the OCT structure image is weighted fused with the phase delay map, and after fusion, the fused image is placed at the corresponding position of the OCT structure image in the second image to form a final image.

[0010] Preferably, the step of acquiring the intracavity interference signal further includes processing the interference signal, including the following steps:

[0011] Preprocessing the collected H channel and V channel interference signals;

[0012] The preprocessed interference signal is Fourier transformed to obtain the Fourier domain complex matrix of the H channel and the Fourier domain complex matrix of the V channel.

[0013] Further preferably, generating the OCT structure diagram according to the interference signal comprises the following steps:

[0014] Select half of the Fourier domain complex matrices of the H channel and the V channel, IMG_H and IMG_V, and parse them into OCT two-dimensional matrices; where IMG_H and IMG_V are both m*n two-dimensional matrices;

[0015] The OCT two-dimensional matrix is ​​normalized to obtain the OCT structure map;

[0016] The OCT structure diagram is transformed into polar coordinates to obtain an OCT structure pie chart.

[0017] Further preferably, the Stokes matrix is ​​calculated based on the polarization information of the interference signal, including: selecting half areas IMG_H and IMG_V of the Fourier domain complex matrices of the H channel and the V channel, IMG_H and IMG_V are both m*n two-dimensional matrices, and calculating the Stokes matrix according to formula (1);

[0018]

[0019] Among them, Re represents the real part of the complex number, Im represents the imaginary part of the complex number, and IMG_V * is the conjugate complex number of IMG_V; I, Q, U, V are all Stokes matrix parameters, and are all m*n two-dimensional matrices.

[0020] Further preferably, based on the Stokes matrix, the depolarization coefficient DI is calculated according to the following formula (2):

[0021]

[0022] Among them, I, Q, U, and V are all Stokes matrix parameters.

[0023] Further preferably, the calculation of the phase delay matrix DPPR based on the Stokes matrix comprises the following steps:

[0024] Construct Stokes parameters P according to the Stokes matrix, P = (Q / I, U / I, V / I), P is a three-dimensional matrix of m*n*3;

[0025] Any element in the Stokes parameter P is represented by P i,j,k (1≤i≤m, 1≤j≤n, 1≤k≤3), for any element P in the first set of data of the third dimension of the Stokes parameter P i,j,1 , taking the arbitrary element as the center, construct an r*r window (r=3,5,7...);

[0026] Subtract the first row from the rth row of the window to obtain a 1*r array, and calculate the average value of the 1*r array;

[0027] Repeat this operation for the three sets of data in the third dimension of the parameter matrix P to obtain a 1*3 vector α. Calculate the modulus of the vector α as the element P i,j,k Phase delay value DOPU at i,j,k ;

[0028] The phase delay value DPPR i,j is the phase delay value of a single point, and the DOPU of each element point is calculated along the m and n dimensions of the parameter matrix P respectively. i,j,k , and obtain the phase delay matrix of the parameter matrix P.

[0029] Preferably, the method further includes color coding the phase delay matrix DPPR, converting it into a color image, and performing polar coordinate conversion to obtain a phase delay map.

[0030] Further preferably, the fusion of the NIRS spectral image and the depolarization coefficient map comprises: multiplying the depolarization coefficient DI with the NIRS matrix to obtain a fused matrix N f :

[0031] N f =DI'·N Formula (3)

[0032] Where DI' represents the transpose of the matrix DI, · represents the dot product of the matrix, and N is the NIRS matrix.

[0033] Further preferably, the weighted fusion of the OCT structure image and the phase delay image includes weighting the phase delay image with a weighting coefficient w, and weighting the OCT structure image with 1-w;

[0034] The weighting coefficient w is calculated using the following formula (4):

[0035]

[0036] Among them, DI is the depolarization coefficient matrix and erf is the error function.

[0037] The present invention also provides a multimodal OCT imaging system for implementing the steps of the above-mentioned multimodal information fusion method, comprising a swept frequency light source, a polarization detection module, a reference arm, a sample arm, a first balanced detector, a second balanced detector and an image processing module;

[0038] The frequency-sweeping light source is used to emit frequency-sweeping laser;

[0039] The reference arm is used to receive part of the frequency-sweep laser to form a reference light signal;

[0040] The sample arm is used to receive another part of the frequency-sweep laser, irradiate the frequency-sweep laser to the detection part, and form a sampling light signal according to the light signal reflected back from the detection part;

[0041] The polarization detection module is used to polarize the signals input or output from the reference arm and the sample arm to obtain a polarization reference light signal and a polarization sampling light signal;

[0042] The first balanced detector and the second balanced detector are used to obtain an interference signal after the reference light signal and the polarization sampling light signal are coupled and interfered, and perform photoelectric conversion;

[0043] The image processing module is used to generate an OCT structure image, a NIRS spectrum image, a depolarization coefficient image, and a phase delay image according to the converted interference signal;

[0044] The OCT structure image is fused with the NIRS spectral image to form a first image, and the NIRS spectral image is fused with the depolarization coefficient map. After fusion, the fused image is placed at the corresponding position of the NIRS spectral image in the first image to form a second image; the OCT structure image is weighted fused with the phase delay map, and after fusion, the fused image is placed at the corresponding position of the OCT structure image in the second image to form a final image.

[0045] Compared with the prior art, the multimodal information fusion method and system disclosed in this application have at least the following advantages:

[0046] By adopting the multimodal information fusion method provided in the present application, intracavitary OCT, PSOCT and NIRS multimodal information can be obtained and multimodal fusion visualization can be performed by withdrawing the catheter once. The fused structural OCT contains calcified plaque information, the polarization phase delay image contains fibrous plaque information, and the NIRS and depolarization coefficient images contain lipid plaque information. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic flow chart of a multimodal information fusion method provided by the present invention.

[0048] Figure 2 This is an OCT structure diagram in a multimodal information fusion method provided by the present invention.

[0049] Figure 3 This is an OCT structure image and a NIRS image in a multimodal information fusion method provided by the present invention.

[0050] Figure 4 An OCT structure diagram and a depolarization coefficient diagram in a multimodal information fusion method provided by the present invention.

[0051] Figure 5 The present invention provides a PSOCT phase delay in a multimodal information fusion method.

[0052] Figure 6 The present invention provides a multimodal fusion image in a multimodal information fusion method.

[0053] Figure 7 A schematic diagram of the structure of a multimodal OCT imaging system provided in Example 1 of the present invention.

[0054] Figure 8 A schematic diagram of the structure of a multimodal OCT imaging system provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0055] The present invention is further described in detail below through the accompanying drawings and specific embodiments.

[0056] like Figure 1 As shown, a multimodal information fusion method provided by an embodiment of one aspect of the present invention comprises the following steps:

[0057] S1. Obtain the intracavity interference signal, generate an OCT structure diagram based on the intensity information of the interference signal; generate a NIRS spectral image based on the spectral information of the interference signal;

[0058] S2, calculating the Stokes matrix based on the polarization information of the interference signal;

[0059] S3, calculating the depolarization coefficient matrix DI based on the Stokes matrix and generating a depolarization coefficient graph;

[0060] S4, calculating the phase delay matrix DPPR based on the Stokes matrix, and generating a phase delay graph;

[0061] S5. Fuse the OCT structure image with the NIRS spectral image to form a first image; fuse the NIRS spectral image with the depolarization coefficient map, and place the fused image at a position corresponding to the NIRS spectral image of the first image to form a second image; perform weighted fusion on the OCT structure image and the phase delay map, and place the fused image at a position corresponding to the OCT structure image in the second image to form a final image.

[0062] It should be noted that the order of the above steps is not unique and can be changed as needed. In S1, the step of obtaining the intracavity interference signal also includes processing the interference signal, which includes the following steps:

[0063] Preprocessing the collected H channel and V channel interference signals;

[0064] Perform Fourier transform on the preprocessed interference signal to obtain a Fourier domain complex matrix of the H channel and a Fourier domain complex matrix of the V channel;

[0065] Furthermore, generating the OCT structure diagram according to the intensity information of the interference signal includes the following steps:

[0066] Select half of the Fourier domain complex matrices of the H channel and the V channel, IMG_H and IMG_V, and parse them into OCT two-dimensional matrices; where IMG_H and IMG_V are both m*n two-dimensional matrices;

[0067] The OCT two-dimensional matrix is ​​normalized to obtain the OCT structure map;

[0068] Specifically, the collected H and V channel interference signals are preprocessed by background subtraction, windowing, dispersion compensation, etc., and then Fourier transform is performed respectively to obtain the Fourier domain complex matrices of the two channels. Since the signal after Fourier transform is symmetrical on the left and right, any half can be taken for analysis. In this embodiment, the left half is selected for illustration, and the left half is taken to obtain IMG_H and IMG_V. IMG_H and IMG_V are both m*n two-dimensional matrices, where n is the number of lines in a frame of image, and m is the number of pixels on each line.

[0069] The OCT two-dimensional matrix is ​​obtained by the following formula:

[0070] Stru=10*lg(|IMG_H| 2 +|IMG_V| 2 )

[0071] Among them, Stru represents the OCT two-dimensional matrix, || represents the modulus of the complex number, and lg represents the logarithm with base 10.

[0072] Normalize the OCT two-dimensional matrix Stru:

[0073] Stru_s=255*(Stru-RgSt(1)) / (RgSt(2)-RgSt(1))

[0074] Wherein, Stru_s represents the normalized OCT structure diagram, RgSt represents the normalization parameter, RgSt(1) represents the lower limit, and RgSt(2) represents the upper limit.

[0075] The method further includes performing polar coordinate transformation on the OCT structure diagram to obtain an OCT structure pie chart Stru_r, such as Figure 2 shown.

[0076] In S2, the Stokes matrix is ​​calculated based on the polarization information of the interference signal. Specifically, the Stokes matrix is ​​constructed from IMG_H and IMG_V:

[0077]

[0078] Among them, Re represents the real part of the complex number, Im represents the imaginary part of the complex number, and IMG_V * is the conjugate complex number of IMG_V; I, Q, U, V are all Stokes matrix parameters, and are all m*n two-dimensional matrices.

[0079] In S3, based on the Stokes matrix, the depolarization coefficient matrix DI is calculated according to the following formula (2):

[0080]

[0081] Among them, I, Q, U, and V are all Stokes matrix parameters.

[0082] The depolarization coefficient matrix DI is transformed into a pie chart by polar coordinate conversion to obtain the depolarization coefficient graph DI_r.

[0083] In S4, it also includes color coding the phase delay matrix DPPR, converting it into a color image, and performing polar coordinate conversion to obtain a phase delay map.

[0084] The phase delay matrix DPPR is calculated based on the Stokes matrix to generate a phase delay graph, including the following steps:

[0085] Construct Stokes parameters P according to the Stokes matrix, P = (Q / I, U / I, V / I), P is a three-dimensional matrix of m*n*3;

[0086] Any element in the Stokes parameter P is represented by P i,j,k (1≤i≤m, 1≤j≤n, 1≤k≤3), for any element P in the first set of data of the third dimension of the Stokes parameter P i,j,1, taking the arbitrary element as the center, construct an r*r window (r=3,5,7...);

[0087] Subtract the first row from the rth row of the window to obtain a 1*r array, and calculate the average value of the 1*r array;

[0088] Repeat the above operation for the other two groups of data in the third dimension of the parameter matrix P to obtain the corresponding average value of each group. The three average values ​​form a 1*3 vector α. Calculate the modulus of the vector α as the element P i,j,k Phase delay value DPPR at (i,j) in the m*n plane i,j ;

[0089] The aforementioned phase delay value DPPR i,j is the phase delay value of a single point, so it is necessary to loop along the m and n dimensions of the parameter matrix P to calculate the DPPR of each element point i,j , and obtain the phase delay matrix DPPR of the parameter matrix P.

[0090] The phase delay matrix DPPR is color-coded, converted into a color image, and polar coordinates are converted to obtain a phase delay graph DPPR_cr, such as Figure 5 shown.

[0091] In S5, the OCT structure image and the NIRS spectral image are fused to form a first image; including placing the NIRS spectral image on the periphery of the OCT structure image to form the first image; Figure 3 shown.

[0092] Further preferably, when fusing the NIRS spectral image with the depolarization coefficient map, the depolarization coefficient DI is multiplied by the NIRS matrix to obtain a fused matrix N f :

[0093] N f =DI'·N Formula (3)

[0094] Where DI' represents the transpose of the matrix DI, · represents the dot product of the matrix, and N is the NIRS matrix.

[0095] The fused matrix N f The second image is formed at a position corresponding to the NIRS spectral image of the first image.

[0096] Further preferably, the weighted fusion of the OCT structure image and the phase delay image includes weighting the phase delay image with a weighting coefficient w, and weighting the OCT structure image with 1-w;

[0097] The weighting coefficient w is calculated using the following formula (4):

[0098]

[0099] Among them, DI is the depolarization coefficient matrix and erf is the error function.

[0100] The fused image is:

[0101] Img=w·DPPR_cr+(1-w)·Stru_r

[0102] The fused image Img is placed at the position corresponding to the second image OCT structure image to form a final image.

[0103] Since the OCT structure image only contains light intensity information, and the calcified plaque has obvious features with clear edges in the OCT structure image, such as Figure 2 As shown in the circle. Therefore, this application uses OCT structural images to display calcification information. NIRS images calculate lipid distribution based on spectral information, so this application uses NIRS images to indicate lipid plaques, such as Figure 3 ; Through the different dimensions of mutually orthogonal polarization information, multiple component results can be analyzed, among which the depolarization index (DI) can reflect the characteristics of lipid plaques, and the phase delay DPPR can reflect the characteristics of fibrous plaques. The image is as follows Figure 4 , Figure 5 This application integrates OCT structural images, NIRS images, and polarization information to enhance system visualization and facilitate image reading.

[0104] The present invention also provides a multimodal OCT imaging system for implementing the steps of the above-mentioned multimodal information fusion method, comprising a swept frequency light source, a polarization detection module, a reference arm, a sample arm, a first balanced detector, a second balanced detector and an image processing module;

[0105] The frequency-sweeping light source is used to emit frequency-sweeping laser;

[0106] The reference arm is used to receive part of the frequency-sweep laser to form a reference light signal;

[0107] The sample arm is used to receive another part of the frequency-sweep laser, irradiate the frequency-sweep laser to the detection part, and form a sample light signal according to the light signal reflected back from the detection part;

[0108] The polarization detection module polarizes the reference light signal returned by the reference arm, performs polarization-maintaining interference with the sample light signal returned by the sample arm, obtains an interference light signal, and then uses a polarization beam splitter to split the interference light signal into orthogonal H channel and V channel interference light signals;

[0109] The first balanced detector and the second balanced detector are used to obtain the interference light signals of the H channel and the V channel and perform photoelectric conversion;

[0110] The image processing module is used to generate an OCT structure image, a NIRS spectral image and a PSOCT polarization image according to the converted interference signal; the PSOCT polarization image includes a depolarization coefficient image and a phase delay image;

[0111] The OCT structure image is fused with the NIRS spectral image to form a first image; the NIRS spectral image is fused with the depolarization coefficient map, and after fusion, it is placed at the corresponding position of the NIRS spectral image in the first image to form a second image; the OCT structure image is weighted fused with the phase delay map, and after fusion, it is placed at the corresponding position of the OCT structure image in the second image to form a final image.

[0112] Example 1 Figure 7 As shown, the polarization detection module is connected to the return light output end of the sample arm and the reference arm; it includes: an online fiber polarizer and a polarization-maintaining coupler; the online fiber polarizer is connected to the return light output end of the reference arm, and the input end of the polarization-maintaining coupler is connected to the output end of the online fiber polarizer and the return light output end of the sample arm.

[0113] The light source uses a swept frequency light source in the 1310nm band. The swept frequency laser emitted by the light source is divided into two parts of 25% and 75% through a coupler. The 25% part enters the reference arm and the 75% part enters the sample arm. The light entering the reference arm enters from port 1 of circulator 1, exits from port 2, and goes to a collimator. The collimator converts the light in the optical fiber into parallel spatial light. The parallel spatial light is focused by a lens and irradiated onto a reflector and then reflected back. The front and rear positions of the reflector are adjustable to adjust the optical path so that the light of the reference arm and the sample arm interfere. The light entering the sample arm enters from port 1 of circulator 2 and exits from port 2. After the light exits from port 2 of circulator 2, it enters the optical fiber rotary connector. The optical fiber rotary connector drives the core of the imaging catheter to complete a 360° rotation scan. The light exits vertically from the imaging catheter, irradiates the tissue to be tested, and then reflects back. The reflected light carries the optical information of the tissue to be tested and returns to circulator 2 along the original optical path. The light returned from the reference arm and the sample arm is respectively converged to the polarization detection module through port 3 of circulator 1 and circulator 2 to complete the coupling interference. The light from the reference arm is first polarized by an online fiber polarizer with a polarization angle of 45°. The light after interference in the polarization-maintaining coupler is divided into orthogonal H channel and V channel through polarization beam splitter 1 and polarization beam splitter 2. The two channels are respectively photoelectrically converted by balanced detector 1 and balanced detector 2. The converted electrical signal enters the DAQ for acquisition, and data processing is completed on the host computer.

[0114] Example 2 Figure 8 As shown, the input end of the polarization detection module is connected to the output end of the polarization controller, including a circulator, a first polarization beam splitter, a second polarization beam splitter and a polarization-maintaining coupler; the input end of the circulator is connected to the output end of the polarization controller, the output end of the circulator is connected to the first polarization beam splitter, the first polarization beam splitter and the second polarization beam splitter are respectively connected to the polarization-maintaining coupler; the polarization-maintaining coupler is connected to the reference arm and the sample arm.

[0115] The same 1310nm band swept frequency light source is used. The light emitted by the light source enters the circulator through a single-mode optical fiber. There is a polarization controller on the single-mode optical fiber before entering the circulator to adjust the polarization state of the input light, modulate the light entering the circulator into linear polarized light, and the polarization state of the light must be maintained in the subsequent optical path. The light output by the circulator is transmitted to the first polarization beam splitter through the polarization-maintaining optical fiber, and the light output by the first polarization beam splitter is transmitted to the polarization-maintaining coupler through the polarization-maintaining optical fiber. The splitting ratio of the polarization-maintaining coupler is 50:50. The two beams of light split by the polarization-maintaining coupler enter the reference arm and the sample arm respectively through the polarization-maintaining optical fiber. The reference arm includes a collimator, a 22.5° quarter-wave plate, a focusing lens, and a reflector. The light of the reference arm is reflected back along the original path at the reflector; the sample arm is first a fiber driver, whose function is to drive the imaging catheter to rotate and retract scanning by a motor. The optical part of the imaging catheter consists of a fiber connector, a polarization-maintaining optical fiber, a 45° quarter-wave plate, a self-focusing optical fiber, and a deflection prism, of which the 45° quarter-wave plate is in the form of an optical fiber. The deflection prism makes the light in the catheter shoot out vertically and illuminate the tissue, and the scattered light carrying the optical information of the tissue returns to the catheter. The light returned by the reference arm and the sample arm interferes at the polarization-maintaining coupler and is divided into two beams, which are then divided into orthogonal H channel and V channel by polarization beam splitter 1 and polarization beam splitter 2 respectively. The H channel and V channel are converted into electrical signals by photoelectric conversion at balanced detector 1 and balanced detector 2 respectively, and collected by the DAQ data acquisition unit to the host computer for calculation and processing.

[0116] Obviously, the above embodiments are only examples for clear explanation, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A multimodal information fusion method, characterized in that: The following steps are involved: Obtain the intracavity interference signal, generate the OCT structure image according to the intensity information of the interference signal; generate the NIRS spectrum image according to the spectrum information of the interference signal; Calculate the Stokes matrix based on the polarization information of the interference signal; Calculate the depolarization coefficient matrix DI based on the Stokes matrix and generate a depolarization coefficient map; Calculate the phase delay matrix DPPR based on the Stokes matrix and generate a phase delay graph; The OCT structural image is fused with the NIRS spectral image to form a first image; The NIRS spectral image and the depolarization coefficient map are fused and placed at the corresponding position of the NIRS spectral image in the first image to form a second image; the OCT structure map and the phase delay map are weighted fused and placed at the corresponding position of the OCT structure map in the second image to form a final image.

2. The multimodal information fusion method according to claim 1, characterized in that: The step of obtaining the intracavity interference signal further includes processing the interference signal, including the following steps: Preprocessing the collected H channel and V channel interference signals; The preprocessed interference signal is Fourier transformed to obtain the Fourier domain complex matrix of the H channel and the Fourier domain complex matrix of the V channel.

3. The multimodal information fusion method according to claim 2, characterized in that: The method of generating an OCT structure diagram according to the intensity information of the interference signal comprises the following steps: Select half of the Fourier domain complex matrices of the H channel and the V channel, IMG_H and IMG_V, and parse them into OCT two-dimensional matrices, where IMG_H and IMG_V are both m*n two-dimensional matrices; The OCT two-dimensional matrix is ​​normalized to obtain the OCT structure map; The OCT structure diagram is transformed into polar coordinates to obtain an OCT structure pie chart.

4. The multimodal information fusion method according to claim 2, characterized in that: Calculating the Stokes matrix based on the polarization information of the interference signal, including: selecting half regions IMG_H and IMG_V of the Fourier domain complex matrices of the H channel and the V channel, IMG_H and IMG_V are both m*n two-dimensional matrices, and calculating the Stokes matrix according to formula (1); Among them, Re represents the real part of the complex number, Im represents the imaginary part of the complex number, and IMG_V * is the conjugate complex number of IMG_V; I, Q, U, V are all Stokes matrix parameters, and are all m*n two-dimensional matrices.

5. The multimodal information fusion method according to claim 4, characterized in that: Based on the Stokes matrix, the depolarization coefficient matrix DI is calculated according to the following formula (2): Among them, I, Q, U, and V are all Stokes matrix parameters.

6. The multimodal information fusion method according to any one of claims 1 to 4, characterized in that: The calculation of the phase delay matrix DPPR based on the Stokes matrix comprises the following steps: Construct Stokes parameters P according to the Stokes matrix, P = (Q / I, U / I, V / I), P is a three-dimensional matrix of m*n*3; Any element in the Stokes parameter P is represented by P i,j,k (1≤i≤m, 1≤j≤n, 1≤k≤3), for any element P in the first set of data of the third dimension of the Stokes parameter P i,j,1 , taking the arbitrary element as the center, construct an r*r window (r=3,5,7...); Subtract the first row from the rth row of the window to obtain a 1*r array, and calculate the average value of the 1*r array; Repeat the above operation for the other two groups of data in the third dimension of the parameter matrix P to obtain the corresponding average value of each group. The three average values ​​form a 1*3 vector α. Calculate the modulus of the vector α as the element P i,j,k Phase delay value DPPR at (i,j) in the m*n plane i,j ; The phase delay value DPPR i,j is the phase delay value of a single point, and the DPPR of each element point is calculated along the m and n dimensions of the parameter matrix P respectively. i,j , and obtain the phase delay matrix DPPR of the parameter matrix P.

7. The multimodal information fusion method according to claim 6, characterized in that: The method also includes color coding the phase delay matrix DPPR, converting it into a color image, and performing polar coordinate conversion to obtain a phase delay map.

8. The multimodal information fusion method according to claim 1, characterized in that: The fusion of the NIRS spectral image and the depolarization coefficient map includes: multiplying the depolarization coefficient matrix DI with the NIRS matrix to obtain a fused matrix N f : N f =DI'·N Formula (3) Where DI' represents the transpose of the matrix DI, · represents the dot product of the matrix, and N is the NIRS matrix.

9. The multimodal information fusion method according to claim 1, characterized in that: The weighted fusion of the OCT structure image and the phase delay image includes weighting the phase delay image with a weighting coefficient w and weighting the OCT structure image with 1-w; The weighting coefficient w is calculated using the following formula (4): Among them, DI is the depolarization coefficient matrix and erf is the error function.

10. A multimodal OCT imaging system, used to implement the steps of the multimodal information fusion method described in any one of claims 1 to 9, characterized in that: It includes a frequency sweeping light source, a polarization detection module, a reference arm, a sample arm, a first balanced detector, a second balanced detector and an image processing module; The frequency-sweeping light source is used to emit frequency-sweeping laser; The reference arm is used to receive part of the frequency-sweep laser to form a reference light signal; The sample arm is used to receive another part of the frequency-sweep laser, irradiate the frequency-sweep laser to the detection part, and form a sampling light signal according to the light signal reflected back from the detection part; The polarization detection module is used to polarize the signals input or output from the reference arm and the sample arm to obtain a polarization reference light signal and a polarization sampling light signal; The first balanced detector and the second balanced detector are used to obtain an interference signal after the reference light signal and the polarization sampling light signal are coupled and interfered, and perform photoelectric conversion; The image processing module is used to generate an OCT structure image, a NIRS spectrum image, a depolarization coefficient image, and a phase delay image according to the converted interference signal; The OCT structure image is fused with the NIRS spectral image to form a first image, and the NIRS spectral image is fused with the depolarization coefficient map. After fusion, the fused image is placed at the corresponding position of the NIRS spectral image in the first image to form a second image; the OCT structure image is weighted fused with the phase delay map, and after fusion, the fused image is placed at the corresponding position of the OCT structure image in the second image to form a final image.

Images