Parathyroid gland spatial position positioning system and method based on fusion imaging and Raman spectroscopy
Through the fusion imaging and Raman spectroscopy, the data is collected and processed by laser probes, combined with spatial positioning methods, the problem of insufficient recognition accuracy in spatial position positioning of parathyroid gland is solved, and the precise positioning and high-precision recognition of parathyroid gland is achieved .
Patent Information
- Application Number
- CN202411109951.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The prior art lacks recognition accuracy in spatial position positioning of parathyroid glands, which fails to effectively solve the problem of insufficient recognition accuracy of parathyroid glands.
The fusion imaging and Raman spectroscopy method are used to collect Raman spectroscopy data and image data through laser probes, preprocess and fit, and precise positioning of the parathyroid gland is achieved in combination with spatial positioning methods.
It improves the real-time and accuracy of the spatial location of parathyroid gland, ensuring the accurate identification and positioning of parathyroid gland.
Smart Images

Figure CN118614880B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image positioning, and in particular to a parathyroid gland spatial position positioning system and method by integrating imaging and Raman spectroscopy. Background Art
[0002] At present, the system for identifying and locating the spatial position of the parathyroid glands includes a near-infrared light real-time identification parathyroid laparoscope system proposed in the patent application document with patent application number 202022519994.1; the system provides a near-infrared light real-time identification parathyroid laparoscope, through a coaxial bundling design, that is, the cold light source light guide, laser light guide, and camera light guide are bundled in the same optical tube, and the fusion registration technology is used to perform parathyroid autofluorescence and surrounding surgical field fusion imaging to locate the parathyroid gland in real time, and the near-infrared light irradiation and imaging system is combined with the laparoscope imaging system. Chinese patent application CN114343568A Device and method for identifying the position of parathyroid glands and evaluating the survival rate of parathyroid glands and Chinese patent CN215305781U Device for identifying the position of parathyroid glands and evaluating the survival rate of parathyroid glands China, through the endoscope component, it is connected to the light source unit and used to obtain the image information of the parathyroid surgical area or parathyroid glands irradiated with light.
[0003] There is an existing Chinese patent application CN115035049A. This method sets a marker point and calculates the image scanning error based on the difference between the measured position of the marker point in the image and the standard position information, and calculates the positioning accuracy error based on the calculated image scanning error. However, this method only compares a single type of image data and does not consider the accuracy of the captured image, which has great limitations. Summary of the invention
[0004] 1. Technical issues to be resolved
[0005] In view of the deficiencies in the prior art, the present invention provides a parathyroid gland spatial position positioning system that integrates imaging and Raman spectroscopy, which has the advantages of real-time positioning and accurate positioning, and solves the problem of insufficient parathyroid gland identification accuracy.
[0006] (II) Technical solution
[0007] In order to solve the above-mentioned technical problem of insufficient parathyroid gland recognition accuracy, the present invention provides the following technical solutions:
[0008] The present invention discloses a parathyroid gland spatial position positioning method by integrating imaging and Raman spectroscopy, which specifically comprises the following steps:
[0009] S1. Based on the principle of spectral detection, a laser probe is used to collect data from the parathyroid model area through three optical fiber probes to obtain the collected Raman spectrum data and image data;
[0010] The three optical fibers of the laser probe include: a cold light source optical fiber, a laser optical fiber, and a camera optical fiber;
[0011] The three optical fibers of the laser light guide, the camera light guide, and the cold light source light guide are arranged in the same laser probe in a concentric circle manner;
[0012] S2, preprocessing the collected Raman spectrum data to obtain preprocessed Raman spectrum data;
[0013] S3, preprocessing the collected image data to obtain preprocessed image data;
[0014] S4, fitting the preprocessed Raman spectrum data and the preprocessed image data to obtain fitted combined image data;
[0015] S5. Based on the obtained fitted combined image data, the parathyroid model is accurately positioned by a spatial positioning method.
[0016] The present invention adopts the spectrum detection principle, uses a laser probe to collect data on the parathyroid model area through three optical fiber probes, obtains collected Raman spectrum data and image data, and simultaneously obtains preprocessed Raman spectrum data and preprocessed image data by step-by-step processing of the collected Raman spectrum data and image data. At the same time, the preprocessed Raman spectrum data and the preprocessed image data are fitted by an image fitting method, and the parathyroid model is accurately positioned by a spatial positioning method, thereby ensuring the real-time and accuracy of the spatial positioning of the parathyroid gland.
[0017] Preferably, the preprocessing of the collected Raman spectrum data comprises the following steps:
[0018] S21, Raman spectrum data normalization processing;
[0019] The Raman spectrum data was normalized by the maximum and minimum normalization method;
[0020] The maximum and minimum normalization method conversion function is as follows:
[0021] ;
[0022] in, Represents the minimum value of the collected Raman spectrum data, Represents the maximum value of the collected Raman spectrum data, represents the collected Raman spectrum data, Indicates the maximum and minimum normalized Raman spectral data;
[0023] S22, Raman spectrum data filtering processing;
[0024] The Raman spectral data after filtering is set as the pre-processed Raman spectral data.
[0025] Preferably, the Raman spectrum data filtering processing step comprises the following steps:
[0026] The Raman spectrum data is filtered by a filtering algorithm;
[0027] Set the window composed of the collected Raman spectrum data to , Indicates Raman spectrum data value of a window;
[0028] ;
[0029] in, Indicates the window size, Indicates Raman spectrum data of windows, Represents the filtered Raman spectrum data of windows, Indicates the set spectral signal intensity, Indicates the intensity of the spectral signal collected per unit time;
[0030] Set the signal-to-noise ratio as the noise evaluation index standard;
[0031] The signal-to-noise ratio formula is as follows:
[0032] ;
[0033] in, represents the signal-to-noise ratio, represents the filtered Raman spectral data, represents the Raman spectrum data after maximum and minimum normalization, Represents the first Raman spectrum data after maximum and minimum normalization, Indicates the maximum and minimum normalized Raman spectroscopy data, It represents the Raman spectrum data without filtering after acquisition;
[0034] The larger the signal-to-noise ratio is set, the stronger the noise suppression effect is;
[0035] Set the signal-to-noise ratio threshold. When the signal-to-noise ratio calculated from the filtered Raman spectrum data is less than the set signal-to-noise ratio threshold, it means that the filtering result is unqualified and filtering is performed again. When the signal-to-noise ratio calculated from the filtered Raman spectrum data is greater than or equal to the set signal-to-noise ratio threshold, it means that the filtering result is qualified.
[0036] The present invention processes the collected Raman spectrum data by using a data normalization method and a Savitzky-Golay filtering method, and simultaneously establishes a Raman spectrum data evaluation index by setting a signal-to-noise ratio threshold, thereby improving the accuracy of the Raman spectrum data.
[0037] Preferably, the preprocessing of the collected image data comprises the following steps:
[0038] S31, representing each pixel in the collected image data in RGB format;
[0039] S32, performing image enhancement on the collected image data by using a histogram equalization algorithm;
[0040] The output image after histogram equalization is set as the preprocessed image data.
[0041] Preferably, the image enhancement of the collected image data by using a histogram equalization algorithm comprises the following steps:
[0042] S321, based on the gray value distribution in the collected image data, calculating the pixel probability distribution of the collected image data, the gray value being the value of the pixel point satisfying the condition R=G=B;
[0043] The pixel probability distribution formula of the collected image data is as follows:
[0044] ;
[0045] in, represents the pixel probability of the acquired image data, Indicates the gray value is The number of pixels, is the total number of pixels;
[0046] S322, constructing an image cumulative distribution function according to the calculated pixel probability distribution of the acquired image data;
[0047] The formula for constructing the cumulative distribution function of the image is as follows:
[0048] ;
[0049] Where L represents the total number of gray values in the image, Indicates The gray value after the gray value is mapped by the cumulative distribution function;
[0050] S323, performing image enhancement on the collected image data based on the constructed image cumulative distribution function;
[0051] Multiply the gray value mapped by the cumulative distribution function by (L-1) through the transformation function;
[0052] The transformation function formula is as follows:
[0053] ;
[0054] in, Indicates about The transformation function is used to make the gray level of the equalized image consistent with the original image before normalization;
[0055] According to the mapping relationship between the image cumulative distribution function and the transformation function, the image after histogram equalization is output with reference to the pixels in the original image.
[0056] The present invention performs image enhancement on the collected image data by using a histogram equalization algorithm, performs image enhancement on the collected image data based on the grayscale value distribution in the collected image data and a constructed image cumulative distribution function, thereby ensuring the accuracy of the image data.
[0057] Preferably, the fitting of the preprocessed Raman spectrum data and the preprocessed image data comprises the following steps:
[0058] S41, dividing the preprocessed Raman spectrum data and the preprocessed image data into data sub-blocks of equal size;
[0059] S42, in each data sub-block, calculating the absolute value of the grayscale difference of the central pixel in the four directions of up, down, left, and right, to obtain the grayscale difference in the corresponding directions;
[0060] The grayscale difference calculation formula is as follows:
[0061] ;
[0062] in, represents grayscale difference, Represents the gray value of the center pixel, Represents the gray value in the corresponding direction;
[0063] S43, arranging the grayscale difference calculation results in each direction and setting the threshold Determine the primary selection point;
[0064] Select the grayscale difference mean greater than the threshold The central pixel point of is taken as the initial selection point;
[0065] S44. In the set of preliminary points, select a local window with the center as the preliminary point, and calculate the covariance matrix and error grayscale respectively. , and set the grayscale limit Identify alternative locations;
[0066] S45. Calculate the weights among the candidate points , select the point with the maximum weight as the feature point;
[0067] The weight calculation formula is as follows:
[0068] ;
[0069] in, represents the calculated weight, Represents the covariance matrix The determinant of Represents the covariance matrix traces;
[0070] S46, looping step S41 to step S45 until each area is processed;
[0071] S47, re-measuring Raman spectrum data of feature points in the pre-processed image data, and pre-processing the re-measured Raman spectrum data to obtain pre-processed Raman spectrum data;
[0072] S48, performing fitting based on the preprocessed Raman spectrum data and the feature points in the preprocessed image data by a direct averaging algorithm to obtain fitted combined image data;
[0073] The direct averaging algorithm formula is as follows:
[0074] ;
[0075] in, represents the calculated average value, Represents the first feature points, Indicates The Raman spectrum data corresponding to the feature points, Indicates the number of feature points;
[0076] The feature points in the preprocessed image data and the preprocessed Raman spectrum data of the corresponding feature points are matched one by one, and the average value calculated by the direct average algorithm is used as the Raman spectrum data of the combined image data.
[0077] The feature points in the preprocessed image data and the preprocessed Raman spectrum data of the corresponding feature points are matched one by one, and the average value calculated by the direct average algorithm is used as the Raman spectrum data of the combined image data.
[0078] Preferably, the covariance matrix and the roundness of the error ellipse are calculated separately. The following steps are involved:
[0079] The variance matrix is represented as follows:
[0080] ;
[0081] in, represents the covariance matrix, Indicates the grayscale difference above the center pixel. Indicates the grayscale difference below the center pixel. Indicates the grayscale difference to the left of the center pixel. Indicates the grayscale difference corresponding to the right of the central pixel;
[0082] The roundness calculation formula of the error ellipse is as follows:
[0083] ;
[0084] in, Indicates the calculated circularity.
[0085] The present invention improves the accuracy of combined image data by dividing the preprocessed Raman spectrum data and the preprocessed image data, calculating the image grayscale and selecting feature points, and fitting the preprocessed Raman spectrum data and the preprocessed image data through a direct averaging algorithm.
[0086] Preferably, the precise positioning by a spatial positioning method based on the obtained fitted combined image data comprises the following steps:
[0087] Setting the coordinates of the center position of the parathyroid model area and selecting the feature points of the multi-frame combined image data for real-time fitting;
[0088] By comparing the feature points of the combined image data of the current frame with the feature points of the previous frame, if the overlap is higher than 80%, it is used as a basis to determine whether they are the same parathyroid model;
[0089] After confirming that it is the same parathyroid model, the center position coordinates of the current laser probe are connected with the center position coordinates of the set parathyroid model area, and the center position coordinates of the laser probe are adjusted so that the center position coordinates of the current laser probe coincide with the center position coordinates of the set parathyroid model area, thereby completing the precise positioning of the parathyroid model.
[0090] The present invention achieves accurate positioning of the parathyroid model by selecting feature points in multiple frame images through multi-frame comparison, thereby improving the accuracy of parathyroid spatial positioning.
[0091] The present invention also discloses a parathyroid gland spatial position positioning system, comprising: a laser probe, a Raman spectrum data processing module, an image data processing module, a data fitting module and a spatial positioning module;
[0092] The laser probe includes a cold light source optical fiber, a laser optical fiber and a camera optical fiber;
[0093] The cold light source light guide optical fiber is used to generate a cold light source light guide beam and provide a light source;
[0094] The laser light guide optical fiber is used to generate an imaging light guide beam and collect image data;
[0095] The laser light guide optical fiber is used to generate a laser light guide beam and collect Raman spectrum data;
[0096] The Raman spectrum data processing module is used to process the collected Raman spectrum data;
[0097] The image data processing module is used to process the collected image data;
[0098] The data fitting module is used to fit the processed image data and Raman spectrum data;
[0099] The spatial positioning module is used to perform positioning according to the fitted image data.
[0100] Preferably, the present invention can also use a photoacoustic imaging device to replace the above-mentioned laser light-guiding optical fiber to collect image data, thereby obtaining a parathyroid gland spatial position positioning system that integrates imaging and Raman spectroscopy.
[0101] Preferably, the present invention may also use autofluorescence imaging technology or laser speckle technology to obtain image data.
[0102] Preferably, the present invention may also use an endoscope to collect image data.
[0103] (III) Beneficial effects
[0104] Compared with the prior art, the present invention provides a parathyroid gland spatial location method that integrates imaging and Raman spectroscopy, which has the following beneficial effects:
[0105] 1. The invention adopts the spectral detection principle and uses a laser probe to collect data on the parathyroid model area through three optical fiber probes to obtain the collected Raman spectrum data and image data. At the same time, the collected Raman spectrum data and image data are processed step by step to obtain preprocessed Raman spectrum data and preprocessed image data. At the same time, the preprocessed Raman spectrum data and preprocessed image data are fitted by image fitting, and the parathyroid model is accurately positioned by a spatial positioning method, thereby ensuring the real-time and accuracy of the spatial positioning of the parathyroid gland.
[0106] 2. The invention processes the collected Raman spectral data by using a data normalization method and a Savitzky-Golay filtering method, and establishes a Raman spectral data evaluation index by setting a signal-to-noise ratio threshold, thereby improving the accuracy of the Raman spectral data.
[0107] 3. The invention performs image enhancement on the collected image data by using a histogram equalization algorithm, performs image enhancement on the collected image data based on the grayscale value distribution in the collected image data and a constructed image cumulative distribution function, thereby ensuring the accuracy of the image data.
[0108] 4. The invention improves the accuracy of the combined image data by dividing the preprocessed Raman spectrum data and the preprocessed image data, calculating the image grayscale and selecting feature points, and fitting the preprocessed Raman spectrum data and the preprocessed image data through a direct averaging algorithm.
[0109] 5. The invention uses a multi-frame comparison method to select feature points in multiple frame images to complete the precise positioning of the parathyroid model, thereby improving the accuracy of the spatial positioning of the parathyroid gland. BRIEF DESCRIPTION OF THE DRAWINGS
[0110] Figure 1 The figure is a schematic diagram of the spatial location flow of the parathyroid gland using fusion imaging and Raman spectroscopy of the present invention. DETAILED DESCRIPTION
[0111] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0112] Example 1
[0113] This embodiment discloses a method for spatially locating the parathyroid gland by integrating imaging and Raman spectroscopy, which specifically includes the following steps:
[0114] S1. Based on the principle of spectral detection, a laser probe is used to collect data from the parathyroid model area through three optical fiber probes to obtain the collected Raman spectrum data and image data;
[0115] The three optical fibers of the laser probe include: a cold light source optical fiber, a laser optical fiber, and a camera optical fiber;
[0116] The three optical fibers of the laser light guide, the camera light guide, and the cold light source light guide are arranged in the same laser probe in a concentric circle manner;
[0117] S2, preprocessing the collected Raman spectrum data to obtain preprocessed Raman spectrum data;
[0118] The preprocessing of the collected Raman spectral data includes the following steps:
[0119] S21, Raman spectrum data normalization processing;
[0120] The Raman spectrum data was normalized by the maximum and minimum normalization method;
[0121] The maximum and minimum normalization method conversion function is as follows:
[0122] ;
[0123] in, Represents the minimum value of the collected Raman spectrum data, Represents the maximum value of the collected Raman spectrum data, represents the collected Raman spectrum data, Indicates the maximum and minimum normalized Raman spectral data;
[0124] S22, Raman spectrum data filtering processing;
[0125] The Raman spectrum data is filtered by a filtering algorithm;
[0126] Set the window composed of the collected Raman spectrum data to , Indicates Raman spectrum data value of a window;
[0127] ;
[0128] in, Indicates the window size, Indicates Raman spectrum data of windows, Represents the filtered Raman spectrum data of windows, Indicates the set spectral signal intensity, Indicates the intensity of the spectral signal collected per unit time;
[0129] Set the signal-to-noise ratio as the noise evaluation index standard;
[0130] The signal-to-noise ratio formula is as follows:
[0131] ;
[0132] in, represents the signal-to-noise ratio, represents the filtered Raman spectral data, represents the Raman spectrum data after maximum and minimum normalization, Represents the first Raman spectrum data after maximum and minimum normalization, Indicates the maximum and minimum normalized Raman spectroscopy data, It represents the Raman spectrum data without filtering after acquisition;
[0133] Furthermore, the larger the signal-to-noise ratio is set, the stronger the noise suppression effect is;
[0134] Furthermore, a signal-to-noise ratio threshold is set. When the signal-to-noise ratio calculated from the filtered Raman spectrum data is less than the set signal-to-noise ratio threshold, it indicates that the filtering result is unqualified, and filtering is performed again. When the signal-to-noise ratio calculated from the filtered Raman spectrum data is greater than or equal to the set signal-to-noise ratio threshold, it indicates that the filtering result is qualified.
[0135] The filtered Raman spectrum data is set as the pre-processed Raman spectrum data;
[0136] S3, preprocessing the collected image data to obtain preprocessed image data;
[0137] The preprocessing of the acquired image data includes the following steps:
[0138] S31, representing each pixel in the collected image data in RGB format;
[0139] S32, performing image enhancement on the collected image data by using a histogram equalization algorithm;
[0140] Image enhancement of the acquired image data by using the histogram equalization algorithm includes the following steps:
[0141] S321, based on the gray value distribution in the collected image data, calculating the pixel probability distribution of the collected image data, the gray value being the value of the pixel point satisfying the condition R=G=B;
[0142] The pixel probability distribution formula of the collected image data is as follows:
[0143] ;
[0144] in, represents the pixel probability of the acquired image data, Indicates the gray value is The number of pixels, is the total number of pixels;
[0145] S322, constructing an image cumulative distribution function according to the calculated pixel probability distribution of the acquired image data;
[0146] The formula for constructing the cumulative distribution function of the image is as follows:
[0147] ;
[0148] Where L represents the total number of gray values in the image, Indicates The gray value after the gray value is mapped by the cumulative distribution function;
[0149] S323, performing image enhancement on the collected image data based on the constructed image cumulative distribution function;
[0150] Multiply the gray value mapped by the cumulative distribution function by (L-1) through the transformation function;
[0151] The transformation function formula is as follows:
[0152] ;
[0153] in, Indicates about The transformation function is used to make the gray level of the equalized image consistent with the original image before normalization;
[0154] According to the mapping relationship between the image cumulative distribution function and the transformation function, refer to the pixels in the original image and output the image after histogram equalization;
[0155] Set the output histogram equalized image as the preprocessed image data;
[0156] S4, fitting the preprocessed Raman spectrum data and the preprocessed image data to obtain fitted combined image data;
[0157] Fitting the preprocessed Raman spectrum data and the preprocessed image data comprises the following steps:
[0158] S41, dividing the preprocessed Raman spectrum data and the preprocessed image data into data sub-blocks of equal size;
[0159] S42, in each data sub-block, calculating the absolute value of the grayscale difference of the central pixel in the four directions of up, down, left, and right, to obtain the grayscale difference in the corresponding directions;
[0160] The grayscale difference calculation formula is as follows:
[0161] ;
[0162] in, represents grayscale difference, Represents the gray value of the center pixel, Represents the gray value in the corresponding direction;
[0163] S43, arranging the grayscale difference calculation results in each direction and setting the threshold Determine the primary selection point;
[0164] Select the grayscale difference mean greater than the threshold The central pixel point of is taken as the initial selection point;
[0165] S44. In the set of preliminary points, select a local window with the center as the preliminary point, and calculate the covariance matrix and error grayscale respectively. , and set the grayscale limit Identify alternative locations;
[0166] The covariance matrix is represented as follows:
[0167] ;
[0168] in, represents the covariance matrix, Indicates the grayscale difference above the center pixel. Indicates the grayscale difference below the center pixel. Indicates the grayscale difference to the left of the center pixel. Indicates the grayscale difference corresponding to the right of the central pixel;
[0169] The roundness calculation formula of the error ellipse is as follows:
[0170] ;
[0171] in, represents the calculated roundness, Represents the covariance matrix The determinant of Represents the covariance matrix traces;
[0172] S45. Calculate the weights among the candidate points , select the point with the maximum weight as the feature point;
[0173] The weight calculation formula is as follows:
[0174] ;
[0175] in, Represents the calculated weight;
[0176] S46, looping step S41 to step S45 until each area is processed;
[0177] S47, re-measuring Raman spectrum data of feature points in the pre-processed image data, and pre-processing the re-measured Raman spectrum data to obtain pre-processed Raman spectrum data;
[0178] S48, performing fitting based on the preprocessed Raman spectrum data and the feature points in the preprocessed image data by a direct averaging algorithm to obtain fitted combined image data;
[0179] The direct averaging algorithm formula is as follows:
[0180] ;
[0181] in, represents the calculated average value, Represents the first feature points, Indicates The Raman spectrum data corresponding to the feature points, Indicates the number of feature points;
[0182] The feature points in the preprocessed image data are matched one by one with the preprocessed Raman spectrum data of the corresponding feature points, and the average value calculated by the direct average algorithm is used as the Raman spectrum data of the combined image data;
[0183] The preprocessed Raman spectrum data and the preprocessed image data are combined into data blocks, and the coincident feature points in the combined data blocks are selected to calculate the grayscale values of the coincident feature points, and then the grayscale values of the coincident feature points in the preprocessed Raman spectrum data and the preprocessed image data are added to calculate the average value;
[0184] The average value of the calculated grayscale values of the coincident feature points is used as the grayscale value of the combined image data after fitting;
[0185] S5. Based on the obtained fitted combined image data, the parathyroid model is accurately positioned by a spatial positioning method;
[0186] Based on the obtained fitted combined image data, accurate positioning by spatial positioning method includes the following steps:
[0187] Setting the coordinates of the center position of the parathyroid model area and selecting the feature points of the multi-frame combined image data for real-time fitting;
[0188] By comparing the feature points of the combined image data of the current frame with the feature points of the previous frame, if the overlap is higher than 80%, it is used as a basis to determine whether they are the same parathyroid model;
[0189] Furthermore, after confirming that it is the same parathyroid model, the center position coordinates of the current laser probe are connected with the center position coordinates of the set parathyroid model area, and the center position coordinates of the laser probe are adjusted so that the center position coordinates of the current laser probe coincide with the center position coordinates of the set parathyroid model area, thereby completing the precise positioning of the parathyroid model.
[0190] Example 2
[0191] This embodiment also discloses a parathyroid gland spatial position positioning system integrating imaging and Raman spectroscopy, comprising: a laser probe, a Raman spectroscopy data processing module, an image data processing module, a data fitting module and a spatial positioning module;
[0192] The laser probe includes a cold light source optical fiber, a laser optical fiber and a camera optical fiber;
[0193] The cold light source light guide optical fiber is used to generate a cold light source light guide beam and provide a light source;
[0194] The laser light guide optical fiber is used to generate an imaging light guide beam and collect image data;
[0195] Furthermore, this embodiment may also use autofluorescence imaging technology or photoacoustic imaging technology or laser speckle technology or endoscope to obtain image data.
[0196] The laser light guide optical fiber is used to generate a laser light guide beam and collect Raman spectrum data;
[0197] The Raman spectrum data processing module is used to process the collected Raman spectrum data;
[0198] The image data processing module is used to process the collected image data;
[0199] The data fitting module is used to fit the processed image data and Raman spectrum data;
[0200] The spatial positioning module is used to perform positioning according to the fitted image data.
[0201] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for spatial positioning of parathyroid glands by fusion imaging and Raman spectroscopy, characterized in that: The following steps are involved: S1. Based on the principle of spectral detection, a laser probe is used to collect data from the parathyroid model area through three optical fiber probes to obtain the collected Raman spectrum data and image data; S2, preprocessing the collected Raman spectrum data to obtain preprocessed Raman spectrum data; S3, preprocessing the collected image data to obtain preprocessed image data; S4, fitting the preprocessed Raman spectrum data and the preprocessed image data to obtain fitted combined image data; S5. Based on the obtained fitted combined image data, the parathyroid model is accurately positioned by a spatial positioning method; The fitting of the pre-processed Raman spectrum data and the pre-processed image data comprises the following steps: S41, dividing the pre-processed image data into data sub-blocks of equal size; S42, in each data sub-block, calculating the absolute value of the grayscale difference of the central pixel in the four directions of up, down, left, and right, to obtain the grayscale difference in the corresponding directions; The grayscale difference calculation formula is as follows: ; in, represents grayscale difference, Represents the gray value of the center pixel, Represents the grayscale value in the corresponding direction; S43, arranging the grayscale difference calculation results in each direction and setting the threshold Determine the primary selection point; Select the grayscale difference mean greater than the threshold The central pixel point of is taken as the initial selection point; S44. In the set of preliminary points, select a local window with the center as the preliminary point, and calculate the covariance matrix and error grayscale respectively. , and set the grayscale limit Identify alternative locations; S45. Calculate the weights among the candidate points , select the maximum weight point as the feature point; The weight calculation formula is as follows: ; in, represents the calculated weight, Represents the covariance matrix The determinant of Represents the covariance matrix traces; S46, looping step S41 to step S45 until each area is processed; S47, re-measuring Raman spectrum data of feature points in the pre-processed image data, and pre-processing the re-measured Raman spectrum data to obtain pre-processed Raman spectrum data; S48, performing fitting based on the preprocessed Raman spectrum data and the feature points in the preprocessed image data by a direct averaging algorithm to obtain fitted combined image data; The direct averaging algorithm formula is as follows: ; in, represents the calculated average value, Represents the first feature points, Indicates The Raman spectrum data corresponding to the feature points, Indicates the number of feature points; The feature points in the preprocessed image data are matched one by one with the preprocessed Raman spectrum data of the corresponding feature points, and the average value calculated by the direct average algorithm is used as the Raman spectrum data of the combined image data; The preprocessing of the collected Raman spectrum data comprises the following steps: S21, Raman spectrum data normalization processing; The Raman spectrum data was normalized by the maximum and minimum normalization method; The maximum and minimum normalization method conversion function is as follows: ; in, Represents the minimum value of the collected Raman spectrum data, Represents the maximum value of the collected Raman spectrum data, represents the collected Raman spectrum data, Indicates the maximum and minimum normalized Raman spectral data; S22, Raman spectrum data filtering processing; The filtered Raman spectrum data is set as the pre-processed Raman spectrum data; The preprocessing of the collected image data comprises the following steps: S31, representing each pixel in the collected image data in RGB format; S32, performing image enhancement on the collected image data by using a histogram equalization algorithm; Set the output histogram equalized image as the preprocessed image data; The precise positioning based on the obtained fitted combined image data by a spatial positioning method comprises the following steps: Setting the coordinates of the center position of the parathyroid model area and selecting the feature points of the multi-frame combined image data for real-time fitting; By comparing the feature points of the combined image data of the current frame with the feature points of the previous frame, if the overlap is higher than 80%, it is used as a basis to determine whether they are the same parathyroid model; After confirming that it is the same parathyroid model, the center position coordinates of the current laser probe are connected with the center position coordinates of the set parathyroid model area, and the center position coordinates of the laser probe are adjusted so that the center position coordinates of the current laser probe coincide with the center position coordinates of the set parathyroid model area, thereby completing the precise positioning of the parathyroid model.
2. The parathyroid gland spatial location positioning method based on fusion imaging and Raman spectroscopy according to claim 1, characterized in that: The Raman spectroscopy data filtering process comprises the following steps: The Raman spectrum data is filtered by a filtering algorithm; Set the window composed of the collected Raman spectrum data to , Indicates Raman spectrum data value of a window; ; in, Indicates the window size, Indicates Raman spectrum data of windows, Represents the filtered Raman spectrum data of windows, Indicates the set spectral signal intensity, Indicates the intensity of the spectral signal collected per unit time; Set the signal-to-noise ratio as the noise evaluation index standard; The signal-to-noise ratio formula is as follows: ; in, represents the signal-to-noise ratio, represents the filtered Raman spectral data, represents the Raman spectrum data after maximum and minimum normalization, Represents the first Raman spectrum data after maximum and minimum normalization, Indicates the maximum and minimum normalized Raman spectroscopy data, It represents the Raman spectrum data without filtering after acquisition; The larger the signal-to-noise ratio is set, the stronger the noise suppression effect is; Set the signal-to-noise ratio threshold. When the signal-to-noise ratio calculated from the filtered Raman spectrum data is less than the set signal-to-noise ratio threshold, it means that the filtering result is unqualified and filtering is performed again. When the signal-to-noise ratio calculated from the filtered Raman spectrum data is greater than or equal to the set signal-to-noise ratio threshold, it means that the filtering result is qualified.
3. The parathyroid gland spatial location method based on fusion imaging and Raman spectroscopy according to claim 1, characterized in that: The method of performing image enhancement on the collected image data by using a histogram equalization algorithm comprises the following steps: S321, based on the gray value distribution in the collected image data, calculating the pixel probability distribution of the collected image data, the gray value being the value of the pixel point satisfying the condition R=G=B; The pixel probability distribution formula of the collected image data is as follows: ; in, represents the pixel probability of the acquired image data, Indicates the gray value is The number of pixels, is the total number of pixels; S322, constructing an image cumulative distribution function according to the calculated pixel probability distribution of the acquired image data; The formula for constructing the cumulative distribution function of the image is as follows: ; Where L represents the total number of gray values in the image. Indicates The gray value after the gray value is mapped by the cumulative distribution function; S323, performing image enhancement on the collected image data based on the constructed image cumulative distribution function; Multiply the gray value mapped by the cumulative distribution function by (L-1) through the transformation function; The transformation function formula is as follows: ; in, Indicates about The transformation function is used to make the gray level of the equalized image consistent with the original image before normalization; According to the mapping relationship between the image cumulative distribution function and the transformation function, the image after histogram equalization is output with reference to the pixels in the original image.
4. The method for spatial positioning of parathyroid glands by fusion imaging and Raman spectroscopy according to claim 1, characterized in that: The covariance matrix and the circularity of the error ellipse are calculated separately The following steps are involved: The covariance matrix is represented as follows: ; in, represents the covariance matrix, Indicates the grayscale difference above the center pixel. Indicates the grayscale difference below the center pixel. Indicates the grayscale difference to the left of the center pixel. Indicates the grayscale difference corresponding to the right of the central pixel; The roundness calculation formula of the error ellipse is as follows: ; in, Indicates the calculated circularity.
5. A parathyroid gland spatial location positioning system using fusion imaging and Raman spectroscopy to implement the method described in any one of claims 1 to 4, characterized in that: It includes a laser probe, a Raman spectrum data processing module, an image data processing module, a data fitting module and a space positioning module; The laser probe includes a cold light source optical fiber, a laser optical fiber and a camera optical fiber; The cold light source light guide optical fiber is used to generate a cold light source light guide beam and provide a light source; The imaging light guide optical fiber is used to generate an imaging light guide beam and collect image data; The laser light guide optical fiber is used to generate a laser light guide beam and collect Raman spectrum data; The Raman spectrum data processing module is used to process the collected Raman spectrum data; The image data processing module is used to process the collected image data; The data fitting module is used to fit the processed image data and Raman spectrum data; The spatial positioning module is used to perform positioning according to the fitted image data.
Citation Information
Patent Citations
Device and method for recognizing parathyroid gland position and evaluating parathyroid gland survival rate
CN114343568A
Image positioning precision measuring method
CN115035049A
Near-infrared light real-time recognition parathyroid gland endoscope system
CN214632077U
Device for recognizing position of parathyroid gland and evaluating survival rate of parathyroid gland
CN215305781U
Thyroid surgery endoscope device based on fluorescence lifetime and Raman spectrum imaging
CN114732448A
