A method and system for extracting tissue optical parameters based on long short-term memory network

Abstract

A method and system for extracting tissue optical parameters based on a long short-term memory network belongs to the technical field of the intersection of machine learning and biomedicine, and solves the problem of inaccurate extraction of tissue optical parameters caused by only utilizing part of the diffuse reflection light information and ignoring the correlation of spectral data sequence characteristics when extracting tissue optical parameters. The present invention introduces a long short-term memory network into a tissue spatial resolution diffuse reflection spectrum generation model, fully explores the correlation in spectral data points, and increases the model prediction ability while reducing model parameters. When training the tissue spatial resolution diffuse reflection spectrum generation model, the optical fiber probe structure is not considered, and all spatial resolution diffuse reflection spectra emitted from the tissue surface are included in a pre-data set, the data dimension is larger, and the model accuracy is higher after training, which effectively improves the accuracy of the tissue optical parameter and diffuse reflection spectrum mapping model, thereby improving the efficiency and accuracy of tissue optical parameter extraction.

Description

Technical Field

[0001] The present invention belongs to the technical field of intersection of machine learning and biomedicine, and relates to a method and system for extracting tissue optical parameters based on a long-short term memory (LSTM) network. Background Art

[0002] The changes in the optical properties of human skin tissue (including absorption coefficient and scattering coefficient) are closely related to the physiological state of the tissue and the internal microenvironment of the tissue, which provides a basis for clinical applications such as tissue morphology research, detection of diseases such as tumors, dynamic monitoring of metabolism, and photodynamic therapy. The tissue diffuse reflectance spectrum originates from the backscattered light modulated by the tissue, carries the information of the optical properties of the tissue, and can be used to extract the absorption coefficient and scattering coefficient of the tissue. The measurement of tissue diffuse reflectance spectrum usually uses a fiber optic probe with a specific structure, and then measures the spatially resolved diffuse reflectance spectrum using different illumination-detection radial distances, or measures the diffuse reflectance spectrum at different wavelengths under a single light source-detection radial distance, or measures the spatial frequency domain diffuse reflectance spectrum. In order to extract tissue optical parameters from the actual measured tissue diffuse reflectance spectrum, it is first necessary to determine the forward model from the tissue optical parameters to the diffuse reflectance spectrum. At present, the main methods for determining the forward model are the slow-beam approximation method and the Monte Carlo method. The diffuse approximation method is the first-order spherical harmonic expansion of the radiation transfer equation, which can be used to analyze the diffuse reflectance spectrum under specific tissue optical parameters. However, this method requires that the medium scattering coefficient is much larger than the absorption coefficient, and the illumination-detection radial distance is large enough, which is not completely true for human tissue. The Monte Carlo method can simulate photon transport in tissues with arbitrary optical parameters and geometric shapes, and is considered the gold standard for photon transport simulation. However, as the modeling accuracy increases, the operation time of the Monte Carlo method will also increase. In order to solve the limitation of Monte Carlo simulation consuming computing resources, methods such as hybrid Monte Carlo method, ratio-scaling Monte Carlo method, and graphics card accelerated Monte Carlo method have been developed.Among them, the hybrid Monte Carlo method uses the characteristic that the diffuse approximation has a large error at a position with a small radial distance between illumination and detection, and uses Monte Carlo simulation to improve accuracy at the proximal end of the illumination source, and uses diffuse approximation to improve efficiency at the distal end of the illumination source (see the literature Hayashi T, et al. Hybrid Monte Carlo-diffusion method for light propagation in tissue with a low-scattering region [J]. and the literature Proceedings of SPIE-The International Society for Optical Engineering, 2001, 42 (16): 2888-96. and the literature Zhu C, Liu Q. Hybrid method for fast Monte Carlo simulation of diffuse reflectance from a multilayered tissue model with tumor-like heterogeneities [J]. Journal of Biomedical Optics, 2012, 17 (1): 010501.), but it is difficult to determine the division standard between the proximal end and the distal end. The ratio scaling Monte Carlo method assumes that the path of photons in the tissue is determined by the scattering coefficient, and the weight of photons overflowing the tissue surface is determined by the absorption coefficient. Under specific tissue structure and optical parameters, the standard Monte Carlo method is first used for photon tracing to record the weight, position and number of photons that overflow the tissue surface and the number of interactions with the tissue. If the method needs to be extended to other scattering coefficients, the ratio of the overflow position is scaled; if it needs to be extended to other absorption coefficients, the overflow weight is redefined (see the literature Palmer GM, et al. Monte Carlo-based inverse model for calculating tissue optical properties. Part I: Theory and validation on synthetic phantoms. [J]. Applied Optics, 2006, 45 (5): 1062-71.). This method has a significant acceleration effect, but it is only applicable to homogeneous media, and the scaling process will inevitably introduce certain errors. Graphics card accelerated Monte Carlo method Considering the natural parallelism of the Monte Carlo method, it is transplanted to a graphics card with better computing power for large-scale parallel computing, and an acceleration effect of 2-3 orders of magnitude can be achieved with only one graphics card.In summary, considering the computational efficiency and model accuracy, the graphics card accelerated Monte Carlo method is more effective (see Lu B, Li J, et al. GPU-based Monte Carlo simulation for light propagation in complex heterogeneous tissues [J]. Optics Express, 2010, 18 (7): 6811.).

[0003] After determining the forward model of tissue optical parameters to diffuse reflectance spectrum, it is necessary to combine the iterative fitting algorithm to extract optical parameters. There are two main implementation methods. First, starting from tissue physiological parameters (such as melanin concentration, blood volume fraction, blood oxygen saturation, etc., see the literature Liu C, et al. Experimental validation of an inversefluorescence Monte Carlo model to extract concentrations of metabolicallyrelevant fluorophores from turbid phantoms and a murine tumor model. [J]. Journal of Biomedical Optics, 2012, 17 (7): 87-95.), first calculate the tissue optical parameters at different wavelengths, and then use the aforementioned forward model to calculate a certain illumination-detection radial distance, tissue diffuse reflectance spectrum at different wavelengths, and compare it with the actual measured tissue diffuse reflectance spectrum to calculate the variance of the two. After continuous iteration, the tissue physiological parameters corresponding to the minimum variance are output. Finally, the optical parameters of the tissue within the entire band are inverted. Secondly, for a specific wavelength, starting from the tissue optical parameters (see the literature Dongqing Peng, Li H. Study for noninvasive determination of optical properties of bio-tissue using spatially resolved diffuse reflectance [J]. Proc SPIE, 2012, 8553: 30.), the aforementioned mapping model is used to calculate the spatially resolved diffuse reflection light of the tissue under multiple illumination-detection radial distances, and the spatially resolved diffuse reflection light of the tissue is compared with the actual measured spatially resolved diffuse reflection light of the tissue and the variance is calculated. After continuous iteration, the tissue optical parameters corresponding to the minimum variance are output. Then the set wavelength is changed and the aforementioned process is repeated until the tissue optical parameters of the entire band range are output. Both of the above methods require a large number of iterative operations, which seriously affect the execution speed of the algorithm, and the performance of the optimization algorithm during the iteration process will directly affect the execution speed of the entire algorithm.

[0004] In view of the limitations of the iterative method, the main improvement plan at present is to further speed up the operation speed of the tissue diffuse reflectance spectrum forward model. The specific implementation method is to obtain a large-scale data set of tissue optical parameters and diffuse reflectance spectra through multiple executions of the forward model, and then use the table lookup method (see Zhu Dan et al., Skin physiological parameters and optical property parameter measurement method based on reflectance spectrum measurement, application number: 201010525672.6;) or train an artificial neural network (see Chenxi Li, et al. Artificial neural network method for determining optical properties from double integrating spheres measurements [J]. and CHINESEOPTICS LETTERS, 2010, 8 (2): 173-176. Tsui SY, et al. Modelling spatially-resolved diffuse reflectance spectra of a multi-layered skin model by artificial neural networks trained with Monte Carlo simulations. [J]. Biomedical Optics Express, 2018, 9 (4): 1531.) to establish a direct mapping model between tissue optical parameters and tissue diffuse reflectance light. In the subsequent iteration process, the table lookup method performs interpolation operations in the data set according to the tissue optical parameters to obtain the diffuse reflection light intensity. This method can greatly improve the operation speed of the forward model and thus speed up the efficiency of the iterative model.

[0005] Although there are many studies on the improvement of existing technologies, there are still several defects: (1) When establishing the tissue spatial resolution diffuse reflectance spectrum generation model, the fiber probe structure in the actual measurement system is incorporated. When performing spectral measurement, the fiber probe structure is not optimized. It is uncertain whether the fiber probe structure used is most conducive to optical parameter extraction. (2) The accuracy of the table lookup method is limited by the step size of the optical parameters in the data table and the table lookup interpolation algorithm. The smaller the parameter step size, the higher the accuracy, but it also brings the burden of data loading and directly affects the efficiency of the table lookup. In addition, the performance of the table lookup interpolation algorithm also directly affects the accuracy of the table lookup. (3) When establishing the tissue spatial resolution diffuse reflectance spectrum generation model based on artificial neural network, the amount of data directly affects the accuracy of the algorithm. The graphics card accelerated Monte Carlo method can generate enough data for deep learning algorithms to learn and ensure the accuracy of the model. However, current reports show that the artificial neural networks used are all shallow neural networks (the maximum number of layers currently disclosed in existing literature is 6 layers), with limited learning ability, and cannot mine the nonlinear mapping relationship between tissue optical parameters and diffuse reflectance spectra. However, if the network depth is simply increased, it will cause problems such as explosive growth in model parameters and complexity and vanishing gradients during the solution process.

[0006] In view of the defects of the prior art, the present invention takes the structural limitations of the fiber optic probe into consideration during the iterative optimization process. When establishing a tissue spatially resolved diffuse reflectance spectrum generation model, the present invention incorporates all spatially resolved diffuse reflectance light information escaping the tissue surface to improve the accuracy of the forward model; when processing data, the accuracy defects of the table lookup method and the sequence characteristics of the tissue diffuse reflectance spectrum are considered, the table lookup method and the traditional artificial neural network are abandoned, and the long short-term memory network in the recurrent neural network is used to process the spectral data, while fully mining the sequence characteristics of the spectral data and reducing the model parameters; when determining the actual fiber optic probe, the influence of different excitation fiber and collection fiber distances on the accuracy of optical parameter extraction is considered to determine the optimal combination of excitation fiber and collection fiber distances. Summary of the invention

[0007] The purpose of the present invention is to design a method and system for extracting tissue optical parameters based on long short-term memory network, so as to solve the problem of inaccurate extraction of tissue optical parameters caused by the prior art of extracting tissue optical parameters by only utilizing part of the diffuse reflected light information and ignoring the correlation of spectral data sequence characteristics.

[0008] The present invention solves the above technical problems through the following technical solutions:

[0009] A method for extracting tissue optical parameters based on long short-term memory network comprises the following steps:

[0010] S1. Determine the structural parameters and optical parameters of the multi-layer skin tissue optical model according to the tissue type to be tested;

[0011] S2. For the multi-layer skin tissue optical model described in step S1, a Monte Carlo method based on graphics card acceleration is used to simulate the spatially resolved diffuse reflectance spectra emitted from the tissue surface under different optical parameters, and a data set consisting of tissue optical parameters and tissue spatially resolved diffuse reflectance spectra is generated; at the same time, in order to make full use of diffuse reflectance spectrum information, the data set does not consider the limitation of the detection range of the optical fiber probe, and all spatially resolved diffuse reflectance spectra emitted from the tissue surface are included in the data set;

[0012] S3. Design a tissue spatially resolved diffuse reflectance spectrum generation model based on long short-term memory network, use tissue optical parameters and tissue spatially resolved diffuse reflectance spectrum datasets as training sets, and use error back propagation algorithm to optimize the generation model parameters;

[0013] S4. According to Lambert-Beer law and Mie scattering theory, the tissue optical parameters are calculated from the tissue physiological parameters. The model is generated by combining the distance between the illumination fiber and the collection fiber core in the fiber optic probe and the spatially resolved diffuse reflectance spectrum of the tissue. The tissue physiological parameters are extracted by least square fitting, and the tissue optical parameters are inverted.

[0014] S5. After determining the method for extracting tissue physiological parameters, the tissue physiological parameter extraction error is used as the cost function, and an iterative fitting method is used to optimize the core distance between the illumination fiber and the collection fiber in the optical fiber probe. Under the optimized core distance between the illumination fiber and the collection fiber, step S4 is executed to obtain the physiological and optical parameters of skin tissue.

[0015] The present invention overcomes the limitation of the prior art that, when extracting tissue optical parameters, only part of the diffuse reflection light information can be utilized and the sequence characteristics of spectral data are ignored. Taking into account the sequence characteristics of spatially resolved diffuse reflection light of tissues, a long short-term memory network is introduced to mine the correlation between spectral data, effectively improving the accuracy of the tissue optical parameter and diffuse reflection spectrum mapping model, thereby improving the efficiency and accuracy of tissue optical parameter extraction.

[0016] Furthermore, the structural parameters of the skin tissue optical model in step S1 include: the number of tissue layers m, the thickness of each layer d; the optical parameters of the skin tissue optical model include: the absorption coefficient μ of each layer of tissue a , scattering coefficient μ s , anisotropy coefficient g and refractive index n.

[0017] Furthermore, the long short-term memory network described in step S3 is a unidirectional network or a bidirectional network.

[0018] Furthermore, the specific implementation steps of combining the distance between the core of the illumination fiber and the collection fiber in the fiber optic probe and the spatially resolved diffuse reflectance spectrum of the tissue to generate a model, extracting the physiological parameters of the tissue by using least squares fitting, and inverting the optical parameters of the tissue as described in step S4 are as follows:

[0019] (1) Randomly initialize tissue physiological parameters;

[0020] (2) Calculate tissue absorption coefficient and scattering coefficient at different wavelengths based on Lambert-Beer law and Mie scattering theory;

[0021] (3) Inputting tissue absorption coefficient and scattering coefficient into the tissue spatially resolved diffuse reflectance spectrum generation model to calculate the spatially resolved diffuse reflectance spectrum emitted from the tissue surface at different wavelengths;

[0022] (4) Calculate the diffuse reflection light intensity of tissue detected by the fiber optic probe at different wavelengths according to the fiber optic probe structure;

[0023] (5) Calculate the error between the diffuse reflectance spectrum generated by the model and the measured diffuse reflectance spectrum, and correct the input physiological parameters according to the error;

[0024] (6) Repeating the processes (2) to (5) until the diffuse reflectance spectrum generated by the model coincides with the measured diffuse reflectance spectrum. At this time, the corresponding physiological parameters are the physiological parameters of the tissue to be measured.

[0025] (7) Substitute the extracted tissue physiological parameters into Lambert-Beer law and Mie scattering theory to invert tissue optical parameters.

[0026] Furthermore, the specific implementation steps of using the iterative fitting method to optimize the distance between the cores of the illumination fiber and the collection fiber in the optical fiber probe described in step S5 are as follows:

[0027] (a) Under typical physiological parameters of skin tissue, the tissue absorption coefficient and scattering coefficient at different wavelengths are calculated using the Lambert-Beer law and Mie scattering theory;

[0028] (b) Using the tissue spatially resolved diffuse reflectance spectrum generation model, the spatially resolved diffuse reflectance spectrum emitted from the tissue surface at different wavelengths is calculated;

[0029] (c) randomly initialize the distance between the illumination fiber and the collection fiber core in the fiber optic probe, and calculate the spectrally resolved diffuse reflectance spectrum emitted from the tissue surface;

[0030] (d) according to the skin tissue physiological parameter extraction method, using the spectrally resolved diffuse reflectance spectrum as input, inverting the tissue physiological parameters, and calculating the error between the parameters and the typical physiological parameters of the skin tissue; and correcting the distance between the illumination fiber and the collection fiber core according to the error size;

[0031] (e) Repeat (d) until the difference between the physiological parameters extracted by the model and the typical physiological parameters of skin tissue is minimal. At this time, the corresponding distance between the core of the illumination fiber and the collection fiber is the optimal distance.

[0032] A tissue optical parameter extraction system based on long short-term memory network, comprising: a model parameter determination module, a data set generation module, a model optimization module, a tissue optical parameter inversion module, and a tissue optical parameter optimization module;

[0033] The model parameter determination module is used to determine the structural parameters and optical parameters of the multi-layer skin tissue optical model according to the tissue type to be measured;

[0034] The data set generation module uses a Monte Carlo method based on graphics card acceleration to simulate the spatially resolved diffuse reflectance spectra emitted from the tissue surface under different optical parameters for the multi-layer skin tissue optical model described in the model parameter determination module, and generates a data set consisting of tissue optical parameters and tissue spatially resolved diffuse reflectance spectra; at the same time, in order to make full use of diffuse reflectance spectrum information, the data set does not consider the limitation of the detection range of the optical fiber probe, and includes all spatially resolved diffuse reflectance spectra emitted from the tissue surface into the data set;

[0035] The model optimization module is used to design a tissue spatially resolved diffuse reflectance spectrum generation model based on a long short-term memory network, using tissue optical parameters and tissue spatially resolved diffuse reflectance spectrum data sets as training sets, and using an error back propagation algorithm to optimize the generation model parameters;

[0036] The tissue optical parameter inversion module is used to calculate tissue optical parameters from tissue physiological parameters according to Lambert-Beer's law and Mie scattering theory, combine the distance between the illumination fiber and the collection fiber core in the fiber optic probe and the tissue spatially resolved diffuse reflectance spectrum to generate a model, use least squares fitting to extract tissue physiological parameters, and invert tissue optical parameters;

[0037] The tissue optical parameter optimization module is used to determine the tissue physiological parameter extraction method, use the tissue physiological parameter extraction error as the cost function, and use the iterative fitting method to optimize the core distance between the illumination fiber and the collection fiber in the optical fiber probe. Under the optimized core distance between the illumination fiber and the collection fiber, return to the tissue optical parameter inversion module to obtain the physiological and optical parameters of skin tissue.

[0038] Furthermore, the structural parameters of the skin tissue optical model in the model parameter determination module include: the number of tissue layers m, the thickness of each layer d; the optical parameters of the skin tissue optical model include: the absorption coefficient μ of each layer of tissue a , scattering coefficient μ s , anisotropy coefficient g and refractive index n.

[0039] Furthermore, the long short-term memory network described in the model optimization module is a unidirectional network or a bidirectional network.

[0040] Furthermore, the specific implementation steps of combining the distance between the core of the illumination fiber and the collection fiber in the fiber optic probe and the spatially resolved diffuse reflectance spectrum of the tissue to generate a model, extracting the physiological parameters of the tissue by using least squares fitting, and inverting the optical parameters of the tissue are as follows:

[0041] (1) Randomly initialize tissue physiological parameters;

[0042] (2) Calculate tissue absorption coefficient and scattering coefficient at different wavelengths based on Lambert-Beer law and Mie scattering theory;

[0043] (3) Inputting tissue absorption coefficient and scattering coefficient into the tissue spatially resolved diffuse reflectance spectrum generation model to calculate the spatially resolved diffuse reflectance spectrum emitted from the tissue surface at different wavelengths;

[0044] (4) Calculate the diffuse reflection light intensity of tissue detected by the fiber optic probe at different wavelengths according to the fiber optic probe structure;

[0045] (5) Calculate the error between the diffuse reflectance spectrum generated by the model and the measured diffuse reflectance spectrum, and correct the input physiological parameters according to the error;

[0046] (6) Repeating the processes (2) to (5) until the diffuse reflectance spectrum generated by the model coincides with the measured diffuse reflectance spectrum. At this time, the corresponding physiological parameters are the physiological parameters of the tissue to be measured.

[0047] (7) Substitute the extracted tissue physiological parameters into Lambert-Beer law and Mie scattering theory to invert tissue optical parameters.

[0048] Furthermore, the specific implementation steps of using the iterative fitting method to optimize the distance between the cores of the illumination fiber and the collection fiber in the optical fiber probe described in the tissue optical parameter optimization module are as follows:

[0049] (a) Under typical physiological parameters of skin tissue, the tissue absorption coefficient and scattering coefficient at different wavelengths are calculated using the Lambert-Beer law and Mie scattering theory;

[0050] (b) Using the tissue spatially resolved diffuse reflectance spectrum generation model, the spatially resolved diffuse reflectance spectrum emitted from the tissue surface at different wavelengths is calculated;

[0051] (c) randomly initialize the distance between the illumination fiber and the collection fiber core in the fiber optic probe, and calculate the spectrally resolved diffuse reflectance spectrum emitted from the tissue surface;

[0052] (d) according to the skin tissue physiological parameter extraction method, using the spectrally resolved diffuse reflectance spectrum as input, inverting the tissue physiological parameters, and calculating the error between the parameters and the typical physiological parameters of the skin tissue; and correcting the distance between the illumination fiber and the collection fiber core according to the error size;

[0053] (e) Repeat (d) until the difference between the physiological parameters extracted by the model and the typical physiological parameters of skin tissue is minimal. At this time, the corresponding distance between the core of the illumination fiber and the collection fiber is the optimal distance.

[0054] The advantages of the present invention are:

[0055] The present invention utilizes the characteristics of spectral data being similar to signals such as speech and text, and introduces the most effective long short-term memory network in the field of speech and text recognition into the tissue spatially resolved diffuse reflectance spectrum generation model, fully exploits the correlation in spectral data points, and increases the model prediction ability while reducing model parameters; when training the tissue spatially resolved diffuse reflectance spectrum generation model, the present invention does not consider the fiber probe structure, and includes all spatially resolved diffuse reflectance spectra emitted from the tissue surface into a pre-data set, so that the data dimension is larger and the model accuracy is higher after training; the present invention overcomes the limitation of the prior art that only part of the diffuse reflectance light information can be utilized and the sequence characteristics of the spectral data are ignored when extracting tissue optical parameters, and considers the sequence characteristics of the tissue spatially resolved diffuse reflectance light, introduces the long short-term memory network, exploits the correlation between the spectral data, and effectively improves the accuracy of the tissue optical parameter and diffuse reflectance spectrum mapping model, thereby improving the efficiency and accuracy of the tissue optical parameter extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a structural diagram of the method for extracting tissue physiological and optical parameters proposed in the present invention;

[0057] Figure 2 is the spatially resolved diffuse reflectance light emitted from the tissue surface under different absorption coefficients and scattering coefficients in the present invention; wherein the incident and emission distances in (a) to (f) are 0 mm, 0.2 mm, 0.4 mm, 1.6 mm, 1.8 mm and 2 mm respectively; in addition, μ a，epi is the absorption coefficient of the first layer of tissue; μ a，derm is the absorption coefficient of the second layer of tissue; μ s is the tissue scattering coefficient;

[0058] Figure 3 This is the tissue spatial resolution diffuse reflectance spectrum generation model based on long short-term memory network proposed in the present invention, where tanh represents the hyperbolic function, δ represents the indicator function, μ a，epi is the absorption coefficient of the first layer of tissue, μ a，derm is the absorption coefficient of the second layer of tissue, μ sis the tissue scattering coefficient, r The i-th diffuse reflection intensity generated for the model;

[0059] Figure 4 The variation of training error, validation error and test error with the number of iterations during the training process of the tissue spatial resolution diffuse reflectance spectrum generation model of the present invention;

[0060] Figure 5 This is a schematic diagram of the structure of a tissue diffuse reflectance spectrum measurement probe of the present invention;

[0061] Figure 6 The spatially resolved diffuse reflectance spectrum of tissue and the collection efficiency of the optical fiber probe under different incident and exit distances of the present invention;

[0062] Figure 7 The tissue diffuse reflectance spectra generated by the model are generated under different radial distances between the illumination optical fiber and the collection optical fiber of the present invention;

[0063] Figure 8 A tissue physiological and optical parameter extraction model for the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments:

[0066] Embodiment 1

[0067] like Figure 1 As shown, the method for extracting tissue physiological and optical parameters based on long-term and short-term networks according to the present invention is mainly divided into the following steps:

[0068] 1. Abstract skin tissue optical model, simplifying the skin tissue into a double-layer structure, where the upper layer absorption comes from melanin and the lower layer absorption comes from hemoglobin, that is, the absorption coefficients are independent of each other; the size and concentration of the scattering particles in the upper and lower layers are uniform, and the scattering coefficients are the same;

[0069] Assume that skin tissue is a double-layer structure consisting of the epidermis and dermis, where the absorption coefficient of the epidermis is determined by melanin and ranges from 2 to 100 cm -1 ; The absorption coefficient of the dermis is determined by hemoglobin and ranges from 2 to 50 cm -1 ; The scattering coefficient of the epidermis and dermis is the same, ranging from 2 to 100 cm -1 ; The refractive index n of the epidermis and dermis is 1.4, and the anisotropy coefficient g is 0.9. It is further assumed that the thickness of the epidermis is 0.07 cm and the dermis is optically infinitely thick.

[0070]

[0071] 2. Use the graphics card to accelerate the Monte Carlo method to generate tissue optical parameters - spatially resolved diffuse reflectance spectral data sets. This process does not need to consider the specific structure of the collection fiber, and simultaneously processes all spatial diffuse reflectance spectral information emitted from the tissue surface

[0072] During the simulation, the number of traced photons was set to 106, the longitudinal and radial resolutions were 0.02 mm, and the longitudinal, radial, and angular grid numbers were all 100. The refractive index of the medium surrounding the tissue was 1.0, and the incident light was irradiated to the tissue through an optical fiber with a core diameter of 400 um and a core refractive index of 1.52.

[0073] During the simulation, the tissue refractive index, anisotropy coefficient, and thickness were set as fixed values, the tissue absorption coefficient and scattering coefficient step size was 2 cm-1, and the total number of simulations was 50×50×25. The radial spatial resolution was set to 0.02 cm, the radial spatial sampling points were 100, and the photon overflow distance beyond 2 cm was uniformly recorded as 2 cm. After the simulation, the spatially resolved diffuse reflection light overflowing the tissue surface was recorded, and the incident light distribution was processed by convolution operation (Gaussian distribution, spot diameter of 0.4 mm) to obtain the final diffuse reflection spectrum. Thus, the optical parameters can be obtained to obtain the spatially resolved tissue diffuse reflection spectrum database, which has 3 dimensions as input (dermis absorption coefficient, epidermis absorption coefficient, tissue scattering coefficient), and 100 dimensions as output, corresponding to the diffuse reflection light emitted from the tissue surface at different radial distances.

[0074] During the simulation, the above parameters were brought into the graphics card accelerated Monte Carlo method for photon tracing. After the simulation, the spatially resolved diffuse reflectance spectrum emitted from the tissue surface was recorded, and then a convolution operation was performed according to the incident light conditions (Gaussian distribution, spot diameter of 400um) to obtain the recorded spatially resolved diffuse reflectance spectrum. Subsequently, the tissue absorption and scattering coefficients were continuously updated (the step size was 2cm). -1 ), and re-simulate to establish tissue optical parameters and spatially resolved diffuse reflectance spectroscopy data sets such as Figure 2 As shown. After simulation, a spatially resolved diffuse reflectance spectrum is obtained for each combination of the upper layer absorption coefficient, the lower layer absorption coefficient, and the scattering coefficient of each tissue. The number of optical parameter combinations is 50×50×25, totaling 62,500. In summary, the input feature of the established tissue optical parameter-spatial resolved diffuse reflectance spectrum dataset is an optical parameter matrix of size 62,500×3, and the output is a diffuse reflectance matrix of size 62,500×100. Figure 2 The incident and exit distances in (a) to (f) are 0 mm, 0.2 mm, 0.4 mm, 1.6 mm, 1.8 mm and 2 mm respectively. a，epi is the absorption coefficient of the first layer of tissue; μ a，derm is the absorption coefficient of the second layer of tissue; μ s is the tissue scattering coefficient.

[0075] 3. Design a long short-term memory network with tissue optical parameters as input and tissue spatially resolved diffuse reflectance spectrum as output, deeply mine the sequence features in spectral data, and improve the model's predictive ability for spectral data without increasing the complexity of the model.

[0076] After generating tissue optical parameters and tissue spatially resolved diffuse reflectance spectrum data sets, it is necessary to learn the mapping relationship between optical parameters and spatially resolved diffuse reflectance spectra. When establishing the forward model in the iterative algorithm, the model input is tissue optical parameters, and the output is spatially resolved tissue diffuse reflectance spectra overflowing the tissue surface. The present invention takes into account the natural sequence characteristics of diffuse reflectance spectra, uses long short-term memory networks to simulate the mapping relationship between tissue optical parameters and spatially resolved diffuse reflectance spectra, and deeply mines the hidden correlations between spectral data.

[0077] Considering the sequence characteristics of spatially resolved diffuse reflectance spectra, the designed long short-term memory network can be a unidirectional network or a bidirectional network. In a unidirectional network, the diffuse reflection light intensity emitted from any spatial position on the tissue surface is only related to the diffuse reflection light between the incident point of the illumination light source and the diffuse reflection light exit point; in a bidirectional network, the diffuse reflection light intensity emitted from any spatial position on the tissue surface is related to all diffuse reflection lights emitted from the tissue surface.

[0078] Figure 3 The figure shows the structural block diagram of the tissue spatial resolution diffuse reflectance spectrum generation model based on long short-term memory network, where μ a，epi , μ a，derm , μ s are the tissue epidermis absorption coefficient, dermis absorption coefficient and tissue scattering coefficient, r <1> ~r <n>< / n> is the spatially resolved diffuse light intensity overflowing the tissue surface at different illumination-collection radial distances, n is the length of the generated sequence, that is, the number of radial grids in the Monte Carlo simulation. a represents the hidden state of the network, where a <0> It is the initial hidden state and is set to a zero vector during the model building process. In addition, the number and size of the network hidden layers determine the learning ability and model complexity of the model. The more hidden layers there are and the larger the size of the hidden layers, the more complex the model is and the stronger the model learning ability is. However, increased model complexity will lead to increased training difficulty and increase the risk of overfitting. Combined with the strong correlation between adjacent data points of tissue diffuse reflectance spectra and the characteristics of computer computing performance, the long-term and short-term network involved in the present invention has 2 hidden layers, and the hidden layer size is 2n (n is an integer ≥3).

[0079] After the long short-term memory network model is determined, it is necessary to substitute the training data for model parameter optimization. The present invention randomly extracts 80% as a training set, 10% as a validation set, and 10% as a test set from the tissue optical parameters and spatially resolved diffuse reflectance spectroscopy data sets. The final number of training set samples is 50,000, the number of test set samples is 6,250, and the number of test set samples is 6,250. Batch gradient descent is used for parameter optimization during training, the batch size is 32, and the gradient descent algorithm uses momentum gradient descent. Before optimization, the input parameters are normalized so that their variance is 1. After each parameter update, the current model is used to verify the training set, validation set, and test set, and the average error of the spatially resolved diffuse reflectance spectrum in each data set is recorded. Figure 4 The specific changes of training error, validation error and test error are shown, that is, as the number of iterations increases, the training error, validation error and test error gradually decrease; and after 10,000 iterations, the training error, validation error and test error are basically the same and remain unchanged, indicating that the training results of the long short-term memory network model converge to the optimal, which minimizes the error between the model's predicted value and the actual result. Therefore, the model can achieve a higher accuracy when predicting the validation set data.

[0080] The forward model directly uses the above pre-trained tissue optical parameters to the spatially resolved tissue diffuse reflectance spectroscopy dataset. 80% are randomly selected as training sets, 10% as validation sets, and 10% as test sets. The model input is a 3D feature vector consisting of the dermis absorption coefficient, epidermis absorption coefficient, and tissue scattering coefficient. The output is a 100-dimensional sequence data consisting of diffuse reflectance spectra emitted from the tissue surface at different radial distances. The number of hidden layers contained in the long short-term memory model is 2, and the hidden layer size is 2 n (n is an integer), the input of each unit in the hidden layer includes sample features and a hidden state, and the output includes tissue diffuse reflectance intensity and a hidden state. The output hidden state of the previous layer is connected to the input hidden state of the next layer in sequence to form a recurrent network structure. The model is trained on the training set, and the stochastic gradient descent algorithm is used for parameter optimization during the training process. After the model is iteratively converged, the tissue spatial resolution diffuse reflectance spectrum generation model is finally obtained.

[0081] 4. Combine the fiber optic probe structure and tissue diffuse reflectance spectrum generation model to extract tissue physiological and optical parameters through iterative algorithm

[0082] Figure 5 The figure shows the model flow of tissue physiological and optical parameter extraction: First, the tissue physiological parameters are randomly initialized, and the tissue absorption coefficient and scattering coefficient at different wavelengths are calculated according to the Lambert-Beer law and Mie scattering theory; then, the generative model is used to simulate the spatially resolved diffuse reflectance spectra of tissues at different wavelengths, and the tissue diffuse reflectance intensity at different wavelengths collected by the probe is calculated in combination with the fiber optic probe structure; finally, it is compared with the measured tissue diffuse reflectance spectrum, the error between the two is calculated, and the physiological parameters are updated and optimized, and the corresponding physiological parameters when the error is minimized are output. After obtaining the physiological parameters, the tissue absorption coefficient and scattering coefficient are calculated in combination with the Lambert-Beer law and Mie scattering theory.

[0083] 5. Using the error in tissue optical parameter extraction as the cost function, optimize the core distance between the optical fiber probe illumination fiber and the collection fiber

[0084] Figure 6 The figure shows the structure of the optical fiber probe used in the present invention. The core diameters of the illumination optical fiber and the collection optical fiber are both 400um, and a certain distance is maintained between the cores of the excitation optical fiber and the collection optical fiber.

[0085] Figure 7 The spatially resolved diffuse reflectance spectra of tissues under different optical parameters and the collection efficiency of the optical fiber probe under different core distances of the illumination fiber and the collection fiber are shown. It can be seen from the figure that with the increase of radial distance, the intensity of spatially resolved diffuse reflectance light will gradually decrease, and when the radial distance is 2000um, the diffuse reflectance light intensity is almost 0; in addition, when the core distances of the illumination fiber and the collection fiber are different, the collection efficiency of the optical fiber probe is significantly different.

[0086] Figure 8 It is shown that under given optical parameters, different core distances between the illumination fiber and the collection fiber will lead to obvious differences in diffuse reflection light at different wavelengths, indicating that the fiber probe structure has a significant impact on the tissue diffuse reflection spectrum. To this end, the present invention determines a method for optimizing the structure of the fiber probe. First, under typical physiological parameters of skin tissue, the Lambert-Beer law and Mie scattering theory are used to calculate the tissue absorption coefficient and scattering coefficient of different wavelengths; secondly, the spatially resolved diffuse reflection spectrum generation model of the tissue is used to calculate the spatially resolved diffuse reflection spectrum emitted from the tissue surface at different wavelengths; then, the core distance between the illumination fiber and the collection fiber in the fiber probe is randomly initialized, and the wavelength-resolved diffuse reflection spectrum emitted from the tissue surface is calculated; and according to the skin tissue physiological parameter extraction method, the wavelength-resolved diffuse reflection spectrum is used as input to invert the tissue physiological parameters, and calculate the error between the parameters and the typical physiological parameters of skin tissue. Finally, the core distance between the illumination fiber and the collection fiber obtained by iterative fitting is the optimal distance.

[0087] Parameter extraction model: According to Lambert-Beer's law and Mie scattering theory, the tissue absorption coefficient and scattering coefficient at different wavelengths are calculated, and the spatially resolved diffuse reflectance spectra of tissues at different wavelengths are simulated. The diffuse reflectance light intensity of tissues at different wavelengths collected by the probe is calculated in combination with the fiber optic probe structure; the physiological parameters of tissues are extracted through an iterative fitting algorithm, and the tissue absorption coefficient and scattering coefficient are inverted. Fiber optic probe optimization method: Under different distances between the excitation fiber and the collection fiber, the tissue spatially resolved diffuse reflectance spectrum generation model is continuously called to output the physiological parameters corresponding to the minimum difference between the diffuse reflectance spectrum generated by the model and the measured diffuse reflectance spectrum, and the difference between the parameter and the actual parameter is calculated. The optimal fiber optic probe structure is output with the minimum difference as the evaluation index.

[0088] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting tissue optical parameters based on long short-term memory network, characterized in that: The following steps are involved: S1. Determine the structural parameters and optical parameters of the multi-layer skin tissue optical model according to the tissue type to be tested; The multi-layer skin tissue optical model is a double-layer structure, in which the upper layer absorption comes from melanin and the lower layer absorption comes from hemoglobin, that is, the absorption coefficients are independent of each other; the scattering particles in the upper and lower layers are uniform in size and concentration, and the scattering coefficients are the same; S2. For the multi-layer skin tissue optical model described in step S1, a Monte Carlo method based on graphics card acceleration is used to simulate the spatially resolved diffuse reflectance spectra emitted from the tissue surface under different optical parameters, and a data set consisting of tissue optical parameters and tissue spatially resolved diffuse reflectance spectra is generated; at the same time, in order to make full use of diffuse reflectance spectrum information, the data set does not consider the limitation of the detection range of the optical fiber probe, and all spatially resolved diffuse reflectance spectra emitted from the tissue surface are included in the data set; S3. Design a tissue spatially resolved diffuse reflectance spectrum generation model based on long short-term memory network, use tissue optical parameters and tissue spatially resolved diffuse reflectance spectrum datasets as training sets, and use error back propagation algorithm to optimize the generation model parameters; S4. According to Lambert-Beer's law and Mie scattering theory, the tissue optical parameters are calculated from the tissue physiological parameters. The model is generated by combining the distance between the illumination fiber and the collection fiber core in the fiber optic probe and the spatially resolved diffuse reflectance spectrum of the tissue. The tissue physiological parameters are extracted by least square fitting, and the tissue optical parameters are inverted. The implementation steps are as follows: (1) Randomly initialize tissue physiological parameters; (2) Calculate tissue absorption coefficient and scattering coefficient at different wavelengths based on Lambert-Beer law and Mie scattering theory; (3) Input the tissue absorption coefficient and scattering coefficient into the tissue spatially resolved diffuse reflectance spectrum generation model to calculate the spatially resolved diffuse reflectance spectrum emitted from the tissue surface at different wavelengths; (4) Calculate the diffuse reflection light intensity of tissue detected by the fiber optic probe at different wavelengths based on the fiber optic probe structure; (5) Calculate the error between the diffuse reflectance spectrum generated by the model and the measured diffuse reflectance spectrum, and correct the input physiological parameters according to the error size; (6) Repeat the process (2) to (5) until the diffuse reflectance spectrum generated by the model coincides with the measured diffuse reflectance spectrum. At this time, the corresponding physiological parameters are the physiological parameters of the tissue to be measured. (7) Substitute the extracted tissue physiological parameters into Lambert-Beer law and Mie scattering theory to invert tissue optical parameters; S5. After determining the method for extracting tissue physiological parameters, the tissue physiological parameter extraction error is used as the cost function, and an iterative fitting method is used to optimize the core distance between the illumination fiber and the collection fiber in the optical fiber probe. Under the optimized core distance between the illumination fiber and the collection fiber, step S4 is executed to obtain the physiological and optical parameters of skin tissue.

2. The method for extracting tissue optical parameters based on long short-term memory network according to claim 1, characterized in that: The structural parameters of the skin tissue optical model in step S1 include: the number of tissue layers m, the thickness of each layer d; the optical parameters of the skin tissue optical model include: the absorption coefficient μ of each layer of tissue a , scattering coefficient μ s , anisotropy coefficient g and refractive index n.

3. The method for extracting tissue optical parameters based on long short-term memory network according to claim 2, characterized in that: The long short-term memory network described in step S3 is a unidirectional network or a bidirectional network.

4. The method for extracting tissue optical parameters based on long short-term memory network according to claim 3, characterized in that: The specific implementation steps of using the iterative fitting method to optimize the distance between the cores of the illumination optical fiber and the collection optical fiber in the optical fiber probe described in step S5 are as follows: (a) Under the typical physiological parameters of skin tissue, the tissue absorption coefficient and scattering coefficient at different wavelengths are calculated using the Lambert-Beer law and Mie scattering theory; (b) Use the tissue spatially resolved diffuse reflectance spectrum generation model to calculate the spatially resolved diffuse reflectance spectrum emitted from the tissue surface at different wavelengths; (c) Randomly initialize the distance between the illumination fiber and the collection fiber core in the fiber optic probe, and calculate the spectrally resolved diffuse reflectance spectrum emitted from the tissue surface; (d) according to the method for extracting physiological parameters of skin tissue, using the spectrally resolved diffuse reflectance spectrum as input, inverting the physiological parameters of the tissue, and calculating the error between the parameters and the typical physiological parameters of skin tissue; and correcting the distance between the core of the illumination fiber and the collection fiber according to the error size; (e) Repeat (d) until the difference between the physiological parameters extracted by the model and the typical physiological parameters of skin tissue is minimal. At this time, the corresponding distance between the core of the illumination fiber and the collection fiber is the optimal distance.

5. A tissue optical parameter extraction system based on long short-term memory network, characterized in that: include: Model parameter determination module, data set generation module, model optimization module, tissue optical parameter inversion module, tissue optical parameter optimization module; The model parameter determination module is used to determine the structural parameters and optical parameters of the multi-layer skin tissue optical model according to the tissue type to be measured; The multi-layer skin tissue optical model is a double-layer structure, in which the upper layer absorption comes from melanin and the lower layer absorption comes from hemoglobin, that is, the absorption coefficients are independent of each other; the scattering particles in the upper and lower layers are uniform in size and concentration, and the scattering coefficients are the same; The data set generation module uses a Monte Carlo method based on graphics card acceleration to simulate the spatially resolved diffuse reflectance spectra emitted from the tissue surface under different optical parameters for the multi-layer skin tissue optical model described in the model parameter determination module, and generates a data set consisting of tissue optical parameters and tissue spatially resolved diffuse reflectance spectra; at the same time, in order to make full use of diffuse reflectance spectrum information, the data set does not consider the limitation of the detection range of the optical fiber probe, and includes all spatially resolved diffuse reflectance spectra emitted from the tissue surface into the data set; The model optimization module is used to design a tissue spatially resolved diffuse reflectance spectrum generation model based on a long short-term memory network, using tissue optical parameters and tissue spatially resolved diffuse reflectance spectrum data sets as training sets, and using an error back propagation algorithm to optimize the generation model parameters; The tissue optical parameter inversion module is used to calculate tissue optical parameters from tissue physiological parameters according to Lambert-Beer's law and Mie scattering theory, combine the distance between the illumination fiber and the collection fiber core in the fiber optic probe and the tissue spatially resolved diffuse reflectance spectrum to generate a model, use least squares fitting to extract tissue physiological parameters, and invert tissue optical parameters. The implementation steps are as follows: (1) Randomly initialize tissue physiological parameters; (2) Calculate tissue absorption coefficient and scattering coefficient at different wavelengths based on Lambert-Beer law and Mie scattering theory; (3) Input the tissue absorption coefficient and scattering coefficient into the tissue spatially resolved diffuse reflectance spectrum generation model to calculate the spatially resolved diffuse reflectance spectrum emitted from the tissue surface at different wavelengths; (4) Calculate the diffuse reflection light intensity of tissue detected by the fiber optic probe at different wavelengths based on the fiber optic probe structure; (5) Calculate the error between the diffuse reflectance spectrum generated by the model and the measured diffuse reflectance spectrum, and correct the input physiological parameters according to the error size; (6) Repeat the process (2) to (5) until the diffuse reflectance spectrum generated by the model coincides with the measured diffuse reflectance spectrum. At this time, the corresponding physiological parameters are the physiological parameters of the tissue to be measured. (7) Substitute the extracted tissue physiological parameters into Lambert-Beer law and Mie scattering theory to invert tissue optical parameters; The tissue optical parameter optimization module is used to determine the tissue physiological parameter extraction method, use the tissue physiological parameter extraction error as the cost function, and use the iterative fitting method to optimize the core distance between the illumination fiber and the collection fiber in the optical fiber probe. Under the optimized core distance between the illumination fiber and the collection fiber, return to the tissue optical parameter inversion module to obtain the physiological and optical parameters of skin tissue.

6. The tissue optical parameter extraction system based on long short-term memory network according to claim 5, characterized in that: The structural parameters of the skin tissue optical model in the model parameter determination module include: the number of tissue layers m, the thickness of each layer d; the optical parameters of the skin tissue optical model include: the absorption coefficient μ of each layer of tissue a , scattering coefficient μ s , anisotropy coefficient g and refractive index n.

7. The tissue optical parameter extraction system based on long short-term memory network according to claim 6, characterized in that: The long short-term memory network described in the model optimization module is a unidirectional network or a bidirectional network.

8. The tissue optical parameter extraction system based on long short-term memory network according to claim 7, characterized in that: The specific implementation steps of using the iterative fitting method to optimize the distance between the cores of the illumination fiber and the collection fiber in the fiber probe described in the tissue optical parameter optimization module are as follows: (a) Under the typical physiological parameters of skin tissue, the tissue absorption coefficient and scattering coefficient at different wavelengths are calculated using the Lambert-Beer law and Mie scattering theory; (b) Use the tissue spatially resolved diffuse reflectance spectrum generation model to calculate the spatially resolved diffuse reflectance spectrum emitted from the tissue surface at different wavelengths; (c) Randomly initialize the distance between the illumination fiber and the collection fiber core in the fiber optic probe, and calculate the spectrally resolved diffuse reflectance spectrum emitted from the tissue surface; (d) according to the method for extracting physiological parameters of skin tissue, using the spectrally resolved diffuse reflectance spectrum as input, inverting the physiological parameters of the tissue, and calculating the error between the parameters and the typical physiological parameters of skin tissue; and correcting the distance between the core of the illumination fiber and the collection fiber according to the error size; (e) Repeat (d) until the difference between the physiological parameters extracted by the model and the typical physiological parameters of skin tissue is minimal. At this time, the corresponding distance between the core of the illumination fiber and the collection fiber is the optimal distance.

Images