Reconstruction method, device and system for diffused light tomography and medium

Abstract

The invention discloses a reconstruction method, device and system for diffused light tomography and a medium. The reconstruction method comprises the step of utilizing a processor to reconstruct optical parameter spatial distribution of a to-be-measured body by utilizing a reconstruction model comprising a prior distribution coding network, a feature coding network and a decoding network based on measurement data obtained after the to-be-measured body is measured by utilizing a diffused light tomography technology, generating corresponding priori hidden space parameter distribution information by using the trained priori distribution coding network and generating hidden space sampling; based on the measurement data, extracting structural feature implicit information in the measurement data by using the trained feature coding network; performing fusion processing on the hidden space samples and the structural feature hidden information to generate fused hidden space samples; and on the basis of fused hidden space sampling, optical transmission parameter space distribution of the to-be-detected body is reconstructed by using a decoding network. According to the invention, the three-dimensional image of the to-be-measured body can be reconstructed with high precision based on the diffused light tomography data.

Description

Technical Field

[0001] The present application belongs to the field of medical artificial intelligence, and in particular, relates to a reconstruction method, device, system and medium for diffuse optical tomography. Background Art

[0002] Early diagnosis and treatment evaluation of brain diseases are particularly important. Brain functional imaging technologies include functional magnetic resonance imaging (fMRI), electrophysiological imaging (EEG), magnetoencephalography (MEG), functional near-infrared spectroscopy (fNIRS), etc., which can quantitatively image the spatiotemporal state of brain activity and provide important reference for the early diagnosis and treatment of brain diseases.

[0003] Among them, fNIRS, as an emerging brain functional imaging technology, has the advantages of low cost, high portability, non-invasiveness, and no ionizing radiation. It is suitable for large-scale clinical screening and daily status monitoring and evaluation of patients with brain diseases. The diffuse optical tomography (DOT) technology based on fNIRS provides technical possibilities for 3D reconstruction of images inside brain tissue, but as an inverse problem, DOT image reconstruction is also ill-posed: on the one hand, the layered structure of the brain (scalp, skull, cerebrospinal fluid, gray matter, white matter) makes the detection of the state of the cerebral cortex susceptible to physiological interference from shallow tissues; on the other hand, the strong scattering of the brain's internal tissues makes it easy for photons to migrate randomly during transmission inside the tissues, resulting in higher noise sensitivity of DOT imaging results, especially for deep tissue detection. Therefore, how to improve the credibility, accuracy and spatial resolution of the reconstruction results when reconstructing 3D images based on diffuse optical tomography measurement data is an urgent problem to be solved. Summary of the invention

[0004] The present application is provided to solve the above-mentioned defects existing in the prior art. A reconstruction method, device, system and medium for diffuse optical tomography are needed, which can perform more reliable, more accurate and higher spatial resolution 3D image reconstruction based on diffuse optical tomography measurement data at a deeper depth, thereby promoting the clinical applicability of related technologies such as fNIRS brain functional imaging.

[0005] According to a first scheme of the present application, a reconstruction method for diffuse light tomography is provided, the reconstruction method comprising, using a processor, based on measurement data obtained after measuring the object to be measured using diffuse light tomography technology, and using a reconstruction model including a prior distribution encoding network, a feature encoding network and a decoding network, to reconstruct the optical parameter spatial distribution of the object to be measured, specifically comprising: based on the measurement data, using a trained prior distribution encoding network to generate prior latent space parameter distribution information corresponding to the measurement data, and generating latent space sampling based on the prior latent space parameter distribution information; based on the measurement data, using a trained feature encoding network to extract structural feature latent information in the measurement data; fusing the latent space sampling and the structural feature latent information to generate a fused latent space sampling; based on the fused latent space sampling, using the decoding network to reconstruct the optical transmission parameter spatial distribution of the object to be measured, so as to generate a reconstructed three-dimensional image of the object to be measured.

[0006] According to a second scheme of the present application, a reconstruction device for diffuse optical tomography is provided, the reconstruction device comprising at least a processor and a memory, the memory storing computer executable instructions, and the processor performing the operations of the reconstruction method for diffuse optical tomography as described in the various embodiments of the present application when executing the computer executable instructions.

[0007] According to the third scheme of the present application, a near-infrared brain function imaging system is provided, comprising a near-infrared optical data acquisition device and a reconstruction device for diffuse light tomography; wherein the near-infrared optical data acquisition device is used to acquire near-infrared data of an object; and the reconstruction device is used to use the near-infrared data to perform the operations of the reconstruction method for diffuse light tomography as described in the various embodiments of the present application.

[0008] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing a program is provided, wherein the program enables a processor to perform the operations of the reconstruction method for diffuse optical tomography as described in various embodiments of the present application.

[0009] The reconstruction method, device, system and medium for diffuse optical tomography provided by various embodiments of the present application, by constructing a reconstruction model including a prior distribution coding network, a feature coding network and a decoding network, in the process of reconstructing the optical parameter spatial distribution of the object to be measured based on the diffuse optical measurement data of the object to be measured, can not only use the pre-trained prior distribution coding network to make a relatively accurate estimate of the latent space parameter distribution information corresponding to the measurement data, but also use the pre-trained feature coding network to extract the potential structural feature information of the object to be measured from the measurement data, and use it to constrain the latent space parameter distribution information generated based on the latent space parameter distribution information. The latent space sampling data is used to make the decoding network more reliable and accurate when predicting the spatial distribution of the optical transmission parameters of the object to be tested based on the constrained latent space sampling data. In particular, the utilization of the structural feature information of the object to be tested can effectively reduce the inappropriateness of solving the inverse problem of diffuse light tomography reconstruction based on the probability model, improve the detection ability of deep tissues, and thus achieve reliable, accurate, higher spatial resolution and more robust 3D image reconstruction of the object to be tested at a deeper depth, which is of great significance in promoting the use of fNIRS technology in early screening, diagnosis and prognosis evaluation of brain diseases and in a wider range of clinical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 A schematic flow chart of a reconstruction method for diffuse optical tomography according to an embodiment of the present application is shown.

[0012] Figure 2 A block diagram showing some components of a reconstruction model in the inference application phase according to an embodiment of the present application.

[0013] Figure 3 A schematic diagram of the network structure of the training phase of the reconstruction model according to an embodiment of the present application is shown.

[0014] Figure 4 Another schematic diagram of the network structure of the training phase of the reconstruction model according to an embodiment of the present application is shown.

[0015] Figure 5 A schematic diagram showing the composition structure of a feature encoding network according to an embodiment of the present application is shown.

[0016] Figure 6A schematic diagram showing reconstruction results of the reconstruction method according to the present application under different types of media and target numbers.

[0017] Figure 7 A schematic diagram showing a comparison of the multi-target image reconstruction performance of the reconstruction method of an embodiment of the present application under different signal-to-noise ratios is shown. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation examples. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation examples, but are not intended to limit the present application.

[0019] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprises" and similar words used in this application mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of covering other elements. In this application, the arrows shown in the figures of each step are only examples of the execution order, not limitations. The technical solution of this application is not limited to the execution order described in the embodiments. The various steps in the execution order can be combined, decomposed, or the order can be changed, as long as the logical relationship of the execution content is not affected.

[0020] All terms (including technical terms or scientific terms) used in this application have the same meaning as those understood by ordinary technicians in the field to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined here. Techniques and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the techniques and equipment should be considered as part of the specification.

[0021] In the present application, the arrows shown in the figures of each step are merely examples of the execution order, not limitations. The technical solution of the present application is not limited to the execution order described in the embodiments. The various steps in the execution order can be combined for execution, can be decomposed for execution, and can be reordered, as long as the logical relationship of the execution content is not affected.

[0022] Figure 1 A schematic flow chart of a reconstruction method for diffuse optical tomography according to an embodiment of the present application is shown. Figure 2 A block diagram showing some components of a reconstruction model in the inference application phase according to an embodiment of the present application.

[0023] like Figure 1As shown, the reconstruction method includes using a processor, based on the measurement data obtained after measuring the object using the diffuse light tomography technology, using Figure 2 The reconstruction model 200 shown includes a prior distribution encoding network 201, a feature encoding network 202 and a decoding network 203 to reconstruct the spatial distribution of optical parameters of the object to be tested. The object to be tested can be, for example, the whole body of a living being, or a specific organ of a living being, such as the head of a subject, the abdomen of a subject, etc. As long as it is a thick tissue body, and under multi-point excitation such as near-infrared light, the time, space and spectral distribution measurement information of the diffuse light on its surface can reflect the three-dimensional distribution of the optical characteristic parameters inside the tissue body, no specific limitation is made here. In addition, the diffuse light tomography technology includes irradiating the object to be tested with near-infrared light (600nm-900nm), and the wavelength range of the near-infrared light used can be set as required according to the characteristics of the object to be tested. The specific measurement method can, for example, arrange a light source and a light source detector arranged in a certain topological structure on the surface of the object to be tested, and form an effective detection channel between the light source detector and at least part of the light source. During measurement, incident light is emitted from the light source to the object to be tested, and the diffuse light in the outgoing light of the corresponding detection channel obtained from the light source detector is used as measurement data. The number, placement, connection method and other topological structures of light sources and light source detectors can be set as needed. The measurement values ​​of each detection channel can be arranged in a predetermined order as a one-dimensional array, a two-dimensional matrix, or in any other applicable data format, which will not be elaborated here.

[0024] The above process can be specifically implemented by executing the following steps 101-104.

[0025] First, in step 101, based on the measurement data, the trained prior distribution encoding network 201 is used to generate prior latent space parameter distribution information corresponding to the measurement data, and a latent space sampling is generated based on the prior latent space parameter distribution information.

[0026] In some embodiments, the prior distribution encoding network 201 can be implemented as a deep network based on multiple continuously stacked convolutional layers, so as to extract multi-level feature representations layer by layer and learn the characteristic parameter distribution of the object to be measured in the latent space from the measurement data.

[0027] In step 102, based on the measurement data, the structural feature implicit information in the measurement data is extracted using the trained feature encoding network 202. It is worth noting that there is no order between step 102 and step 101, and they can usually be executed synchronously and in parallel.

[0028] Then, in step 103, the latent space sampling and the structural feature latent information are fused to generate a fused latent space sampling.

[0029] Next, in step 104, based on the fused latent space sampling, the decoding network 203 is used to reconstruct the spatial distribution of the optical transmission parameters of the object to be measured, so as to generate a reconstructed three-dimensional image of the object to be measured.

[0030] According to the reconstruction method for diffuse optical tomography of the embodiment of the present application, through pre-training, on the one hand, the trained prior distribution coding network can make a relatively accurate estimate of the latent space parameter distribution information reflecting the structure and function information of the object to be tested based on the measurement data and the probability model; on the other hand, the feature coding network is further used to extract the potential structural feature information of the object to be tested from the measurement data, and it is fused with the latent space sampling generated based on the latent space parameter distribution information, so that the fused latent space sampling can more accurately and with higher precision reflect the optical transmission parameter spatial distribution information of the object to be tested, especially it can improve the shortcomings of diffuse optical tomography technology in deep tissue detection, thereby realizing reliable, accurate, higher spatial resolution and more robust 3D image reconstruction of the object to be tested at a deeper depth. The numerical simulation and physical model experimental results of various scenes show that the reconstruction method of the embodiment of the present application can reconstruct the object to be tested with a depth of 1.5cm, and the spatial resolution of the reconstructed image (the minimum spatial distance that can be distinguished as different objects / absorbers during reconstruction, also known as the effective resolution) can reach 1cm. The performance superiority relative to other existing technologies will be specifically explained in combination with the experimental results in the following text.

[0031] In some embodiments, the feature encoding network can be implemented as any applicable type of neural network and includes a convolution module. The reconstruction method can further include, using the processor: introducing an attention mechanism in the convolution module of the feature encoding network and outputting an attention weight matrix associated with the tissue depth as implicit information of the structural features in the measurement data.

[0032] Taking into account the shortcomings of DOT image reconstruction in terms of deep tissue detection and anti-interference capabilities, using the attention weight matrix carrying tissue depth-related information as a priori condition constraints for latent space sampling will help the decoding network selectively focus on areas where optical properties have changed significantly. On the one hand, it can improve the sensitivity to changes in deep tissue parameter characteristics, and on the other hand, it can effectively remove measurement data disturbances caused by spatial noise rather than actual tissue structure, thereby enhancing the credibility, accuracy, precision and robustness of DOT image reconstruction.

[0033] Figure 3 FIG. 4 is a schematic diagram showing a network structure of a training phase of a reconstruction model according to an embodiment of the present application. Figure 3 As shown, during the training phase of the reconstruction model, a training network structure 300 different from that used in the inference application phase will be used.

[0034] and Figure 2 Compared with the reconstruction model 200 in the training network structure 300, a posterior distribution coding network 301 is introduced in the training network structure 300. Using the training samples in the training sample set, based on the loss function composed of the weighted sum of the first loss function and the second loss function, the prior distribution coding network 201, the feature coding network 202, the decoding network 203 and the posterior distribution coding network 301 are collaboratively trained. In addition, during the collaborative training process, the network parameters of the prior distribution coding network 201, the feature coding network 202 and the decoding network 203 are collaboratively optimized until a preset training cutoff condition is met, such as when the value of the loss function is less than a preset threshold, or the number of training rounds reaches a preset upper limit, etc., the training can be considered to be completed, and the obtained prior distribution coding network 201, the feature coding network 202, and the decoding network 203 can be used as a trained reconstruction model for reconstructing the spatial distribution of optical parameters of the object to be measured.

[0035] In the embodiment of the present application, both the prior distribution and the posterior distribution can be constructed and approximated using a multivariate Gaussian distribution probability model with the same number of dimensions. Therefore, after the training is completed, the prior latent space parameter distribution information generated by the trained prior distribution encoding network 201 will follow the first multivariate Gaussian distribution, while the posterior latent space parameter distribution information generated by the trained posterior distribution encoding network 301 will follow the second multivariate Gaussian distribution, and the first multivariate Gaussian distribution and the second multivariate Gaussian distribution have the same number of dimensions.

[0036] In some embodiments, the training samples for training the network structure 300 can be obtained as follows: First, based on the anatomical information of the object to be tested for three-dimensional image reconstruction, such as a human head, multiple simulated / simulated test objects with ground truth values ​​of the reconstructed three-dimensional image are constructed for training. The simulated / simulated test objects will be collectively referred to as simulated test objects in this application, but it should be noted that in fact, it can be a simulated test object modeled in a simulation system, or a simulated test object constructed in a physical or semi-physical simulation system. This application does not make specific restrictions on this, as long as the simulation system / simulation system is integrated with an optical transmission forward calculation model adapted to the test object, and can simulate / simulate the transmission law of photons in the test object.

[0037] During modeling, each different simulated test object may have the same or different external contours, as well as differentiated internal organizational structures, to simulate different individuals of the same type of test object. On this basis, each constructed simulated test object is simulated and measured to generate corresponding simulated measurement data, and the paired simulated measurement data of each simulated test object and the reconstructed three-dimensional image ground truth are used as training samples.

[0038] In some embodiments, performing simulation measurement on each simulated object to be tested to generate corresponding simulation measurement data specifically includes: setting a topological structure composed of a light source and a light source detector of diffuse light imaging technology on the surface of each simulated object to be tested, wherein the light source is used to emit incident light to each simulated object to be tested, and the light source detector is used to detect the outgoing light, and a detection channel is formed between the light source detector and at least part of the light source; in the case where there is an absorber in each simulated object to be tested, constructing simulation measurement data based on the change amount of the optical transmission parameter of each simulated object to be tested and the change amount of the outgoing light parameter of each detection channel arranged in a preset order, wherein the absorber has an absorption effect on the light emitted by the light source, and the voxel resolution of the simulated object to be tested is between 1mm and 2mm, which is higher than the 5mm commonly used in the prior art. Generally speaking, it can be considered that the higher the voxel resolution (that is, the smaller the size of a single voxel in three-dimensional space), the finer the reconstructed image can be (that is, the higher the spatial resolution), but under the framework of the prior art, it will lead to excessive calculation and too long solution time, and the ill-posedness of the inverse problem of image reconstruction will be further aggravated, and the reconstruction effect, such as spatial resolution, may not be able to be stably improved. However, when adopting the reconstruction algorithm of the embodiment of the present application, since the feature encoding network can provide additional structural feature prior information for the latent space sampling generated based on the prior distribution encoding network, the fused latent space sampling of the input decoding network after fusion has dual-domain constraints of both the measurement domain and the reconstructed image domain, which can not only greatly reduce the ill-posedness of the inverse problem of DOT image reconstruction, but also promote rapid convergence during reconstruction model training and shorten the training time. After completing the training using the samples generated by simulation, it can robustly generate three-dimensional DOT reconstructed images with higher spatial resolution for simulated test objects with higher voxel resolution with an acceptable amount of computation and a shorter solution time.

[0039] Figure 4 Another schematic diagram of the network structure of the training phase of the reconstruction model according to the embodiment of the present application is shown. In other embodiments, such as Figure 4 As shown in the training network 400 in FIG. 1 , before feeding the simulated measurement data of the training sample to the prior distribution coding network 201, the feature coding network 202 and the posterior distribution coding network 301, the simulated measurement data can be upsampled by using the upsampling module 401 according to the voxel resolution of the three-dimensional image to be reconstructed, and the feature data aligned with the voxel resolution of the three-dimensional image to be reconstructed can be output. As an example only, the upsampling module 401 can be implemented as a fully connected network, so that while aligning the dimensions, it can also provide a rough reconstruction effect for sparse measurements, so that each subsequent coding network can perform more refined feature extraction based on high-dimensional features containing richer semantics, and can alleviate the ill-posedness of inverse problem solving to a certain extent.

[0040] like Figure 4 As shown, assuming that the resolution of the three-dimensional image to be reconstructed is 16×26×26, where 16 is the number of layers of the reconstructed image in the depth dimension, and 26×26 is the resolution of each layer of the two-dimensional image, then the dimension of the dimensionally aligned feature data output by the upsampling module 401 can be expressed as (16, 26, 26). In this case, the dimension of the attention weight matrix, which is the implicit information of the structural features, will also match the voxel resolution of the three-dimensional image to be reconstructed, that is, the weight matrix is ​​also a three-dimensional matrix with a dimension of (16, 26, 26).

[0041] Next, when fusing the latent space sampling and the latent information of the structural features, different methods can be used. Assuming that the dimension of the first multivariate Gaussian distribution is 4, the latent space sampling is a three-dimensional matrix of dimension (4, 1, 1) containing 4 random sampling values ​​(e.g., s1, s2, s3, and s4). Then, first, according to the three dimensions of the latent information of the structural features, the three-dimensional matrix can be expanded on the other two dimensions, i.e., 26×26, outside the image layer (the number of layers is 16). The dimension of the latent space sampling three-dimensional matrix after the dimension expansion is (4, 26, 26). When these two dimensions are aligned, for example, the latent space sampling information after the dimension expansion and the latent information of the structural features can be spliced ​​on the image layer dimension, and the dimension of the fused latent space sampling obtained after splicing will be (20, 26, 26). The reconstructed three-dimensional image ground truth of the simulated object to be tested can be spliced ​​on the image layer dimension with the feature data aligned with the dimension, so the matrix dimension of the image information and the fused data input to the posterior distribution encoding network 301 will be (32, 26, 26).

[0042] In addition, in the inference application stage of the reconstruction model after training is completed, the measurement data of the object to be tested must also be upsampled using the upsampling module 401 before being fed to the trained prior distribution encoding network 201 and the trained feature encoding network 202, which will not be repeated here.

[0043] In the simulation / modeling system used to generate simulated measurement data, for example, the extended form of the radiation transfer equation (RTE) of the first-order spherical harmonic function, namely the diffusion equation, can be used as a theoretical model for the propagation of photons in tissues. The diffusion equation in the continuous wave form is shown in equation (1): (1) in represents the position of each voxel, is the absorption coefficient, is the diffusion term, defined as ,in is the reduced scattering coefficient. Represents the light source item, indicating the position The light intensity at represents the light intensity distribution, Represents the speed of light in a vacuum.

[0044] The model shown in equation (1) considers the change in the light absorption coefficient in the tissue Defined as the value measured for each voxel or grid node and each source-detector pair Relative to baseline measurements The logarithmic ratio of is related to, as shown in formula (2): (2) in represents the position of the light source, represents the position of the detector, Indicates that the light source is at position The wavelength produced at The luminous flux of light is formally expressed as the Green function solution of the steady-state diffusion equation. Each measurement channel By voxel It is approximated by the discrete sum over , and is expressed using the following notations in Equations (3) and (4): (3) (4) By combining equations (2), (3) and (4), we can obtain the linear forward model, as shown in equation (5): (5) in, , , Y is the measurement domain of the boundary measurement, X is the image domain of the optical properties in the tissue, A is the sensitivity matrix, and each row of A represents the Next, the measured value Spatial sensitivity relative to the amount of change in the absorption coefficient per voxel within the tissue.

[0045] Based on the numerical simulation of photon transmission theory, a high-density light source and light source detector topology is set on the surface of the simulated object, and one or more absorbers are set in different ways inside the simulated object (equivalent to generating multiple different ground truth values ​​of reconstructed three-dimensional images). ), based on the forward model shown in equation (5) and the theoretical value of A, realistic and rich simulation measurement data (i.e., Corresponding ), a large number of pairs As training samples, supervised training of the reconstruction model will enable the reconstruction model to learn the high-precision representation ability of the inverse mapping of DOT measurement data, that is, to solve F -1 (Y)→X The ability to solve inverse problems, and through the reasonable setting of training samples and optimization of training methods, the generalization ability and robustness of the reconstruction model can be further enhanced.

[0046] Reference Figure 4 To illustrate the construction method of the first loss function and the second loss function. In some embodiments, the first loss function can be constructed, for example, in the following manner: based on the dimensionally aligned feature data, the prior distribution coding network 201 is used to generate the prior latent space parameter distribution information corresponding to the dimensionally aligned feature data; based on the dimensionally aligned feature data together with the reconstructed three-dimensional image ground truth, the posterior distribution coding network 301 is used to generate the posterior latent space parameter distribution information corresponding to the dimensionally aligned feature data; based on the prior latent space parameter distribution information and the posterior latent space parameter distribution information to construct the first loss function, as an example only, the KL divergence of the above two latent space parameter distribution information can be used as the first loss function. As mentioned above, the prior distribution coding network and the posterior distribution coding network can be constructed according to the multivariate Gaussian distribution probability model. In specific implementation, both the prior distribution coding network 201 and the posterior distribution coding network 301 can include multiple stacked modules such as convolutional layers, batch normalization layers, and activation function layers, so as to obtain multivariate Gaussian distribution information with corresponding mean and standard deviation based on the input information mapping.

[0047] In the first multivariate Gaussian distribution, the mean is μ proir , with standard deviation σ proir In the case of , a sample generated by the prior distribution encoding network 201 based on the dimensionally aligned feature data y can be expressed as follows (6): (6) The mean of the second multivariate Gaussian distribution is μ post , with standard deviation σ post In the case of Together with the reconstructed 3D image ground truth Both, so a sample generated can be expressed as follows (7): (7) At the beginning of training, for a specific training sample, the prior latent space parameter distribution information generated by the prior distribution encoding network and the posterior latent space parameter distribution information generated by the posterior distribution encoding network are different. Therefore, the training goal of the first loss function in the loss function is to minimize the KL divergence of the above two multivariate Gaussian distributions. That is, the first loss function can be expressed as the following formula (8): (8) The smaller the first loss function is, the closer the two multivariate Gaussian distributions will be, which means that the trained prior distribution encoding network can more accurately generate the latent space features associated with image reconstruction based on the input measurement data.

[0048] In some embodiments, the second loss function may be constructed based on the reconstructed three-dimensional image ground truth x and the mean square error (MSE) of the three-dimensional image of the simulated object to be tested reconstructed based on the spatial distribution of optical parameters of the simulated object to be tested output by the decoding network or other similar error functions. Taking MSE as an example, the second loss function may be expressed as the following formula (9): (9) in, It is the equivalent mapping function of the entire reconstruction network during the reconstruction model training process.

[0049] In some embodiments, the loss function may be defined as a weighted sum of a first loss function and a second loss function, for example, as expressed in the following formula (10): (10) in, It represents the weight coefficient between the two loss functions, which is used to adjust the relative proportion of the two parts of the loss. The specific value can be set as needed considering factors such as the convergence speed and the performance of the trained model. This application does not impose any restrictions on this.

[0050] In addition, the specific number of dimensions of the first multivariate Gaussian distribution and the second multivariate Gaussian distribution can be determined by experiments to determine the appropriate value, so that the first multivariate Gaussian distribution and the second multivariate Gaussian distribution with a specific number of dimensions can fully and non-redundantly characterize the distribution problem of optical transmission parameters in three-dimensional space, such as the position of the activation area inside the test body, the shape of the activation area, the number of activation areas, the activation degree of the activation area, etc. It is worth noting that since the samples generated based on the first multivariate Gaussian distribution and the second multivariate Gaussian distribution in the embodiment of the present application are all latent space sampling, therefore, the sample data of each dimension does not necessarily explicitly correspond to the above variables, or accurately correspond to the actual values ​​of these variables. Through the above analysis and the comparative experiments of different numbers of dimensions, it is shown that the number of dimensions of the first multivariate Gaussian distribution and the second multivariate Gaussian distribution in the embodiment of the present application is set to between 2 and 6 dimensions, which will make the reconstruction model have a wider applicability, avoiding the poor reconstruction effect caused by too few dimensions, and avoiding the problem of low model training efficiency, difficulty in convergence and even decreased model performance due to too many dimensions, and can achieve a better balance between reconstruction effect and training efficiency. In particular, when the number of dimensions is set to 4, the model performance is close to optimal with acceptable training and application costs.

[0051] Figure 5 FIG. 2 shows a schematic diagram of the composition structure of a feature encoding network according to an embodiment of the present application. Figure 5 As shown, a depth-aware attention mechanism is integrated into the feature encoding network 202 when the dimensionally aligned feature data enters the feature encoding network 202. Figure 5Before the convolution module of "Depth-aware attention"), it can first pass through two convolutional layers (Conv), a batch normalization layer (BN), and an activation function layer (ReLu). During the extraction of organized depth-aware attention, the maximum pooling layer (MaxPool) and the average pooling layer (AvgPool) are used to aggregate information of the same depth. Subsequently, the focus of the model on different depth layers is adjusted through two fully connected layers, and the Sigmoid activation function is applied to obtain the normalized weights for feature map adjustment. In addition, residual connections are added to these layers to prevent the problem of performance degradation caused by excessive network depth. For feature encoding networks, the number of network layers is a key parameter that affects the inverse mapping representation performed by the reconstruction model. Generally, the more layers, the stronger the representation ability, but it does not always show a linear positive correlation trend. Too few or too many layers may not achieve optimal performance. In particular, too many layers may also produce adverse results such as overfitting of the network model or gradient disappearance. The reduction ratio parameter of the feature in the tissue depth perception attention mechanism reflects the scale of fusion of features across different depths. If its value is too low, the attention mechanism may not be able to effectively extract the key information that needs attention. If its value is too high, it may hinder the effective recovery of details in the process of tissue depth perception feature refinement. The specific values ​​of the above hyperparameters (such as the reduction ratio) can be selected through experiments and jointly with other parameters, and this application does not impose any restrictions on this.

[0052] In the practical process of proposing the present application, a numerical simulation platform and a physical model experimental platform were also synchronously constructed to verify the reconstruction method of the embodiment of the present application, and a performance comparison evaluation was conducted in multiple scenes and multiple angles with two representative existing DOT image reconstruction models. One of them is a traditional DOT image reconstruction model based on inverse scattering theory that combines linear methods with regularization technology to constrain the solution space (referred to as Model 1), and the other is a DOT image reconstruction model based on deep learning (referred to as Model 2).

[0053] In the reconstruction method of the embodiment of the present application, the model hyperparameter combination of the reconstruction network is adjusted to the following optimal values ​​through experiments: the number of network layers in the feature encoding network is 10, the number of dimensions of the first multivariate Gaussian distribution / the second multivariate Gaussian distribution is set to 4, and the reduction ratio of the features in the organization depth perception attention mechanism is 4 to construct the final reconstruction network model.

[0054] Figure 6 The following is a schematic diagram showing the reconstruction results of the reconstruction method according to the present application under different types of media and target numbers. Figure 6In the experiment, the object to be tested was set to be a uniform medium and a layered medium, and the numerical simulation experimental results of single-target, dual-target and multi-target reconstruction using the reconstruction method of the present application and two other existing technologies were compared. Specifically, the single-target reconstruction experiment is used to evaluate the model's ability to reconstruct absorbers of different sizes and optical properties, the dual-target reconstruction experiment is used to evaluate the model's ability to distinguish different absorbers, and the multi-target reconstruction experiment is mainly used to test the generalization performance of the model. Figure 6 (a1)-(a3) correspond to single-target reconstruction experiments, (b1)-(b3) correspond to dual-target reconstruction experiments, and (c1)-(c3) correspond to multi-target reconstruction experiments. In the three types of experiments, (a1), (b1), and (c1) represent xy cross-sections at a depth of 15 mm, and the white dotted line represents the target shape contour at y=25 mm; (a2), (b2), and (c2) represent the reconstruction results of the corresponding items after applying the half-maximum full-width threshold; (a3), (b3), and (c3) represent the 3D reconstruction results of the corresponding items. The experiment normalized each reconstruction result based on the maximum absorption coefficient of the target absorber in the reconstructed image, and the normalized value is Figure 6 As shown in the legend on the right, a larger normalized value indicates a larger absorption coefficient and a higher activation level of the target absorber.

[0055] from Figure 6 From the first row of true image values, we can see that in the single-target reconstruction experiments (a1)-(a3), a cylindrical optical absorber with a radius of 7.5 mm and a height of 20 mm is located at the center of the imaging area at a depth of 20 mm. In the dual-target reconstruction experiments (b1)-(b3), two cylindrical absorbers with a radius of 5 mm and a height of 20 mm are located on both sides of the center of the imaging area at a depth of 20 mm, with a spacing of 10 mm between them. In the multi-target reconstruction experiments (c1)-(c3), multiple cylindrical absorbers with different radii and heights are randomly distributed throughout the imaging area.

[0056] like Figure 6 As shown in the reconstruction results in uniform media, Model 1 is unable to provide accurate reconstruction in single-target, dual-target and multi-target experiments. The overall boundary of its reconstructed image is significantly different from the ground truth and tends to shift to the shallow layer of the tissue. In contrast, Model 2 shows higher accuracy in reconstructing the shape and position of the absorber, and can basically effectively distinguish the foreground and background. In the uniform medium and single-target reconstruction experiments, the reconstruction method of the present application performs comparable to Model 2, and the reconstructed single template maintains a high degree of consistency with the image truth in shape, contour and tissue depth, indicating that both methods can effectively reconstruct a single relatively simple optical absorber with simple geometric shapes.

[0057] As the number of target absorbers increases, as shown in (b1)-(b3), both Model 2 and the reconstruction method of the present application can achieve dual-target reconstruction, but in comparison, Model 2 has artifacts in the foreground and background reconstruction, which results in its performance in the reconstruction of the optical characteristic parameters of the target absorber and the clarity of the target contour being lower than that of the reconstruction method of the present application. This shows that the attention mechanism introduced in the reconstruction method of the present application enhances the ability of the reconstruction model to focus on details.

[0058] In the multi-target reconstruction experiments shown in (c1)-(c3), both Model 2 and the reconstruction method of the present application accurately locate optical absorbers with significant variations. However, for smaller absorbers with lower optical properties that coexist, Model 2 tends to underestimate, resulting in invalid reconstruction. It is speculated that this problem occurs because Model 2's use of 3D convolutions tends to focus on the average level of the overall reconstruction, which is more susceptible to background effects when the absorber is small. In contrast, the reconstruction method of the present application combines the embedded features generated from the latent space with the attention mechanism, so that the reconstruction model has the ability to accurately capture the shape contours and reconstruct the target optical characteristic parameters in more complex target absorber conditions.

[0059] Compared with homogeneous media, Figure 6When reconstructing in layered media, the reconstructed images of the three methods all have quality degradation issues in aspects such as contrast between the target absorber and the background and shape recovery. Specifically, Model 1 still fails to accurately reconstruct in all three experiments, the reconstructed boundaries are significantly different from the ground truth and are significantly shifted to the shallow layer. In addition, after applying the half-width threshold, it still fails to filter out unimportant absorbers during reconstruction, the background contains a large number of artifacts, whose values ​​are comparable to the absorber being reconstructed, and the smoothness of the shape contour decreases. This shows that the over-reliance of Model 1 on the forward model limits its flexibility and adaptability during reconstruction. In contrast, Model 2 and the reconstruction method of the present application show significant advantages in reconstructing the shape and position of the absorber, while effectively suppressing background noise. Further, for single-target and dual-target reconstruction, Model 2 is not as effective as the reconstruction method of the present application in distinguishing the foreground from the background, and its reconstructed image has a large number of artifacts at the absorber boundary, resulting in increased fluctuations in the shape contour. In addition, compared with Model 2, the reconstruction method of the present application shows higher spatial resolution and clearly delineates the boundaries of the dual-target absorbers. For multi-target reconstruction, Model 2 can only provide a rough location of the absorber, but cannot accurately distinguish the boundaries between each absorber, and its sensitivity to smaller absorbers is significantly reduced, and there is a problem of overestimating the optical properties of larger absorbers. It is speculated that this phenomenon is due to the fact that when Model 2 is applied to layered medium data, it is difficult to effectively extract the absorber mode, resulting in limited performance. On the contrary, the reconstruction method of the present application shows strong adaptability, has good stability in shape recovery, absorber positioning and spatial resolution, and is least affected by changes in medium properties among the three methods. Further quantitative analysis of performance indicators also shows that the reconstruction method of the present application is less affected by the type of medium on all three data sets, and its reconstruction metric in layered media is even better than that of Model 2 in uniform media. These findings show that the reconstruction method of the present application not only performs well under uniform medium conditions, but also shows strong versatility and generalization between different media types, and has stronger practical value in more complex practical application scenarios.

[0060] In addition, considering the unavoidable ambient light interference and sensor measurement noise in diffuse optical tomography measurement data, this application specifically conducts comparative experiments on the performance of reconstruction methods under noisy conditions.

[0061] Figure 7 A schematic diagram showing the comparison of the multi-target image reconstruction performance of the reconstruction method of the embodiment of the present application under different signal-to-noise ratios is shown. Figure 7In the experiment, the reconstruction method of the present application is still considered to be compared with the aforementioned Model 1 and Model 2, wherein the signal-to-noise ratio (SNR) of the adopted multi-target data sets are 25dB, 15dB and 5dB respectively. In the embodiment of the present application, regarding the image reconstruction performance, an index system commonly used in the field of image processing and image quality assessment is adopted, including SSIM (Structural Similarity Index), PSNR (Peak Signal-to-Noise Ratio), CNR (Contrast-to-Noise Ratio), R value (Pearson Correlation Coefficient, also known as Pearson correlation coefficient), Jaccard index (Jaccard index), and MAE (Mean Absolute Error), the definitions, calculation methods and meanings of the index values ​​of which are well known to those skilled in the art and will not be repeated here. From Figure 7 It can be seen that as the noise level increases, the overall reconstruction performance of the reconstruction method of the present application shows a downward trend, but compared with Model 1 and Model 2, the reconstruction method of the present application shows higher SSIM, PSNR, CNR, R value and Jaccard index, as well as lower MAE, which indicates that it has superior noise resistance and robustness. In particular, even when the SNR is 5dB, the reconstruction results of the present application exceed the performance of Model 1 and Model 2 with SNRs of 15dB and 25dB, indicating that the reconstruction method of the present application has better anti-interference ability than the prior art.

[0062] In some embodiments of the present application, a reconstruction device for diffuse optical tomography is further provided, the reconstruction device comprising at least a processor and a memory, the memory storing computer executable instructions, and the processor executing the reconstruction method for diffuse optical tomography described in each embodiment of the present application when executing the computer executable instructions.

[0063] The processor may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0064] In some embodiments of the present application, a near-infrared brain function imaging system is provided, comprising a near-infrared optical data acquisition device and a reconstruction device for diffuse light tomography; wherein the near-infrared optical data acquisition device is used to acquire near-infrared data of an object; and the reconstruction device is used to use the near-infrared data to execute the reconstruction method for diffuse light tomography described in each embodiment of the present application.

[0065] The reconstruction method, device, system and medium for diffuse light tomography provided in each embodiment of the present application first use high-density diffuse light tomography training data generated based on the anatomical information of the object to be tested, and with the assistance of the posterior distribution coding network, enable the prior distribution coding network constructed based on the multivariate Gaussian distribution probability model to learn the ability to map the measurement data into latent space parameter distribution information, and enable the feature coding network with an attention mechanism to learn the ability to extract structural feature information including depth information from the measurement data, and in the reconstruction model reasoning application stage, only the prior distribution coding network, the feature coding network and the decoding network are used to complete the high-precision reconstruction of the spatial distribution of the optical transmission parameters of the object to be tested, which provides a new approach for high spatial resolution near-infrared brain functional imaging, and the experimental results show that the reconstruction performance is significantly better than a variety of existing DOT image reconstruction technologies, which is of great significance and has great application potential for promoting fNIRS technology in early screening, diagnosis and prognosis evaluation of brain diseases and other application scenarios.

[0066] This application describes various operations or functions that may be implemented as software codes or instructions or defined as software codes or instructions. Such content may be source code or differential code ("incremental" or "patch" code) that can be directly executed ("object" or "executable" form). Software codes or instructions may be stored in a computer-readable storage medium, and when executed, may cause a machine to perform the described functions or operations, and include any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as recordable or non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0067] The exemplary methods described in this application may be implemented at least in part by a machine or computer. In some embodiments, a computer-readable storage medium stores computer program instructions, which, when executed by a processor, cause the processor to perform the operations of the reconstruction method for diffuse optical tomography described in various embodiments of this application.

[0068] The implementation of such a method may include software code, such as microcode, assembly language code, high-level language code, etc. Various software programming techniques may be used to create various programs or program modules. For example, a program portion or program module may be designed with or by means of Java, Python, C, C++, assembly language, or any known programming language. One or more of such software portions or modules may be integrated into a computer system and / or a computer-readable medium. Such software code may include computer-readable instructions for executing various methods. The software code may form a part of a computer program product or a computer program module. In addition, in an example, the software code may be tangibly stored on one or more volatile, non-temporary or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (such as optical disks and digital video disks), cassettes, memory cards or memory sticks, random access memories (RAMs), read-only memories (ROMs), etc.

[0069] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims are to be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the practice of this application, and the examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0070] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more schemes thereof) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the present application. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein as examples or embodiments in the specific embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.

[0071] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A reconstruction method for diffuse optical tomography, characterized in that: The reconstruction method comprises, using a processor, based on the measurement data obtained after measuring the object to be measured using diffuse light tomography technology, and using a reconstruction model including a prior distribution encoding network, a feature encoding network and a decoding network, to reconstruct the optical parameter spatial distribution of the object to be measured, specifically comprising: Based on the measurement data, using a trained prior distribution encoding network, generating prior latent space parameter distribution information corresponding to the measurement data, and generating latent space sampling based on the prior latent space parameter distribution information; Based on the measurement data, using a trained feature encoding network, extracting implicit information of structural features in the measurement data; Fusing the latent space sampling and the latent information of the structural features to generate a fused latent space sampling; Based on the fused latent space sampling, the spatial distribution of the optical transmission parameters of the object to be measured is reconstructed by using the decoding network to generate a reconstructed three-dimensional image of the object to be measured.

2. The reconstruction method according to claim 1, characterized in that: The feature encoding network includes a convolution module, and the reconstruction method further includes, using the processor: introducing an attention mechanism in the convolution module of the feature encoding network, and outputting an attention weight matrix associated with the tissue depth as implicit information of the structural features in the measurement data.

3. The reconstruction method according to claim 1, characterized in that: The reconstruction method further includes, using the processor: introducing a posterior distribution coding network in the training stage, using training samples in the training sample set, and based on a loss function composed of a weighted sum of a first loss function and a second loss function, collaboratively training the prior distribution coding network, the feature coding network, the decoding network, and the posterior distribution coding network.

4. The reconstruction method according to claim 3, characterized in that: The prior distribution and the posterior distribution are constructed using a multivariate Gaussian distribution probability model with the same number of dimensions, the prior latent space parameter distribution information generated by the trained prior distribution encoding network follows a first multivariate Gaussian distribution; the posterior latent space parameter distribution information generated by the trained posterior distribution encoding network follows a second multivariate Gaussian distribution; and, the first multivariate Gaussian distribution and the second multivariate Gaussian distribution.

5. The reconstruction method according to claim 4, characterized in that: The dimensions of the first multivariate Gaussian distribution and the second multivariate Gaussian distribution are 4.

6. The reconstruction method according to claim 3, characterized in that: The training samples are obtained as follows: Based on the anatomical information of the object to be tested, a plurality of simulated objects to be tested with ground truth values ​​of reconstructed three-dimensional images for training are constructed, and simulated measurements are performed on each simulated object to generate corresponding simulated measurement data; The simulated measurement data and the reconstructed 3D image ground truth of each simulated object to be tested are used as training samples.

7. The reconstruction method according to claim 6, characterized in that: The reconstruction method further includes, before feeding the simulated measurement data of the training samples to the prior distribution encoding network, the feature encoding network and the posterior distribution encoding network, upsampling the simulated measurement data according to the voxel resolution of the three-dimensional image to be reconstructed to generate dimensionally aligned feature data.

8. The reconstruction method according to claim 7, characterized in that: The first loss function and the second loss function are constructed as follows: Based on the dimensionally aligned feature data, using the prior distribution encoding network, generating prior latent space parameter distribution information corresponding to the simulated measurement data; Based on the dimensionally aligned feature data together with the reconstructed three-dimensional image ground truth, using the posterior distribution encoding network, generating posterior latent space parameter distribution information corresponding to the simulated measurement data; Constructing the first loss function based on the KL divergence of the prior latent space parameter distribution information and the posterior latent space parameter distribution information; A second loss function is constructed based on a mean square error of the reconstructed three-dimensional image ground truth and the reconstructed three-dimensional image of the simulated object to be tested.

9. The reconstruction method according to claim 1, characterized in that: The fusion processing of the latent space sampling and the latent information of the structural feature specifically includes: according to the three dimensions of the latent information of the structural feature, the latent space sampling is expanded and dimensionally aligned in two dimensions other than the image layer; the latent space sampling information after expansion and dimension alignment is spliced ​​with the latent information of the structural feature in the image layer dimension.

10. The reconstruction method according to claim 6, characterized in that: Performing simulation measurement on each simulated object to be tested to generate corresponding simulation measurement data specifically includes: A topological structure consisting of a light source and a light source detector of diffuse light imaging technology is set on the surface of each simulated object to be tested, wherein the light source is used to emit incident light to each simulated object to be tested, and the light source detector is used to detect the outgoing light, and a detection channel is formed between the light source detector and at least part of the light source; When there is an absorber in each simulated test body, simulated measurement data is constructed based on the change in optical transmission parameters of each simulated test body and the change in output light parameters of each detection channel arranged in a preset order, wherein the absorber has an absorption effect on the light emitted by the light source, and the voxel resolution of the simulated test body is between 1mm and 2mm.

11. The reconstruction method according to any one of claims 1 to 10, characterized in that: The diffuse light tomography technique involves the use of near infrared light.

12. A reconstruction device for diffuse optical tomography, characterized in that: The reconstruction device at least comprises a processor and a memory, wherein the memory stores computer executable instructions, and the processor executes the reconstruction method for diffuse optical tomography according to any one of claims 1 to 11 when executing the computer executable instructions.

13. A near-infrared brain function imaging system, characterized in that: It includes a near-infrared optical data acquisition device and a reconstruction device for diffuse light tomography; wherein, The near-infrared optical data acquisition device is used to collect near-infrared data of the object; The reconstruction device is used to perform the reconstruction method for diffuse optical tomography according to any one of claims 1 to 11 using the near-infrared data.

14. A non-transitory computer-readable storage medium storing a program, the program causing a processor to perform the operations of the reconstruction method for diffuse optical tomography according to any one of claims 1 to 11.

Images