A method for evaluating the quality of biological tissue anatomy based on multimodal data fusion

Abstract

The present invention relates to the technical field of material physical and chemical property testing and analysis, and in particular to a method for evaluating the anatomical quality of biological tissues based on multimodal data fusion. The method comprises the following steps: step 1: synchronous acquisition of spatially registered multimodal data, synchronously acquiring data in the same coordinate system through a spatial registration device of an integrated optical path; resampling the mechanical distribution map and the topological model to a spatial grid of hyperspectral data based on a coordinate mapping relationship; step 2: pathology-guided feature fusion; step 3: generation of an anatomical quality index, inputting the fused feature vector into a pre-trained deep residual network, outputting a quality evaluation index QI, generating an anatomical quality heat map based on the resampled spatial grid coordinates, and marking the QI value of each region. By realizing the precise synchronous acquisition of multimodal data, pathology-guided feature extraction and fusion, and quality evaluation of a deep learning network, the method can more accurately and comprehensively evaluate the anatomical quality of biological tissues.

Description

Technical Field

[0001] The present invention relates to the technical field of material physical and chemical property testing and analysis, and in particular to a biological tissue anatomical quality assessment method based on multimodal data fusion. Background Art

[0002] In the field of biological tissue anatomical quality assessment (such as surgical operations, pathological examinations, and medical education), existing technologies have multi-dimensional technical bottlenecks that restrict the accuracy and practicality of the assessment results.

[0003] First, single-modality assessment methods suffer from incomplete information dimensions: while hyperspectral imaging can capture tissue chemical composition (such as hemoglobin oxygenation status), it cannot simultaneously reflect mechanical properties (such as elastic modulus) and three-dimensional topography (such as cut surface roughness), leading to a one-sided assessment. For example, in the assessment of electrosurgical thermal injury, it is difficult to distinguish tissue hardening from inflammatory response based solely on spectral data, with a false-positive rate exceeding 35%. While micromechanical testing (such as microindentation) can quantify local hardness, contact measurements compromise tissue integrity, and sparse sampling cannot cover the entire area. Three-dimensional topography scanning can characterize surface geometry, but lacks correlation analysis between subcutaneous chemical state and mechanical properties, making it impossible to explain the causes of deformation (such as mechanical tearing or thermal damage). This problem stems from the fact that the anatomical quality of biological tissue is essentially the result of the synergistic interaction of chemical activity, mechanical properties, and geometric topography. A single modality can only reflect local information, and the fragmentation of multidimensional data inevitably leads to inaccurate assessments.

[0004] Secondly, multimodal data fusion faces the dual drawbacks of spatial registration and feature association. First, differences in coordinate systems between devices can lead to registration errors exceeding 50μm, making cellular-level spatial alignment impossible. For example, registration between mass spectrometry images and pathology sections relies on manual labeling, making it difficult to accurately correlate chemical, mechanical, and topographic features at the same location. Second, traditional feature fusion methods (such as early / late fusion) simply splice multimodal data without filtering for biomarkers strongly associated with anatomical quality. For lung cancer bone metastasis prediction, directly splicing CT images with pathological features leads to an over-reliance on imaging and neglect of cytological information, resulting in a 12% decrease in sensitivity. Spatial omics-enhanced models (such as soScope) improve resolution but fail to establish a pathological correlation between topographic roughness and elastic modulus gradients, making it impossible to locate the root cause of the injury. This issue is crucial because accurate evaluation requires understanding the biological connections between multimodal features (e.g., collagen degradation leading to a decrease in elastic modulus). Non-aligned, unfiltered fusion misses key pathological logic.

[0005] Finally, dynamic assessment and real-time feedback mechanisms are lacking: existing methods are mostly limited to static offline analysis, with mechanical testing and topographic scanning taking over 10 minutes per sample. This makes it impossible to quickly output quality indices during surgery, and without a response model that integrates multimodal features with operational parameters (such as tool temperature and feed rate), it is difficult to generate real-time optimization solutions. This deficiency is essential because dissection quality must be dynamically correlated with the operational process, and static assessment cannot support real-time intraoperative decision-making (such as adjusting electrosurgical scalpel power to avoid thermal damage).

[0006] Therefore, there is an urgent need for a biological tissue anatomical quality assessment method based on multimodal data fusion to solve the above problems. Summary of the Invention

[0007] Based on the above objectives, the present invention provides a method for evaluating the quality of biological tissue anatomy based on multimodal data fusion, comprising:

[0008] Step 1: Synchronous acquisition of multimodal data with spatial registration. The following data are collected synchronously in the same coordinate system using a spatial registration device with an integrated optical path:

[0009] Hyperspectral data cube: Scan the anatomical surface to obtain spectral reflectance data with spatial coordinates and wavelength dimensions;

[0010] Tissue mechanical distribution map: Microprobe array indentation testing is used to calculate the elastic modulus value at each point based on the displacement-load curve;

[0011] 3D surface topology model: Obtain submicron-level topography data through laser scanning;

[0012] Based on the coordinate mapping relationship, the mechanical distribution map and topological model are resampled to the spatial grid of the hyperspectral data;

[0013] Step 2: Pathology-guided feature fusion: extract hemoglobin oxygenation parameters and collagen absorption features from the hyperspectral data, extract surface roughness indicators and local curvature changes of the 3D topological model, and extract elastic modulus gradient values ​​and viscoelastic response parameters from the mechanical distribution map;

[0014] The hemoglobin oxygenation parameters and collagen absorption characteristics, the surface roughness index and local curvature variation of the three-dimensional topological model, the elastic modulus gradient value and the viscoelastic response parameters are combined into a high-dimensional matrix, and the dimensionality is reduced to a fusion feature vector that is strongly correlated with the pathological score through supervised manifold learning.

[0015] Step 3: Anatomical quality index generation: input the fused feature vector into the pre-trained deep residual network, output the quality assessment index QI, generate the anatomical quality heat map based on the resampled spatial grid coordinates, and mark the QI value of each area.

[0016] Beneficial effects of the present invention:

[0017] 1. This invention utilizes multimodal data fusion, combining hyperspectral imaging, microindentation testing, and 3D topography scanning data to ensure simultaneous acquisition of chemical composition, mechanical properties, and geometric topography information. Hyperspectral imaging provides chemical composition, microindentation testing provides local hardness, and 3D topography scanning provides surface roughness and morphological characteristics. These three complementary approaches avoid the one-sidedness and incompleteness of information derived from a single modality.

[0018] 2. This invention utilizes multimodal data registration technology and advanced algorithms to solve the spatial alignment problem between different modalities, reducing registration errors and enabling precise spatial alignment of <50μm, ensuring high-precision registration at the cellular level. Specifically, in the registration process between mass spectrometry imaging and pathology sections, the use of CCD-assisted laser marking technology effectively reduces manual marking errors, ensuring that chemical, mechanical, and topographic data are accurately associated with the same location, thereby improving data fusion accuracy and biomarker identification capabilities.

[0019] 3. This invention uses a biologically driven feature screening and fusion method to first screen out key biomarkers that are strongly correlated with tissue anatomical quality. It then uses a manifold learning algorithm to extract the intrinsic connections between multimodal features, thereby optimizing the fusion effect of multimodal data. In the application of predicting lung cancer bone metastasis, by optimizing the feature fusion model, the traditional method avoids the excessive reliance on imaging information, ensures the integration of cytological information, and improves the sensitivity of the model. This feature screening and fusion method effectively improves the accuracy and credibility of the assessment.

[0020] 4. This invention enables immediate assessment of tissue dissection quality during surgery through real-time data processing and feedback mechanisms. This method collects multimodal data in real time and rapidly generates a quality index (QI) and heat map using a deep learning model, providing surgeons with real-time decision support.

[0021] 5. This invention utilizes non-destructive mechanical property measurement technology, combined with spatial registration techniques and interpolation algorithms, to recover the global mechanical distribution from sparse mechanical test data. High-precision 3D reconstruction technology, based on interpolation and supplementation of microindentation data, enables the generation of a full-view elastic modulus distribution map without compromising tissue integrity, thereby improving the spatial resolution of mechanical property assessment.

[0022] 6. This invention constructs a comprehensive assessment model that incorporates chemical, mechanical, and morphological characteristics through correlation analysis of multimodal data and pathological mechanisms. This model can automatically identify tissue damage types and explain the causes of damage through biological reasoning. This mechanism enhances the biological explanatory power of assessment results, enabling more precise identification of the root causes of tissue damage and providing more instructive assessment results for clinical practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 is a flow chart of the steps of the method of the present invention;

[0025] Figure 2 Flow chart of the steps of obtaining the displacement-load curve in step 1 of the method of the present invention;

[0026] Figure 3 The figure is a flowchart of the steps for determining the critical value and setting the threshold value of the method of the present invention. DETAILED DESCRIPTION

[0027] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. It is also noted that, to provide a more detailed description, the following embodiments are best and preferred embodiments, and those skilled in the art may employ alternative methods for implementing certain known technologies. Furthermore, the accompanying drawings are intended only to provide a more detailed description of the embodiments and are not intended to limit the present invention.

[0028] See Figure 1-Figure 3 The present invention provides a method for assessing the quality of biological tissue dissections based on multimodal data fusion. In step 1, the key technology is to achieve spatial registration and synchronized acquisition of multimodal data. Using a spatial registration device with an integrated optical path, the different data modalities are accurately aligned within the same spatial coordinate system, preventing registration errors from affecting subsequent analysis.

[0029] Specifically, hyperspectral imaging technology is used to scan anatomical surfaces, acquiring reflectance data at different wavelengths and simultaneously collecting spatial coordinate information. This reflectance data reflects the chemical composition of the tissue, particularly physiological parameters such as hemoglobin oxygenation status, providing chemical-dimensional information for assessment.

[0030] Microprobe array indentation testing involves locally indenting the tissue surface and obtaining a displacement-load curve. The elastic modulus at each point is calculated from this curve, reflecting the mechanical properties of the tissue. This mechanical data can be used to assess physical properties such as hardness and rigidity.

[0031] Laser scanning technology is used to acquire submicron-level topographic data of the tissue surface and construct a three-dimensional topological model of the tissue surface. This data describes the geometric features of the tissue surface, such as roughness and local curvature.

[0032] Based on the spatial coordinate mapping relationship, the data of the mechanical distribution map and the three-dimensional topological model are resampled to the spatial grid of the hyperspectral data, so that data of different modalities can be compared and analyzed in the same coordinate system to ensure the spatial consistency of the information.

[0033] This step enables the accurate and simultaneous acquisition of multimodal data encompassing hyperspectral, mechanical, and geometric features, resolving the issues of large registration errors and information fragmentation associated with traditional methods. It effectively ensures precise spatial alignment of the data, laying a solid foundation for subsequent fusion analysis.

[0034] During feature extraction, features related to anatomical quality, such as hemoglobin oxygenation and collagen absorption characteristics, are first extracted from the hyperspectral data. These biomarkers provide information on the chemical activity of tissues and help reveal their health.

[0035] Furthermore, the roughness index and local curvature variation of the tissue surface are extracted from the three-dimensional topological model, which further reflects the geometric morphology and surface characteristics of the tissue, especially the indicators related to damage or deformation.

[0036] Furthermore, elastic modulus gradients and viscoelastic response parameters are extracted from the mechanical distribution map. These features help to assess the hardness changes of tissues and possible damage areas.

[0037] The extracted features (chemical, mechanical, and geometric) are combined into a high-dimensional matrix. Using a supervised manifold learning algorithm, these high-dimensional features are reduced in dimensionality to produce a fused feature vector that strongly correlates with the pathology score. This intelligent feature dimensionality reduction process extracts key factors influencing tissue anatomical quality, avoids interference from irrelevant features, and improves assessment accuracy.

[0038] This step maximizes the preservation of critical information about tissue anatomical quality by organically integrating multimodal data features. Pathology-guided feature selection and manifold learning optimize the feature extraction process, ensuring more accurate and efficient data analysis. The resulting fused feature vector has strong pathological relevance, better reflecting tissue health and potential lesion areas.

[0039] The fused feature vectors were fed into a pre-trained deep residual network (ResNet) model for quality assessment. Deep residual networks can effectively extract deep patterns from complex multimodal data and generate an anatomical quality index (QI), which provides a numerical basis for quantitatively assessing tissue anatomical quality.

[0040] The resampled spatial grid coordinates, combined with the calculated QI values, generate a heat map of anatomical quality. This heat map displays the QI values ​​for each region, using color coding to allow physicians to intuitively visualize the spatial distribution of tissue quality and potential areas of damage.

[0041] The quality index generated by the deep residual network provides a quantitative and repeatable evaluation standard for tissue anatomical quality. Simultaneously, the generation of a heat map clearly illustrates the spatial distribution of anatomical quality, providing a scientific basis for real-time decision-making during surgery. This method not only improves assessment efficiency but also enables surgeons to make more precise operational adjustments based on the quality index and heat map, thereby enhancing surgical safety and effectiveness.

[0042] In one possible embodiment, the spatial registration device includes a biological tissue stage with a three-dimensional translation stage. This stage enables precise movement of tissue samples in the X, Y, and Z directions, ensuring accurate positioning of tissue specimens between acquisition systems using different modalities (such as hyperspectral imaging systems and microprobe systems). High-precision stage movement is essential for ensuring accurate data acquisition and subsequent registration.

[0043] The laser locator projects cross-shaped fiducial markers onto the anatomical surface of the tissue, providing reference points for subsequent calibration. The laser locator operates in the same optical path as the hyperspectral imaging system, ensuring that the projected fiducial markers are spatially aligned with the hyperspectral image data. This allows the projected cross-shaped markers to be precisely aligned with the tissue position on the anatomical surface, ensuring synchronization and accuracy for spatial registration.

[0044] The microprobe incorporates a CCD sensor to capture the position of the marker in real time. When the laser locator projects a fiducial crosshair, the CCD sensor accurately detects the marker's position within the probe's coordinate system and records the corresponding pixel coordinates. The CCD sensor's high-resolution imaging enables real-time acquisition of the marker's spatial coordinates, providing reliable data support for the registration process.

[0045] First, calculate the pixel coordinates (X h , Y h ), these coordinates are obtained by the hyperspectral imaging system. Then, the pixel coordinates (X c , Y c Then, the coordinate transformation matrix is ​​solved. The specific steps are to fit the coordinates (X h , Y h ) to the coordinates (X c , Y c ), and then obtain the affine transformation parameters. The affine transformation parameters can realize the spatial mapping relationship between the hyperspectral image coordinates and the CCD sensor coordinates, thereby providing an accurate transformation model for subsequent multimodal data registration.

[0046] Ultimately, by solving the affine transformation matrix, the hyperspectral image and the image data captured by the microprobe sensor can be accurately registered in space, so that the calibration coordinate systems of the two are aligned, thereby ensuring that multimodal data (such as spectral data and mechanical data) are synchronously collected and matched with each other in the same coordinate system.

[0047] Through precise spatial registration solutions, the reliability and accuracy of multimodal data fusion are greatly improved, providing efficient and high-quality data support for subsequent anatomical quality assessment.

[0048] In one possible embodiment, during the loading phase, it is first necessary to apply a normal force to the target load value. The increase in the load should follow a constant rate, so as to ensure that the loading process of the tissue sample is stable and controllable. When setting the target load, the target load value is dynamically set according to the tissue type, taking into account the characteristics of different types of biological tissues, such as elasticity, stiffness, tissue tolerance, etc. The specific setting method is to pre-test similar tissue samples and record the minimum load required to make the displacement reach the set proportion of cell diameter. The proportion of cell diameter is used as a standard for load setting and can be predicted based on the microstructural characteristics of the tissue, so that the change in displacement can accurately reflect the elasticity and mechanical properties of the tissue.

[0049] Once the target load is reached, the system enters the hold phase. The target load is maintained until the rate of change of displacement falls below a critical threshold. This threshold is determined by statistically analyzing the minimum second-order derivative of the displacement decay curve. The displacement decay curve charts the time-varying displacement during load application. The minimum second-order derivative reflects a flattening of the displacement rate, indicating that the tissue has reached a stable internal deformation state under constant load. Therefore, when the rate of change of displacement falls below this critical value, the hold phase ends and preparations begin for the unloading phase.

[0050] The unloading phase involves removing the applied load at the same rate as during the loading phase, until the load reaches zero. This ensures that the unloading process maintains the same rate as the loading process, ensuring that the stress state of the biological tissue sample is fully and uniformly assessed throughout the entire load-hold-unloading process. The unloading phase effectively reflects the elasticity and deformation recovery capacity of the tissue, providing key information about tissue resilience.

[0051] This displacement-load curve acquisition method can effectively evaluate the mechanical properties of tissues through precise load setting, displacement change monitoring and a stable unloading process, ensure accurate assessment of tissue anatomical quality, and provide strong data support for the research and application of biological tissues.

[0052] In one possible embodiment, near-infrared spectroscopy (NIR) is widely used to analyze the composition of biological tissues during collagen analysis. Collagen has specific absorption characteristics in the NIR band, typically manifested as characteristic absorption peaks. First, a biological tissue sample is scanned using NIR spectroscopy, and the sample's reflectivity data in the NIR band is recorded. This reflectivity data is then analyzed to identify the characteristic absorption peaks of collagen, which are typically associated with molecular vibrations. Accurate identification of these peaks lays the foundation for subsequent quantitative analysis of collagen content.

[0053] Once a characteristic absorption peak has been identified, the next step is to calculate the peak area. The peak area is calculated by integration and represents the sample's absorbance at that wavelength. To quantify the relative intensity of the peak, comparison with the baseline area is necessary. The baseline area is calculated by integrating the spectral data on either side of the peak, defining a predefined wavenumber window (i.e., selecting a wavelength range on either side of the peak). The integral of these two areas represents the spectral reflectance of the baseline region.

[0054] The ratio of the absorption peak area to the baseline area (often referred to as the relative area of ​​the absorption peak) can be used as an important indicator for measuring collagen content. A larger ratio generally indicates a higher collagen content, as a larger absorption peak area reflects more light energy absorbed by collagen molecules.

[0055] The specific boundary window of the absorption peak requires statistical analysis based on the spectral data of the in vitro sample library. This step first requires analyzing the spectra of all tissue samples in the sample library, calculating and extracting the derivative extremes of the reflectance data. These extremes typically correspond to the start and end of the absorption peak. By statistically analyzing the distribution of these derivative extremes, the absorption peak window range can be determined. This window range will vary depending on the type of tissue, thus providing a certain degree of flexibility and adaptability.

[0056] In one possible implementation, during supervised manifold learning, the features of the biological tissue must first be weighted. Feature weighting is based on the correlation between each feature and the pathology score. Specifically, the correlation coefficient (e.g., Pearson correlation coefficient or Spearman rank correlation coefficient) can be calculated between each feature dimension and the pathology score. The absolute value of each feature's correlation coefficient is then used as the feature's weight. A larger weight indicates a stronger relationship between the feature and the pathology score, while a smaller weight indicates a weaker relationship. The purpose of feature weighting is to highlight features that are closely correlated with the pathology score and to increase the model's sensitivity to key features.

[0057] The fused feature vector of each sample is used as a node in the graph. Each node represents the feature information of a sample, which includes the fusion results of data from multiple modalities.

[0058] To capture the similarity between samples, nodes are connected based on the Euclidean distance in the pathology scoring space. If the Euclidean distance between two samples in the pathology scoring space is less than a preset radius, a connecting edge is established in the neighborhood graph. In this way, the neighborhood graph can reflect the similarity between samples in the pathology scoring space. The constructed neighborhood graph helps learn the local structure between samples, thereby better revealing the inherent laws of biological tissue.

[0059] After the neighborhood graph is constructed, the next step is to perform spectral analysis on the graph. First, a weighted Laplacian matrix is ​​calculated based on the adjacency matrix constructed from the graph. The weighted Laplacian matrix can reflect the local structural characteristics of the samples by encoding the similarities between the samples.

[0060] By solving the eigenvalues ​​and eigenvectors of the weighted Laplacian matrix, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected as the low-dimensional embedding result. The value of k here is determined by the cumulative contribution rate of the eigenvalues. When the cumulative contribution rate of the eigenvalues ​​exceeds a set threshold, the eigenvectors corresponding to the first k eigenvalues ​​are selected. The purpose of this step is dimensionality reduction. By retaining the largest eigenvalue, the information that best expresses the similarity between samples is extracted, thereby reducing the dimensionality of the data and improving the computational efficiency of the model.

[0061] In one possible implementation, the number of nodes in the input layer of a deep residual network (DRN) equals the dimension of the feature vector obtained after multimodal data fusion. The resulting feature vector incorporates information from different data modalities (e.g., imaging, genetic data, etc.), which is fed into the neural network in the input layer. In this way, the DRN can incorporate comprehensive features from multiple modalities to achieve more accurate anatomical quality assessment of biological tissues.

[0062] Residual blocks are a crucial component of deep residual networks. Each residual block consists of multiple fully connected layers, with the output dimensions of each layer decreasing in geometric order. This design aims to optimize computational efficiency by gradually reducing the network's parameter dimensions and help the network focus on important feature representations. Skip connections are a core design feature of residual networks, allowing inputs to jump directly to deeper layers of the network, alleviating the vanishing gradient problem and making the network easier to train.

[0063] The output dimension of each layer decreases in a geometric sequence, that is, the output dimension of each layer is smaller than that of the previous layer, which helps to gradually abstract the input features and extract more efficient and compact feature representations.

[0064] The output layer of the deep residual network uses a sigmoid activation function and contains only one output node. The output range of the sigmoid activation function is 0 to 1, making it suitable for binary classification problems. In this method, the output node predicts the anatomical quality score (or binary class label) of the biological tissue. The sigmoid function compresses the network's final output to between 0 and 1, facilitating subsequent classification or regression analysis.

[0065] The Huber loss function is a hybrid loss function that combines the advantages of the mean squared error (MSE) and the absolute error (MAE). It uses the mean squared error for small errors and the absolute error for large errors. Specifically, the Huber loss function uses a quadratic loss when the error is less than a threshold and a linear loss when the error is greater than the threshold. This can reduce the impact of outliers on the model and improve robustness.

[0066] The threshold parameter of the Huber loss function (usually denoted as δ) is adjusted by the median of the prediction errors of the in vitro samples. This process automatically selects the most appropriate threshold by analyzing the distribution of prediction errors of the in vitro samples, making the loss function more tolerant to data points with large errors and optimizing the performance of the network during training.

[0067] In one possible embodiment, in order to generate a complete anatomical quality heat map, it is necessary to estimate the QI values ​​of those grid points that are not directly measured. Spatial interpolation methods, especially radial basis function interpolation, are used in this scenario. RBF interpolation is a commonly used smooth interpolation method that predicts the QI value of each unmeasured point by interpolating the value based on the distance of the data point. RBF interpolation works well when processing non-uniformly distributed data and can provide smooth and coherent anatomical quality prediction values. The basic principle is to obtain the predicted value by interpolating the distance-weighted sample data with the function, which is crucial for the continuity of the anatomical image of biological tissue.

[0068] The QI value (Anatomical Quality Index) is a measure of the anatomical quality of biological tissue. To effectively transfer the QI value to the heat map, it needs to be normalized to a color index value. The normalization process is completed by converting the QI value to a certain range (usually 0 to 1). This ensures that the various levels of the QI value can be accurately mapped to the different color intervals of the pseudo-color image.

[0069] The color map used in the pseudocolor mapping is nonlinear, meaning that different ranges of QI values ​​correspond to different color gradients. In this mapping, the color segments corresponding to low QI values ​​(indicating poor anatomical quality) are intensified. This makes low-quality areas more prominent in the heat map, helping analysts focus on these areas.

[0070] To optimize heatmap performance, the infill range for low-QI segments is determined by analyzing a histogram of the QI distribution within the sample library. Specifically, the troughs are the sparsest areas in the QI histogram, and these locations are used to define the infill range for low-QI segments. This approach allows heatmaps to more accurately display areas of poor quality, helping scientists and medical experts quickly identify problem areas.

[0071] In one possible embodiment, in the heat map, the QI value indicates the level of anatomical quality. When the QI value is lower than the dynamic threshold, it indicates that the anatomical quality of the biological tissue in the area is poor and there may be defects. The determination of the dynamic threshold is based on the αth quantile of the QI value distribution of normal areas in the sample library, which ensures that the threshold has a certain degree of dynamic adaptability. By setting the parameter α, clinical experts can adjust the threshold according to different false positive rate requirements to adapt to different clinical needs. For example, when a lower false positive rate is required, α can be set to a lower quantile to ensure a more stringent quality standard.

[0072] For identified defect areas, multimodal features related to the area are first extracted, such as hemoglobin oxygenation parameters and elastic modulus. Hemoglobin oxygenation parameters reflect the oxygenation status of the tissue, while elastic modulus reflects changes in tissue stiffness and elasticity. If the product of the magnitude of change in these two features exceeds a certain threshold, the area is considered thermally damaged. This analysis method is based on the typical physiological characteristics of thermal damage: areas of thermal damage are typically accompanied by decreased oxygenation and increased tissue stiffness.

[0073] Another type of defect may be caused by mechanical tearing, which typically manifests as an increase in tissue surface roughness and a sudden change in the elastic modulus gradient. By calculating the correlation coefficient between the surface roughness increment and the extreme value of the elastic modulus gradient, if the correlation coefficient exceeds a set threshold, the area is identified as a mechanical tear. This method exploits the surface irregularities and hardness gradient changes that often accompany mechanical tearing.

[0074] This analysis allows for attribution analysis of low QI areas identified in the thermal map, determining their specific defect type (such as thermal damage or mechanical tear), and providing information support for further diagnosis or treatment. By continuously adjusting the relevant thresholds in the analysis model, the attribution process can be optimized, improving the accuracy of identifying different defect types.

[0075] By introducing a dynamic threshold setting, the method can be adaptively adjusted based on the distribution of actual data. The αth quantile is determined using the distribution of QI values ​​in normal regions of the sample library, ensuring that the threshold meets clinical misclassification requirements while also flexibly addressing the characteristics of different biological tissues. This avoids overly rigid threshold settings and enhances the method's universality and accuracy.

[0076] By comprehensively analyzing multiple modal data, including hemoglobin oxygenation parameters, elastic modulus, and surface roughness, a more comprehensive and accurate assessment of tissue quality can be provided. Each modal data element provides distinct biophysical properties, and combining this information allows for more precise identification of tissue defects, such as thermal damage and mechanical tears. This multimodal data fusion analysis effectively enhances the system's ability to identify complex lesions.

[0077] Thermal damage and mechanical tears exhibit distinct characteristics in biological tissue, making it difficult to clearly distinguish them using traditional single-modal data. Combining multiple modal features for analysis allows for more precise attribution of defect types. For example, thermal damage is primarily identified by changes in oxygenation parameters and elastic modulus, while mechanical tears are identified by correlations between roughness and elastic modulus gradients. This attribution analysis approach can effectively reduce misdiagnosis and improve the accuracy and reliability of anatomical quality assessments.

[0078] This method can effectively improve the accuracy of anatomical quality assessment by accurately identifying different defect types (such as thermal damage, mechanical tearing, etc.), providing reliable information support for clinical treatment, and has strong flexibility and adaptability.

[0079] In one possible implementation, pathologically confirmed defect areas are selected from an in vitro sample library as standard references. This library contains a large number of pathologically confirmed samples with accurate and precise defect annotations. Pathological confirmation identifies true defect areas and provides a reference for subsequent threshold setting.

[0080] Based on the multimodal data from the anatomical quality assessment, appropriate judgment indicators are selected. These indicators can be a combination of multimodal data such as hemoglobin oxygenation parameters, elastic modulus, surface roughness, etc. Different judgment indicators can reflect different biological characteristics of tissue defects.

[0081] For each selected defect region, a receiver operating characteristic (ROC) curve was calculated using these criteria. The ROC curve is a commonly used binary classification evaluation method used to demonstrate the trade-off between the sensitivity and specificity of a criterion. Analysis of the ROC curve reveals the classification performance of each criterion at different thresholds.

[0082] By analyzing the ROC curve for each metric, we select the point closest to the top-left vertex. This is the parameter value that achieves the optimal balance between sensitivity (true positive rate) and specificity (true negative rate). This point represents the classifier's optimal performance and serves as the critical threshold for determining whether an area is a defect.

[0083] The selected optimal threshold usually means that under this value, the diagnostic model can simultaneously maximize the ability to correctly classify defect areas, reduce the missed diagnosis rate, and avoid overdiagnosis, thereby ensuring the accuracy of the assessment.

[0084] The determined critical value and the set threshold are applied to the actual anatomical quality assessment system to perform quality judgments on different anatomical regions. If the judgment index of an anatomical region exceeds the set threshold, the region can be judged as a defective region.

[0085] In practical applications, the threshold setting is continuously optimized based on new sample data to ensure the accuracy and consistency of the diagnostic results in different types of tissues and defects.

[0086] By utilizing pathologically confirmed defect area data and calculating the ROC curve to determine the optimal threshold, we can ensure that the threshold setting is scientific. This method can minimize the interference of human factors and ensure that the set threshold has high accuracy and reliability.

[0087] By selecting the point closest to the upper left corner of the ROC curve as the optimal threshold, an optimal balance between the sensitivity and specificity of the judgment indicator can be achieved. This optimization process ensures that the system can strike a good compromise between avoiding misdiagnosis (overdiagnosis) and missed diagnosis when identifying defect areas, greatly improving the practicality and effectiveness of the diagnostic system. Based on the analysis of confirmed pathological data from the in vitro sample library, the threshold setting can be adaptive to different types of biological tissues and defects. This means that this method can be applied to a variety of anatomical regions and different pathological conditions, with strong adaptability and universality.

[0088] By accurately setting thresholds, the accuracy of identifying biological tissue defects can be effectively improved. Based on the precise assessment results provided by the system, clinicians can quickly make judgments and take targeted treatment measures, thereby improving treatment outcomes and shortening diagnosis and treatment cycles.

[0089] Because this method uses precise thresholds and judgment indicators to effectively distinguish between normal and defective areas, it reduces the risk of misjudgment (misidentifying a normal area as a defect) and missed detection (ignoring a defective area). This is crucial for improving system stability and reliability, especially in clinical applications, helping to reduce unnecessary interventions and ensure targeted treatment.

[0090] In one possible embodiment, the target biological tissue is first subjected to a cryosectioning process. The cryosectioning method can effectively preserve the structure and cell activity of the tissue and avoid changes during the sample preparation process.

[0091] Next, the cell nuclei are stained using an appropriate staining technique, typically using a common nuclear stain such as DAPI, to ensure that the nuclei are clearly visible under the microscope and to help subsequent image segmentation algorithms accurately identify cell boundaries.

[0092] Image segmentation algorithms are used to process stained tissue sections and segment the boundaries of cell clusters. Based on features such as pixel value, color intensity, and shape, image segmentation algorithms can automatically identify the boundaries between cells and accurately locate each cell cluster. Common image segmentation techniques include threshold-based segmentation methods, region growing algorithms, or deep learning methods, which can extract the outlines of cell clusters from cell images.

[0093] For each cluster, the size of the cluster was determined by calculating the minimum circumscribed circle (the smallest circular enclosing circle). The minimum circumscribed circle is the smallest circle that can completely enclose the cluster, and its diameter reflects the minimum size of the cluster. This process uses a convex hull algorithm to determine the outermost boundary of the cluster, thereby calculating the diameter of the minimum circumscribed circle.

[0094] After calculating the diameter of the minimum circumscribed circle of all cell clusters, a statistical distribution was constructed to analyze the size distribution of each cell cluster.

[0095] Based on these size distributions, the βth percentile is selected as the size of the smallest functional unit. The choice of percentile β varies depending on the tissue type. Different tissue types (such as liver, muscle, and neural tissue) have different cell cluster distribution characteristics, so the value of β can be adjusted within a preset range.

[0096] Finally, the calculated minimum functional unit size (i.e., the value corresponding to the βth percentile) is used as the diameter of the microprobe. This diameter ensures that the microprobe can fit the cell clusters in the target tissue, thereby more accurately assessing the tissue anatomical quality.

[0097] By analyzing the minimum circumscribed circle diameter of cell clusters, we can accurately reflect the structural characteristics of cell clusters in tissues. Compared with directly measuring cell size, calculating the minimum circumscribed circle can avoid errors caused by differences in cell morphology and provide a more robust size measurement standard.

[0098] The choice of β percentile can be adjusted to the needs of different tissue types, allowing the microprobe diameter to be more tailored to the specific tissue anatomy. This customized design allows for greater accuracy and flexibility in the assessment method, enabling its widespread application across different tissue types.

[0099] By calculating the statistical distribution of cell clusters and the minimum circumscribed circle diameter, we can ensure a more scientific and rational selection of microprobe size, avoiding the uncertainty of size selection in traditional methods. This design ensures that the microprobe can effectively enter the target tissue and conduct efficient and accurate anatomical quality assessment.

[0100] The present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention. To provide a thorough understanding of the present invention, specific details are described in detail below in connection with the preferred embodiments of the present invention, but those skilled in the art will be able to fully understand the present invention without these detailed descriptions. Furthermore, to avoid unnecessary confusion regarding the essence of the present invention, well-known methods, processes, procedures, components, and circuits have not been described in detail.

[0101] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for evaluating the quality of biological tissue anatomy based on multimodal data fusion, characterized in that: include: Step 1: Synchronous acquisition of multimodal data with spatial registration. The following data are collected synchronously in the same coordinate system using a spatial registration device with an integrated optical path: Hyperspectral data cube: Scan the anatomical surface to obtain spectral reflectance data with spatial coordinates and wavelength dimensions; Tissue mechanical distribution map: Microprobe array indentation testing is used to calculate the elastic modulus value at each point based on the displacement-load curve; 3D surface topology model: Obtain submicron-level topography data through laser scanning; Based on the coordinate mapping relationship, the mechanical distribution map and topological model are resampled to the spatial grid of the hyperspectral data; Step 2: Pathology-guided feature fusion: extract hemoglobin oxygenation parameters and collagen absorption features from the hyperspectral data, extract surface roughness indicators and local curvature changes of the 3D topological model, and extract elastic modulus gradient values ​​and viscoelastic response parameters from the mechanical distribution map; The hemoglobin oxygenation parameters and collagen absorption characteristics, the surface roughness index and local curvature variation of the three-dimensional topological model, the elastic modulus gradient value and the viscoelastic response parameters are combined into a high-dimensional matrix, and the dimensionality is reduced to a fusion feature vector that is strongly correlated with the pathological score through supervised manifold learning. Step 3: Anatomical quality index generation: input the fused feature vector into the pre-trained deep residual network, output the quality assessment index QI, generate the anatomical quality heat map based on the resampled spatial grid coordinates, and mark the QI value of each area.

2. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 1, characterized in that: The spatial registration device of step 1 includes: Biological tissue stage with three-dimensional translation stage; A laser locator that shares the same optical path as the hyperspectral imaging system projects a cross-reference marker on the anatomical surface; The CCD sensor integrated inside the microprobe captures the position of the marking point in real time; The coordinate registration process is: Calculate the pixel coordinates (X h , Y h ); Calculate the pixel coordinates (X c , Y c ); Solve the coordinate transformation matrix: Fit by least squares method (X h , Y h ) to (X c , Y c ) is the affine transformation parameters of .

3. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 1, wherein: The displacement-load curve of step 1 is obtained by: Loading phase: Increase the normal force at a constant rate to a target value. The target value is set dynamically based on the tissue type. This is done by pre-testing similar tissue samples and recording the minimum load that causes the displacement to reach a set ratio of the cell diameter. Maintaining stage: Maintain the target load until the displacement change rate is lower than the critical threshold, which is determined by the minimum value of the second-order derivative of the displacement attenuation curve; Unloading phase: The load is removed to zero at the same rate.

4. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 1, wherein: The extraction of collagen absorption characteristics in step 2 specifically includes: Identify the characteristic absorption peak of collagen in the near-infrared band; Calculate the ratio of the absorption peak area to the baseline area, where: Baseline area: take the integral of the spectral reflectance of the preset wave number windows on both sides of the absorption peak; Absorption peak window: The boundary is determined by statistics of the extreme points of the spectral derivative of the in vitro sample library.

5. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 1, wherein: Step 2 of supervised manifold learning involves: Feature weighting: Calculate the absolute value of the correlation coefficient between each feature dimension and the pathological score as the weight; Neighborhood graph construction: The sample fusion feature vector is used as the node; If the Euclidean distance between two samples in the pathology score space is less than the set radius, a connecting edge is established; Low-dimensional embedding: Solve the weighted Laplace matrix eigenvectors and take the vectors corresponding to the first k largest eigenvalues ​​as the dimensionality reduction result. The value of k is determined by whether the cumulative contribution rate of the eigenvalues ​​is greater than the set threshold.

6. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 1, characterized in that: The deep residual network in step 3 is trained as follows: Network structure: The number of input layer nodes is equal to the dimension of the fused feature vector; Residual block group: a sequence of fully connected layers with skip connections, where the output dimension of each layer decreases in geometric order; Output layer: single node with Sigmoid activation; Loss function: The Huber loss function is used, and its threshold parameter is adjusted by the median of the prediction error of the in vitro samples.

7. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 1, wherein: The anatomical mass heatmap of step 3 was generated by: Spatial interpolation: For grid points that are not directly measured, the QI value is calculated based on radial basis function interpolation; Pseudocolor mapping: Normalize the QI value to the color index value; A nonlinear color mapping table is used, in which the color gradient of the low QI segment is encrypted, and the range of the encrypted segment is determined by the trough position of the QI value distribution histogram of the sample library.

8. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 1, wherein: Also included, Step 4: Anatomical defect attribution analysis: Identify connected areas with QI values ​​lower than the dynamic threshold in the heat map. The dynamic threshold is the αth quantile of the QI value distribution in the normal area of ​​the sample library. α is set according to the clinical misjudgment rate requirement. Extract multimodal feature statistics of defect areas: If the product of the decrease in hemoglobin oxygenation parameter and the increase in elastic modulus is greater than the critical value, it is determined to be thermal injury; If the correlation coefficient between the surface roughness increment and the extreme value of the elastic modulus gradient is greater than the set threshold, it is judged as mechanical tearing.

9. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 8, characterized in that: The critical value and the set threshold are determined as follows: Select pathologically confirmed defective areas from the in vitro sample library; Calculate the receiver operating characteristic curve of each judgment index in the defect area; The parameter value corresponding to the vertex closest to the upper left corner of the curve is taken as the threshold.

10. The method for evaluating the quality of biological tissue anatomy based on multimodal data fusion according to claim 1, characterized in that: The diameter of the microprobe is determined as follows: Prepare cryosections of target tissues and perform nuclear staining; Identify cell cluster boundaries using image segmentation algorithms; The statistical distribution of the minimum circumscribed circle diameter of the cell cluster was calculated, and the βth percentile of the distribution was taken as the minimum functional unit size, where β was selected within a preset range based on the tissue type.

Images