Meridian detection method and system based on multi-modal data fusion

Abstract

The present application belongs to the technical field of traditional Chinese medicine detection technology and multi-modal data processing cross technology, and provides a traditional Chinese medicine meridian detection method and system based on multi-modal data fusion, which comprises the following steps: collecting functional modal and structural modal data through standardization, first performing systematic deviation analysis and judgment on the characteristic regions of the two modalities, then quantifying the deviation amplitude and deciding whether alignment is needed, if alignment is needed, calculating the precision deviation of multi-modal pre-fusion, constructing a correlation model of systematic deviation amplitude and fusion precision deviation, and finally based on the fusion precision target, backstepping the control quantity, and using a hyperbolic curve space alignment algorithm to realize the accurate alignment of the functional-structural modalities, solving the problem of systematic deviation caused by the lack of functional-structural double space modeling in the existing traditional Chinese medicine meridian detection, improving the multi-modal fusion precision, and providing a more reliable detection basis for traditional Chinese medicine meridian syndrome differentiation.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of traditional Chinese medicine detection technology and multimodal data processing, specifically a method and system for detecting meridians in traditional Chinese medicine based on multimodal data fusion. Background Technology

[0002] Traditional Chinese medicine meridian detection is a key technology that collects physiological and anatomical signals related to human meridians to assist in TCM diagnosis and treatment. With the development of multimodal data fusion technology, the integration of functional modalities (such as infrared thermography reflecting the warmth of qi and blood, and transcutaneous impedance tomography reflecting the conduction of meridian qi) and structural modalities (such as ultrasound reflecting fascia / vascular anatomy, and CT reflecting bone localization) has become an important direction for improving the accuracy of detection.

[0003] However, existing multimodal detection methods for TCM meridians suffer from core technical bottlenecks: the lack of functional-structural dual-space modeling. Existing algorithms are mostly based on anatomical space (physical coordinates of structural modalities) for registration, ignoring the functional specificity of meridians. For example, the "stomach heat" syndrome of the Stomach Meridian appears as a 2-3 mm strip-shaped high-temperature area distributed along the meridian in infrared thermography, while ultrasound can only identify subcutaneous vascular structures of 0.5 mm. The difference in scale and properties between the two leads to a systematic deviation in spatial mapping. This deviation is not a random error, but is caused by the essential difference between functional and structural modalities and the defects in existing registration logic. This directly reduces the information complementarity of multimodal fusion, making the fusion accuracy unable to meet the needs of clinical diagnosis. In addition, existing methods lack quantitative analysis of the correlation between systematic deviation and fusion accuracy, and cannot reverse the deviation according to the fusion accuracy target, resulting in blind and poorly targeted alignment strategies.

[0004] Therefore, this invention provides a method and system for detecting meridians in traditional Chinese medicine based on multimodal data fusion. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0006] The technical solution adopted by this invention to solve its technical problem is: a method for detecting meridians in traditional Chinese medicine based on multimodal data fusion, characterized in that it includes:

[0007] Step 1: Perform deviation analysis on the spatial mapping of the functional and structural modes of the meridians, and verify the stability and directional consistency of the deviations to determine whether there are systematic deviations in the spatial mapping of the functional and structural modes.

[0008] Step 2: If it exists, determine the magnitude of the systematic deviation based on its stability and directional consistency, and determine whether functional and structural modes need to be adjusted and aligned through comparative analysis of the magnitudes of the systematic deviations.

[0009] Step 3: If necessary, pre-fuse the functional modal and structural modal data and calculate the pre-fusion accuracy. By comparing the pre-fusion accuracy with the fusion accuracy under the condition of no systematic deviation, determine the fusion accuracy deviation.

[0010] Step 4: Perform a correlation analysis on the magnitude of systematic deviation and the fusion accuracy deviation to determine the correlation model between the magnitude of systematic deviation and the fusion accuracy deviation.

[0011] Step 5: Based on the correlation between the systematic deviation amplitude and the fusion accuracy deviation, and in conjunction with the target fusion accuracy, determine the adjustment alignment amount of the systematic deviation amplitude, and align the functional-structural modes using the hyperbolic space alignment algorithm.

[0012] Furthermore, the process of performing deviation analysis on the spatial mapping of the functional and structural modes of the meridians is as follows:

[0013] Functional modality feature regions and structural modality feature regions are extracted from the original functional modality and structural modality data of the same subject and the same anatomical region to generate multiple samples.

[0014] For any given sample:

[0015] An offset vector is generated by the coordinate difference between the center point of the functional feature region and the center point of the structural feature region. The direction of the offset vector is the deviation direction, and the magnitude is the center distance.

[0016] Based on the binarized image regions obtained after feature extraction, the pixel sets of functional feature regions and structural feature regions are obtained.

[0017] Calculate the intersection and union of the pixel sets of the functional feature region and the structural feature region, and calculate the ratio of the intersection and union with respect to the pixels to obtain the intersection-union ratio.

[0018] Furthermore, the method for determining whether there is a systematic deviation in the spatial mapping between the functional modes and structural modes is as follows:

[0019] Calculate the coefficient of variation of the center distance and cross-union ratio for all samples. If the coefficients of variation of the center distance and cross-union ratio are both less than the threshold, then the center distance and cross-union ratio of all samples are stable.

[0020] The category with the highest percentage of offset direction in the statistical sample is compared with the preset percentage. If the percentage is greater than the preset percentage, the sample is determined to have the same offset direction.

[0021] If the center distance and crossover ratio of the samples are stable and the offset direction is consistent, then it is determined that there is a systematic deviation in the spatial mapping of functional modes and structural modes.

[0022] If the center distance and crossover ratio of all samples are stable, and the proportion of samples with the same offset direction is greater than the preset proportion, then it is determined that there is a systematic deviation in the spatial mapping of functional mode and structural mode.

[0023] Furthermore, the method for determining whether functional mode and structural mode adjustment and alignment are needed is as follows:

[0024] Calculate the mean of the center distance and crossover ratio for all samples as the center distance and crossover ratio of the systematic bias.

[0025] The systematic deviation magnitude is obtained by normalizing the center distance d between the functional feature region and the structural feature region and then comparing the difference with the intersection and union of these distances.

[0026] The systematic deviation magnitude is compared with the preset deviation. If the systematic deviation magnitude is greater than the preset deviation, the alignment needs to be adjusted.

[0027] Furthermore, the calculation method for the pre-fusion accuracy is as follows:

[0028] Data cleaning, spatial / temporal registration, and feature standardization are performed on the raw multimodal data.

[0029] If the functional modality and structural modality data dimensions are consistent, the functional modality feature matrix and the structural modality feature matrix are concatenated according to the channel dimension, and the functional modality and structural modality are pre-fused at the data layer.

[0030] If the functional modality and structural modality data dimensions are inconsistent, core features are extracted for the functional modality and structural modality respectively, and then the extracted feature vectors are concatenated into a unified fusion feature vector to achieve feature layer pre-fusion.

[0031] Among them, the core features of functional modes and structural modes refer to the peak value and mean value of time-series signals extracted by functional modes, and geometric features such as volume and surface area extracted by structural modes.

[0032] The normalized mutual information between the fused functional modal features and structural modal features is the pre-fusion accuracy.

[0033] Furthermore, the calculation method for the fusion accuracy deviation is as follows:

[0034] Stability analysis is performed on the normalized mutual information after fusion under the condition of historical unbiasedness.

[0035] If the situation is stable, the mean of the normalized mutual information is taken as the benchmark value of the normalized mutual information; otherwise, if the situation is unstable, the maximum value of the historical normalized mutual information is taken as the benchmark value of the normalized mutual information.

[0036] The fusion accuracy deviation is the difference between the pre-fused normalized mutual information and the normalized mutual information baseline value.

[0037] Furthermore, the correlation model between the systematic deviation magnitude and the fusion accuracy deviation is determined as follows:

[0038] Linear regression was used to perform a linear fit between systematic bias and fusion accuracy bias, and the coefficient of determination R was calculated. 2 .

[0039] The coefficient of determination R 2 Compared with a preset threshold, if the coefficient of determination R 2 If the deviation is greater than or equal to the preset threshold, the systematic deviation amplitude and the fusion accuracy deviation are linearly related; otherwise, they are non-linearly related.

[0040] If the magnitude of the systematic deviation and the fusion accuracy deviation are linearly related, then the fitting equation obtained by linear fitting is the correlation model between the magnitude of the systematic deviation and the fusion accuracy deviation.

[0041] If the magnitude of the systematic deviation and the deviation of the fusion accuracy are nonlinear, a candidate nonlinear model is selected based on the data trend, and an iterative optimization algorithm is used to solve the candidate nonlinear model.

[0042] From multiple candidate nonlinear models, the nonlinear model with the highest coefficient of determination R² is selected as the correlation model between the systematic deviation magnitude and the fusion accuracy deviation.

[0043] Furthermore, the method for determining the adjustment amount of the systematic deviation amplitude is as follows:

[0044] Accuracy targets are set based on the clinical / engineering requirements of the target task, and these accuracy targets are substituted into the correlation regression equation between the systematic bias magnitude and the fusion accuracy bias to obtain the target systematic bias magnitude.

[0045] The adjustment amount for the systematic deviation amplitude is the difference between the current deviation amplitude and the target systematic deviation amplitude.

[0046] Furthermore, the alignment process using the hyperbolic space alignment algorithm is as follows:

[0047] The coordinates of the structural modal feature region are (x s ,y s The coordinates of the functional modal feature region are (x... f ,y f The deviation between the two is (∆x,∆y)=(x f -x s ,y f -y s ) .

[0048] The hyperbolic transformation maps functional coordinates to the structural modal coordinate system by introducing the parameter curvature factor k and the reference distance d:

[0049] , Wherein, the curvature factor k is the ratio of the systematic deviation amplitude adjustment amount to the current systematic deviation amplitude, and the reference distance d is taken as 1 / 5 of the average diameter of the structural modal characteristic region.

[0050] Multiple anatomical landmarks were selected from the structural modes, and their coordinates P were recorded. s ={(x s1 ,y s1 ),(x s2 ,y s2 ),...};

[0051] Identify identical marker points from functional modes and record their coordinates P. f ={(xf1,y f1 ),(x f2 ,y f2 ),...};

[0052] Calculate the initial deviation field: Based on the coordinate difference of the marker points, the global deviation field D(x,y)=(∆x(x,y),∆y(x,y)) of the functional mode and the structural mode is generated by interpolation, which reflects the deviation distribution of the whole map.

[0053] For each pixel (x) of the functional modality feature region f ,y f ), mapped to structural modal coordinates (x) through the hyperbolic transformation formula. s ,y s ), generating aligned functional modal feature regions.

[0054] The TCM meridian detection system based on multimodal data fusion includes the following modules:

[0055] Systematic Deviation Judgment Module: Performs deviation analysis on the spatial mapping of the functional and structural modes of the meridians, verifies the stability and directional consistency of the deviations, and determines whether there is a systematic deviation in the spatial mapping of the functional and structural modes.

[0056] Deviation quantification and alignment decision module: If it exists, determine the magnitude of the systematic deviation based on the stability and directional consistency of the systematic deviation, and determine whether functional mode and structural mode adjustment and alignment are required through comparative analysis of the systematic deviation magnitude.

[0057] Multimodal pre-fusion and accuracy assessment module: If necessary, pre-fusion of functional modal and structural modal data is performed, and the pre-fusion accuracy is calculated. By comparing the pre-fusion accuracy with the fusion accuracy under the condition of no systematic deviation, the fusion accuracy deviation is determined.

[0058] Deviation-Accuracy Correlation Construction Module: Performs correlation analysis on the magnitude of systematic deviation and the fusion accuracy deviation to determine the correlation model between the magnitude of systematic deviation and the fusion accuracy deviation.

[0059] Deviation control and hyperbolic alignment module: Based on the correlation between systematic deviation amplitude and fusion accuracy deviation, and combined with the target fusion accuracy, determine the adjustment and alignment amount of systematic deviation amplitude, and combine the hyperbolic space alignment algorithm to align the functional-structural modes.

[0060] The beneficial effects of this invention are as follows: By combining the center distance and intersection-union ratio as dual indicators with the coefficient of variation and the consistency rate of the offset direction, a quantitative determination of systematic deviations in functional-structural modes is achieved, accurately identifying systematic deviations and avoiding misjudging random errors as systematic deviations, thus providing a reliable basis for subsequent alignment; a linear / nonlinear correlation model is established based on historical data to clarify the quantitative relationship between changes in deviation amplitude and changes in fusion accuracy, constructing a quantitative correlation between deviation and accuracy, shifting deviation control from experience-driven to data-driven; a hyperbolic space alignment algorithm is adopted, which adaptively adjusts the correction amplitude through the curvature factor, taking into account the functional modes. Compared to traditional affine registration, the specificity and stability of structural modalities are improved, resulting in enhanced fusion accuracy after alignment and efficient targeted alignment. The fusion process incorporates TCM-specific modalities such as electrophysiology and morphology, extracting meridian-related features such as temperature gradients and acupoint impedance to ensure that the detection results conform to the core logic of TCM holistic view and syndrome differentiation and treatment, and are adapted to TCM clinical needs. The method and process are standardized, supporting combinations of different functional / structural modalities (such as infrared + MRI, impedance + CT), and can be adapted to the detection needs of different meridian sites (such as the Lung Meridian of Hand-Taiyin and the Stomach Meridian of Foot-Yangming) through parameter adjustment, with strong versatility and scalability. Attached Figure Description

[0061] The invention will now be further described with reference to the accompanying drawings.

[0062] Figure 1 This is a flowchart of the steps of the TCM meridian detection method based on multimodal data fusion as described in Embodiment 1 of the present invention.

[0063] Figure 2 This is the logic judgment diagram of the TCM meridian detection method based on multimodal data fusion as described in Embodiment 1 of the present invention.

[0064] Figure 3 This is a flowchart of the TCM meridian detection system based on multimodal data fusion as described in Embodiment 2 of the present invention. Detailed Implementation

[0065] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0066] Example 1: Please refer to Figure 1 As shown in the embodiment of the present invention, the method for detecting meridians in traditional Chinese medicine based on multimodal data fusion includes the following steps:

[0067] Step 1: Perform deviation analysis on the spatial mapping of the functional and structural modes of the meridians, and verify the stability and directional consistency of the deviations to determine whether there are systematic deviations in the spatial mapping of the functional and structural modes.

[0068] Please see Figure 2 As shown, in step one, the process of performing deviation analysis on the spatial mapping of the functional and structural modes of the meridians includes:

[0069] The spatial mapping of functional modalities is the functional feature region, which refers to the spatial region extracted from functional modalities (such as infrared and electrophysiology) that reflects the target functional state, such as the strip-shaped high temperature region of the "stomach heat" syndrome of the stomach meridian.

[0070] The spatial mapping of structural modalities is the structural feature region, which refers to the spatial region that reflects the anatomical entity extracted from structural modalities (such as ultrasound and CT), such as the 0.5mm subcutaneous vascular structure.

[0071] Functional modal feature regions and structural modal feature regions are extracted from the raw acquired data of functional modalities (such as infrared thermograms and impedance distribution maps) and structural modalities (such as ultrasound images and CT tomographic images).

[0072] All data must be collected from the same subject and the same anatomical region (e.g., the section from Zusanli to Liangmen on the Stomach Meridian), and the time interval between collections must be ≤30 minutes (to avoid feature shifts caused by changes in physiological state). For example:

[0073] Infrared thermal imaging: Fixed acquisition distance (e.g., 50cm), ambient temperature (22-25℃), subject position (e.g., supine position, exposure of the detection area for 30 minutes to stabilize skin temperature), and use of the same equipment (to avoid resolution differences).

[0074] Ultrasound images: Fix the probe frequency (e.g., 7-15MHz, to ensure that subcutaneous vessels of 0.5mm can be identified), probe pressure (to avoid tissue deformation), depth range (e.g., focusing on subcutaneous 1-10mm), and record the spatial coordinates of the probe (to facilitate subsequent registration).

[0075] Based on anatomical landmarks (such as bone contours, skin folds, and acupoint locations), the infrared thermogram and ultrasound image are initially aligned, for example:

[0076] In the infrared thermal image, the projections of bones such as the anterior superior iliac spine and the lower border of the patella are marked;

[0077] Marking the ultrasound echo boundaries of the same bone in ultrasound images;

[0078] The anatomical landmarks of the two objects are aligned using affine transformations (translation, rotation, scaling) to ensure that the global deviation is ≤0.5mm.

[0079] Use multimodal registration algorithms (such as mutual information-based registration and feature point matching registration) to further optimize local alignment.

[0080] For any given sample:

[0081] Through the center point (x) of the functional feature region f ,y f ) and structural feature regions (x s ,y s The coordinate difference of the center point generates an offset vector. The direction of the vector is the direction of the deviation, and the magnitude is the center distance.

[0082] The direction of deviation is defined based on the offset vector:

[0083] If |∆x|>|∆y|:

[0084] If ∆x < 0, the negative deviation of the x-axis is greater than that of the y-axis, and the functional area shifts to the left relative to the structural area along the long axis of the meridian.

[0085] If ∆x>0, the deviation in the positive x-axis direction is greater than that in the y-axis, and the functional area shifts to the right along the long axis of the meridian.

[0086] If |∆y|>|∆x|:

[0087] If ∆y<0, the negative deviation of the y-axis is greater than that of the x-axis, and the functional area shifts towards the superficial subcutaneous layer relative to the structural area.

[0088] If ∆y>0, the deviation in the positive y-axis direction is greater than that in the x-axis, and the functional area shifts towards the deeper subcutaneous layer.

[0089] The pixel set S of the functional feature region is obtained from the binarized image region after feature extraction. f and the set of pixels S of structural feature regions s .

[0090] The intersection-union ratio (IUR) is calculated by proportionally scaling the intersection and union of the pixel sets of the functional feature region and the structural feature region. The formula is as follows: , where the molecule |S f ∩S s | represents the number of overlapping pixels between two feature regions, and the denominator is |S. f ∪S s | indicates the number of pixels in the union of two feature regions.

[0091] In step one, the process of determining whether there is a systematic deviation in the spatial mapping between functional modes and structural modes includes:

[0092] Calculate the coefficients of variation of the center distance and crossover ratio for all samples, and compare them with preset thresholds. If the coefficients of variation of the center distance and crossover ratio are both less than the thresholds, then the center distance and crossover ratio of the samples are stable.

[0093] The category with the highest percentage of offset direction in the statistical sample is compared with the preset percentage. If the percentage is greater than the preset percentage, the sample is determined to have the same offset direction.

[0094] If the center distance and crossover ratio of the samples are stable and the offset direction is consistent, then it is determined that there is a systematic deviation in the spatial mapping of functional modes and structural modes.

[0095] The purpose of determining whether there is a systematic deviation in the spatial mapping between functional and structural modes is:

[0096] Through standardized data collection, preliminary registration, and quantitative analysis (center distance, crossover ratio, coefficient of variation, etc.), it is determined whether the deviations in the characteristic regions of functional and structural modes are regular and repeatable, i.e., whether there are systematic deviations (rather than random errors). This is the premise and basis for all subsequent deviation processing. Only when systematic deviations are clearly present should subsequent control procedures be initiated. If it is only a random error, no additional intervention is required.

[0097] Step 2: If it exists, determine the magnitude of the systematic deviation based on its stability and directional consistency, and determine whether functional and structural modes need to be adjusted and aligned through comparative analysis of the magnitudes of the systematic deviations.

[0098] In step two, the process of determining whether functional mode and structural mode adjustment and alignment are needed includes:

[0099] Calculate the mean of the center distance and crossover ratio for all samples as the center distance and crossover ratio of the systematic bias.

[0100] Normalize the center distance d between the two feature regions: Where d is the center distance, d max This represents the length of the image's diagonal.

[0101] The systematic deviation magnitude is obtained by subtracting the normalized center distance from the intersection and union ratio. The systematic deviation magnitude is then compared with the preset deviation. If the systematic deviation magnitude is greater than the preset deviation, alignment adjustment is required.

[0102] Understandably, the physical meaning of the systematic deviation amplitude is as follows: the normalized center distance is a quantitative indicator of the relative deviation between two targets in spatial position, and the intersection-union ratio is a quantitative indicator of the consistency between two targets in spatial coverage, directly reflecting whether the boundaries of the targets match. The systematic deviation amplitude obtained by subtracting the normalized center distance from the intersection-union ratio is a dimensionless and comparable measure that comprehensively quantifies the overall degree of deviation of the target spatial matching from the ideal state caused by system factors. It can also be used to locate the source of deviation and evaluate system performance.

[0103] The purpose of determining whether alignment adjustment is needed is as follows:

[0104] Based on the confirmation of systematic bias, the magnitude of the bias is quantified by calculating the mean of the bias and normalizing it, and then compared with a preset threshold. Finally, a decision is made on whether the bias needs to be adjusted and aligned. This step is crucial, as it not only completes the quantitative description of the bias, but also filters out biases that are too small to affect subsequent analysis by judging the threshold, thus avoiding invalid operations.

[0105] Step 3: If necessary, pre-fuse the functional modal and structural modal data and calculate the pre-fusion accuracy. By comparing the pre-fusion accuracy with the fusion accuracy under the condition of no systematic deviation, determine the fusion accuracy deviation.

[0106] In step three, the calculation process for the pre-fusion accuracy includes:

[0107] Data cleaning, spatial / temporal registration, and feature standardization are performed on the raw data of functional and structural modes.

[0108] The core of pre-fusion is to simplify the fusion logic and ensure consistency with history. Therefore, a fusion strategy with low complexity and high reproducibility is preferred.

[0109] Data layer pre-fusion (suitable for situations where functional modality and structural modality data have the same dimensionality, such as both being 2D images or 3D voxel data):

[0110] Preliminary integration is achieved using feature splicing or weighted averaging:

[0111] Feature concatenation: The functional modality feature matrix and the structural modality feature matrix are concatenated along the channel dimension (e.g., in a 2D image, the single-channel features of the functional modality and the single-channel features of the structural modality are concatenated into a 2-channel feature matrix).

[0112] The functional modal feature matrix and the structural modal feature matrix are obtained by extracting key features that reflect the functional state or anatomical entity from the original modal data and organizing these features into a matrix form that meets the requirements of subsequent fusion (such as feature splicing).

[0113] Weighted average: The eigenvalues ​​of the functional mode and the structural mode are weighted and summed based on the importance ratio of the two modes in historical data.

[0114] Feature layer pre-fusion (suitable for functional modality and structural modality data with significant dimensional differences, such as functional modality being 1D time-series signals and structural modality being 3D voxel data):

[0115] First, extract core features for each of the two modes (e.g., extract peak value and mean value of time-series signal for functional mode, and extract geometric features such as volume and surface area for structural mode), and then concatenate the extracted feature vectors into a unified fusion feature vector.

[0116] Pre-fusion accuracy needs to be calculated by quantifying the complementarity and consistency of information after fusion of two modes. Normalized mutual information is selected as the comparison index. Its core advantage is that it is applicable to various modes (images, signals, geometric data, etc.) and can directly reflect the degree of overlap and complementarity of information of two modes after fusion. It is a general index for evaluating the accuracy of multimodal fusion.

[0117] Mutual information (MI) is used to measure the degree of correlation between two random variables (in this case, functional modal features X and structural modal features Y), that is, the amount of information about Y obtained through X. NMI is a normalization of MI, which eliminates the influence of feature dimension and value range on absolute value, and allows the precision value to be compared horizontally in the [0,1] interval.

[0118] The calculation formula is: ,in:

[0119] , where p(x,y) is the joint probability distribution of X and Y, and p(x) and p(y) are the marginal probability distributions of X and Y, respectively;

[0120] The information entropy of X measures the uncertainty of X;

[0121] Let Y be the information entropy, which measures the uncertainty of Y.

[0122] It should be noted that the value of normalized mutual information (NMI) has the following meanings: NMI=1 indicates that the two modal features are completely correlated, the information after fusion is completely complementary and without redundancy, and the accuracy is optimal; NMI=0 indicates that the two modal features are completely independent, there is no information gain after fusion, and the accuracy is worst. In practical applications, the closer NMI is to 1, the better the pre-fusion effect is; conversely, it indicates that the current modal bias leads to insufficient correlation of fused information.

[0123] In step three, the calculation process for the fusion accuracy deviation includes:

[0124] A stability analysis is performed on the normalized mutual information (NMI) under the historical unbiased condition. If it is stable, the mean of the normalized mutual information (NMI) is taken as the normalized mutual information (NMI) under normal conditions. Otherwise, if it is unstable, the maximum value of the historical normalized mutual information (NMI) is taken as the normal baseline value.

[0125] The fusion accuracy deviation ∆NMI is the difference between the pre-fused NMI and the NMI reference value.

[0126] If ∆NMI≥0, it means that the current pre-fusion accuracy is not affected by the deviation. Conversely, if ∆NMI<0, it means that the current deviation causes the fusion accuracy to decrease. The larger the absolute value of ∆NMI, the more serious the accuracy loss.

[0127] The purpose of evaluating pre-fusion accuracy and quantifying the impact of deviations is as follows:

[0128] Functional and structural modal data (functional, structural, and other modalities) are standardized, preprocessed, and simplified for fusion (pre-fusion). The pre-fusion accuracy is quantified by normalized mutual information (NMI) and compared with the normal accuracy benchmark when there is no bias. The fusion accuracy deviation is calculated. Essentially, the systematic deviation between modalities is transformed into a measurable fusion quality loss, clarifying the actual impact of the deviation on the final multimodal fusion result.

[0129] Step 4: Perform a correlation analysis on the magnitude of systematic deviation and the fusion accuracy deviation to determine the correlation model between the magnitude of systematic deviation and the fusion accuracy deviation.

[0130] In step four, the process of determining the correlation model between the systematic deviation magnitude and the fusion accuracy deviation includes:

[0131] Obtain the calculation results of the modal feature region comparison step in historical detection, the equipment log operation records, and the calculation results of the fusion accuracy verification step.

[0132] The systematic deviation data includes the deviation magnitude of the characteristic regions of each mode.

[0133] Linear regression was used to linearly fit the systematic bias and the fusion accuracy bias. The model expression is Y=aX+b+ε, where Y is the fusion accuracy index, X is the bias amplitude, a and b are the regression coefficients obtained by the least squares method, and ε is the random error term.

[0134] Calculate the coefficient of determination R 2 R 2 =1-(Sum of squared residuals / Sum of squared total deviations), with a value range of [0,1]. The closer R² is to 1, the stronger the linear explanatory power of X for Y (i.e., R² proportion of the change in Y is caused by the linear change of X). The closer R² is to 0, the worse the linear fitting effect.

[0135] The coefficient of determination R 2 Compared with a preset threshold, if the coefficient of determination R 2 If the deviation is greater than or equal to the preset threshold, the systematic deviation amplitude and the fusion accuracy deviation are linearly related; otherwise, they are non-linearly related.

[0136] If the magnitude of the systematic deviation and the fusion accuracy deviation are linearly related, then the fitting equation obtained by linear fitting is the correlation model between the magnitude of the systematic deviation and the fusion accuracy deviation.

[0137] If the magnitude of the systematic bias and the fusion accuracy bias have a non-linear relationship, a non-linear model is selected based on the data trend, for example:

[0138] If Y increases or decreases exponentially with X, choose the exponential model. ;

[0139] If Y grows rapidly with X at first and then slowly, gradually leveling off, then the logarithmic model Y=a∙ln(X)+b is chosen;

[0140] If Y follows a quadratic (parabolic) or cubic curve trend with X, then the polynomial model Y = a0 + a1X + a2X is chosen. 2 +...+a n X n ;

[0141] If Y and X have a power-law relationship, choose the power function model. .

[0142] Iterative optimization algorithms are used to solve nonlinear models, for example:

[0143] Gauss-Newton method: It updates parameters iteratively through linear approximation, and is suitable for scenarios where the residuals are close to a normal distribution.

[0144] The model is evaluated using R² (adjusted for the impact of sample size and number of parameters on R²), the Akaike Information Criterion (AIC), or the Bayesian Information Criterion (BIC):

[0145] The higher the adjusted R², the better; the lower the AIC / BIC, the better (prioritize the model with the smallest AIC / BIC).

[0146] Residual analysis: linear regression is used to verify whether the residuals are randomly distributed (without systematic trend) to ensure that the model has no obvious bias.

[0147] From multiple candidate nonlinear models, the model with the highest adjusted R², the lowest AIC / BIC, and the residuals that meet the assumptions is selected as the final correlation model.

[0148] For example, the independent variable X is the normalized systematic deviation magnitude, and the dependent variable Y is the fusion accuracy deviation. It is assumed that the correspondence between X and Y is obtained through 20 sets of historical monitoring data.

[0149] By plotting the XY scatter plot, it was observed that Y follows a quadratic parabolic trend, first decreasing slowly and then accelerating as X increases. This initially identified the quadratic polynomial model as the core candidate, while the exponential model was used as a comparison model.

[0150] Based on data trends, two candidate models were identified:

[0151] Candidate Model 1 (Quadratic Polynomial Model): Y = a0 + a1X + a2X 2 ;

[0152] Candidate Model 2 (Exponential Model): ;

[0153] Taking candidate model 1 (quadratic polynomial model) as an example, the parameter solution process of Gauss-Newton method is demonstrated (core logic: to minimize the sum of squared residuals by iteratively optimizing the parameters through linear approximation).

[0154] Let the parameter vector be β=[a0,a1,a2]T, and the initial value be set to β0=[0,-1,0]T according to the data trend (preliminary assumption: linear decrease, quadratic term is 0).

[0155] For each sample i, the model prediction value is residual ;

[0156] The Gauss-Newton method transforms a nonlinear problem into a linear least squares problem by constructing a "Jacobi matrix of residuals with respect to parameters," and converges after three iterations.

[0157] The final parameters of the quadratic polynomial model are: a0=0.19, a1=-2.70, a2=2.15, and the model expression is: Y=0.19-2.70X+2.15X²;

[0158] Candidate model 2 also uses the Gauss-Newton method to obtain the exponential model expression: Y = -0.01∙e 2.5X -0.01 (SSE=0.035).

[0159] Calculate the adjusted R², AIC, and BIC of the two candidate models, perform residual analysis, and select the optimal model;

[0160] Finally, the quadratic polynomial model Y = 0.19 - 2.70X + 2.15X² was determined to be the nonlinear correlation model between X and Y.

[0161] The purpose of determining the correlation between the magnitude of systematic deviation and the deviation in fusion accuracy is as follows:

[0162] By using regression analysis (linear / nonlinear) and model evaluation (coefficient of determination R², AIC / BIC, etc.), a quantitative correlation (such as linear relationship, logarithmic relationship, etc.) between the magnitude of systematic deviation and the deviation of fusion accuracy is established. This model is the core tool for subsequent precise control. It answers the question of how much the change in deviation magnitude will lead to how much the fusion accuracy will change, and provides a mathematical basis for target-oriented deviation control.

[0163] Step 5: Based on the correlation between the systematic deviation amplitude and the fusion accuracy deviation, and in conjunction with the target fusion accuracy, determine the adjustment alignment amount of the systematic deviation amplitude, and align the functional-structural modes using the hyperbolic space alignment algorithm.

[0164] In step five, the process of determining the adjustment amount of the systematic deviation amplitude includes:

[0165] The method for setting the target fusion accuracy is determined by the clinical / engineering requirements of the target task;

[0166] Substituting the target fusion accuracy into the correlation regression equation, we can solve for the target systematic bias d. 目标 ;

[0167] The adjustment amount for systematic deviation is the difference between the current deviation magnitude and the target deviation.

[0168] In step five, the alignment process using the hyperbolic space alignment algorithm includes:

[0169] Based on the spatial coordinate differences between the functional and structural modal feature regions, a hyperbolic transformation model with anatomical landmarks as the reference is established.

[0170] The coordinates of the structural modal feature region are (x s ,y s The coordinates of the functional modal feature region are (x... f ,y f The deviation between the two is (∆x,∆y)=(x f -x s ,y f -y s ) .

[0171] The hyperbolic transformation maps functional coordinates to the structural modal coordinate system by introducing parameters k (curvature factor) and d (reference distance):

[0172] , The curvature factor k is determined by the deviation control amount. The larger the control amount, the closer k is to 1, and the larger the correction magnitude. The calculation method is: k = control amount / X 当前 The reference distance d is taken as 1 / 5 of the average diameter of the structural modal characteristic region.

[0173] Select multiple stable anatomical landmarks (such as bone edges and acupoints) from structural modalities (e.g., ultrasound images) and record their coordinates P. s ={(x s1 ,y s1 ),(x s2 ,y s2 ),...} .

[0174] Identify identical marker points from functional modes (such as infrared thermograms) and record their coordinates P. f ={(xf1,y f1 ),(x f2 ,y f2 ),...} .

[0175] Calculate the initial deviation field: Based on the coordinate difference of the marker points, the global deviation field D(x,y)=(∆x(x,y),∆y(x,y)) of the functional mode and the structural mode is generated by interpolation, which reflects the deviation distribution of the whole map.

[0176] For each pixel (x) of the functional modality feature region f ,y f ), mapped to structural modal coordinates (x) through the hyperbolic transformation formula. s ,y s ), generating aligned functional modal feature regions.

[0177] The purpose of alignment is:

[0178] Based on the established association model and the preset fusion accuracy target, the required target deviation range is deduced, the specific deviation control amount is calculated, and then the functional modal feature region is mapped to the structural modal coordinate system through the hyperbolic space alignment algorithm to achieve precise alignment between the two. This step is the final implementation link of the entire process, which directly solves the systematic deviation problem and enables the multimodal fusion accuracy to meet the target requirements of clinical or engineering.

[0179] The technical solution and advantages of this application embodiment are as follows: Deviation analysis is performed on the spatial mapping of the functional and structural modes of meridians, and the stability and directional consistency of the deviation are verified. It is determined whether there is a systematic deviation in the spatial mapping of the functional and structural modes. If so, the magnitude of the systematic deviation is determined based on its stability and directional consistency. A comparative analysis of the magnitude of the systematic deviation is conducted to determine whether adjustment and alignment of the functional and structural modes are necessary. If so, the functional and structural mode data are pre-fused, and the pre-fusion accuracy is calculated. A comparative analysis of the pre-fusion accuracy and the fusion accuracy without systematic deviation is conducted to determine the fusion accuracy deviation. A correlation analysis is performed between the magnitude of the systematic deviation and the fusion accuracy deviation to determine the correlation model between them. Based on the correlation between the magnitude of the systematic deviation and the fusion accuracy deviation, and combined with the target fusion accuracy, the adjustment and alignment amount of the systematic deviation magnitude is determined. Finally, the functional-structural modes are aligned using a hyperbolic spatial alignment algorithm. This application standardizes the collection of functional and structural modal data, first analyzes and determines the systematic deviation of the feature regions of the two modalities, then quantifies the deviation magnitude and decides whether alignment is needed. If alignment is needed, it calculates the accuracy deviation through multimodal pre-fusion, constructs a correlation model between the systematic deviation magnitude and the fusion accuracy deviation, and finally back-calculates the control amount based on the fusion accuracy target. It then uses a hyperbolic space alignment algorithm to achieve accurate alignment of functional and structural modalities. This solves the problem of systematic deviation caused by the lack of functional-structural dual-space modeling in existing TCM meridian detection, improves the accuracy of multimodal fusion, and provides a more reliable detection basis for TCM meridian differentiation.

[0180] Example 2: Please refer to Figure 3 As shown in the embodiment of the present invention, the TCM meridian detection system based on multimodal data fusion includes the following modules:

[0181] Systematic Deviation Judgment Module: Performs deviation analysis on the spatial mapping of the functional and structural modes of the meridians, verifies the stability and directional consistency of the deviations, and determines whether there is a systematic deviation in the spatial mapping of the functional and structural modes.

[0182] The process of performing deviation analysis on the spatial mapping of the functional and structural modes of meridians includes:

[0183] The spatial mapping of functional modalities is the functional feature region, which refers to the spatial region extracted from functional modalities (such as infrared and electrophysiology) that reflects the target functional state, such as the strip-shaped high temperature region of the "stomach heat" syndrome of the stomach meridian.

[0184] The spatial mapping of structural modalities is the structural feature region, which refers to the spatial region that reflects the anatomical entity extracted from structural modalities (such as ultrasound and CT), such as the 0.5mm subcutaneous vascular structure.

[0185] Functional modal feature regions and structural modal feature regions are extracted from the raw acquired data of functional modalities (such as infrared thermograms and impedance distribution maps) and structural modalities (such as ultrasound images and CT tomographic images).

[0186] All collected data must come from the same subject and the same anatomical region (such as the Zusanli to Liangmen section of the Stomach Meridian), and the time interval between collections must be ≤30 minutes (to avoid feature shifts caused by changes in physiological state).

[0187] Based on anatomical landmarks (such as bone contours, skin folds, and acupoint locations), the infrared thermogram and ultrasound image are initially aligned.

[0188] By using affine transformations (translation, rotation, scaling), the anatomical landmarks of the two objects are aligned to ensure that the global deviation is ≤0.5mm.

[0189] Use multimodal registration algorithms (such as mutual information-based registration and feature point matching registration) to further optimize local alignment.

[0190] For any given sample:

[0191] Through the center point (x) of the functional feature region f ,y f ) and structural feature regions (x s ,y s The coordinate difference of the center point generates an offset vector. The direction of the vector is the direction of the deviation, and the magnitude is the center distance.

[0192] The direction of deviation is defined based on the offset vector:

[0193] If |∆x|>|∆y|:

[0194] If ∆x < 0, the negative deviation of the x-axis is greater than that of the y-axis, and the functional area shifts to the left relative to the structural area along the long axis of the meridian.

[0195] If ∆x>0, the deviation in the positive x-axis direction is greater than that in the y-axis, and the functional area shifts to the right along the long axis of the meridian.

[0196] If |∆y|>|∆x|:

[0197] If ∆y<0, the negative deviation of the y-axis is greater than that of the x-axis, and the functional area shifts towards the superficial subcutaneous layer relative to the structural area.

[0198] If ∆y>0, the deviation in the positive y-axis direction is greater than that in the x-axis, and the functional area shifts towards the deeper subcutaneous layer.

[0199] The pixel set S of the functional feature region is obtained from the binarized image region after feature extraction. fand the set of pixels S of structural feature regions s .

[0200] The intersection-union ratio (IUR) is calculated by proportionally scaling the intersection and union of the pixel sets of the functional feature region and the structural feature region. The formula is as follows: , where the molecule |S f ∩S s | represents the number of overlapping pixels between two feature regions, and the denominator is |S. f ∪S s | indicates the number of pixels in the union of two feature regions.

[0201] The process of determining whether there is a systematic deviation in the spatial mapping between functional modes and structural modes includes:

[0202] Calculate the coefficients of variation of the center distance and crossover ratio for all samples, and compare them with preset thresholds. If the coefficients of variation of the center distance and crossover ratio are both less than the thresholds, then the center distance and crossover ratio of the samples are stable.

[0203] The category with the highest percentage of offset direction in the statistical sample is compared with the preset percentage. If the percentage is greater than the preset percentage, the sample is determined to have the same offset direction.

[0204] If the center distance and crossover ratio of the samples are stable and the offset direction is consistent, then it is determined that there is a systematic deviation in the spatial mapping of functional modes and structural modes.

[0205] Deviation quantification and alignment decision module: If it exists, determine the magnitude of the systematic deviation based on the stability and directional consistency of the systematic deviation, and determine whether functional mode and structural mode adjustment and alignment are required through comparative analysis of the systematic deviation magnitude.

[0206] The process of determining whether functional and structural modal alignment is needed includes:

[0207] Calculate the mean of the center distance and cross-union ratio for all samples as the center distance and cross-union ratio of the systematic bias;

[0208] Normalize the center distance d between the two feature regions: , where d is the center distance and dmax is the length of the image diagonal.

[0209] The systematic deviation magnitude is obtained by subtracting the normalized center distance from the intersection and union ratio. The systematic deviation magnitude is then compared with the preset deviation. If the systematic deviation magnitude is greater than the preset deviation, alignment adjustment is required.

[0210] Understandably, the physical meaning of the systematic deviation amplitude is as follows: the normalized center distance is a quantitative indicator of the relative deviation between two targets in spatial position, and the intersection-union ratio is a quantitative indicator of the consistency between two targets in spatial coverage, directly reflecting whether the boundaries of the targets match. The systematic deviation amplitude obtained by subtracting the normalized center distance from the intersection-union ratio is a dimensionless and comparable measure that comprehensively quantifies the overall degree of deviation of the target spatial matching from the ideal state caused by system factors. It can also be used to locate the source of deviation and evaluate system performance.

[0211] Multimodal pre-fusion and accuracy assessment module: If necessary, pre-fusion of functional modal and structural modal data is performed, and the pre-fusion accuracy is calculated. By comparing the pre-fusion accuracy with the fusion accuracy under the condition of no systematic deviation, the fusion accuracy deviation is determined.

[0212] The calculation process for the pre-fusion accuracy includes:

[0213] Data cleaning, spatial / temporal registration, and feature standardization are performed on the raw data of functional and structural modes.

[0214] The core of pre-fusion is to simplify the fusion logic and ensure consistency with history. Therefore, a fusion strategy with low complexity and high reproducibility is preferred.

[0215] Data layer pre-fusion (suitable for situations where functional modality and structural modality data have the same dimensionality, such as both being 2D images or 3D voxel data):

[0216] Preliminary integration is achieved using feature splicing or weighted averaging:

[0217] Feature concatenation: The functional modality feature matrix and the structural modality feature matrix are concatenated along the channel dimension (e.g., in a 2D image, the single-channel features of the functional modality and the single-channel features of the structural modality are concatenated into a 2-channel feature matrix).

[0218] The functional modal feature matrix and the structural modal feature matrix are obtained by extracting key features that reflect the functional state or anatomical entity from the original modal data and organizing these features into a matrix form that meets the requirements of subsequent fusion (such as feature splicing).

[0219] Weighted average: The eigenvalues ​​of the functional mode and the structural mode are weighted and summed based on the importance ratio of the two modes in historical data.

[0220] Feature layer pre-fusion (suitable for functional modality and structural modality data with significant dimensional differences, such as functional modality being 1D time-series signals and structural modality being 3D voxel data):

[0221] First, extract core features for each of the two modes (e.g., extract peak value and mean value of time-series signal for functional mode, and extract geometric features such as volume and surface area for structural mode), and then concatenate the extracted feature vectors into a unified fusion feature vector.

[0222] Pre-fusion accuracy needs to be calculated by quantifying the complementarity and consistency of information after fusion of two modes. Normalized mutual information is selected as the comparison index. Its core advantage is that it is applicable to various modes (images, signals, geometric data, etc.) and can directly reflect the degree of overlap and complementarity of information of two modes after fusion. It is a general index for evaluating the accuracy of multimodal fusion.

[0223] Mutual information (MI) is used to measure the degree of correlation between two random variables (in this case, functional modal features X and structural modal features Y), that is, the amount of information about Y obtained through X. NMI is a normalization of MI, which eliminates the influence of feature dimension and value range on absolute value, and allows the precision value to be compared horizontally in the [0,1] interval.

[0224] The calculation formula is: ,in:

[0225] , where p(x,y) is the joint probability distribution of X and Y, and p(x) and p(y) are the marginal probability distributions of X and Y, respectively;

[0226] The information entropy of X measures the uncertainty of X;

[0227] Let Y be the information entropy, which measures the uncertainty of Y.

[0228] It should be noted that the value of normalized mutual information (NMI) has the following meanings: NMI=1 indicates that the two modal features are completely correlated, the information after fusion is completely complementary and without redundancy, and the accuracy is optimal; NMI=0 indicates that the two modal features are completely independent, there is no information gain after fusion, and the accuracy is worst. In practical applications, the closer NMI is to 1, the better the pre-fusion effect is; conversely, it indicates that the current modal bias leads to insufficient correlation of fused information.

[0229] The calculation process for the fusion accuracy deviation includes:

[0230] A stability analysis is performed on the normalized mutual information (NMI) under the historical unbiased condition. If it is stable, the mean of the normalized mutual information (NMI) is taken as the normalized mutual information (NMI) under normal conditions. Otherwise, if it is unstable, the maximum value of the historical normalized mutual information (NMI) is taken as the normal baseline value.

[0231] The fusion accuracy deviation ∆NMI is the difference between the pre-fused NMI and the NMI reference value;

[0232] If ∆NMI≥0, it means that the current pre-fusion accuracy is not affected by the deviation. Conversely, if ∆NMI<0, it means that the current deviation causes the fusion accuracy to decrease. The larger the absolute value of ∆NMI, the more serious the accuracy loss.

[0233] Deviation-Accuracy Correlation Construction Module: Performs correlation analysis on the magnitude of systematic deviation and the fusion accuracy deviation to determine the correlation model between the magnitude of systematic deviation and the fusion accuracy deviation.

[0234] The process of determining the correlation model between the systematic deviation magnitude and the fusion accuracy deviation includes:

[0235] Obtain the calculation results of the modal feature region comparison step in historical detection, the equipment log operation records, and the calculation results of the fusion accuracy verification step.

[0236] The systematic deviation data includes the deviation magnitude of the characteristic regions of each mode.

[0237] Linear regression was used to linearly fit the systematic bias and the fusion accuracy bias. The model expression is Y=aX+b+ε, where Y is the fusion accuracy index, X is the bias amplitude, a and b are the regression coefficients obtained by the least squares method, and ε is the random error term.

[0238] Calculate the coefficient of determination R 2 R 2 =1-(Sum of squared residuals / Sum of squared total deviations), with a value range of [0,1]. The closer R² is to 1, the stronger the linear explanatory power of X for Y (i.e., R² proportion of the change in Y is caused by the linear change of X). The closer R² is to 0, the worse the linear fitting effect.

[0239] The coefficient of determination R 2 Compared with a preset threshold, if the coefficient of determination R 2 If the deviation is greater than or equal to the preset threshold, the relationship between the systematic deviation magnitude and the fusion accuracy deviation is linear; otherwise, the relationship is non-linear.

[0240] If the magnitude of the systematic deviation and the fusion accuracy deviation are linearly related, then the fitting equation obtained by linear fitting is the correlation model between the magnitude of the systematic deviation and the fusion accuracy deviation.

[0241] If the magnitude of the systematic bias and the fusion accuracy bias have a non-linear relationship, a non-linear model is selected based on the data trend, for example:

[0242] If Y increases or decreases exponentially with X, choose the exponential model. ;

[0243] If Y grows rapidly with X at first and then slowly, gradually leveling off, then the logarithmic model Y=a∙ln(X)+b is chosen;

[0244] If Y follows a quadratic (parabolic) or cubic curve trend with X, then the polynomial model Y = a0 + a1X + a2X is chosen. 2 +...+a n X n ;

[0245] If Y and X have a power-law relationship, choose the power function model. ;

[0246] Iterative optimization algorithms are used to solve nonlinear models, for example:

[0247] Gauss-Newton method: It updates parameters iteratively through linear approximation, and is suitable for scenarios where the residuals are close to a normal distribution.

[0248] The model is evaluated using R² (adjusted for the impact of sample size and number of parameters on R²), the Akaike Information Criterion (AIC), or the Bayesian Information Criterion (BIC):

[0249] The higher the adjusted R², the better; the lower the AIC / BIC, the better (prioritize the model with the smallest AIC / BIC).

[0250] Residual analysis: linear regression was used to verify whether the residuals were randomly distributed (without systematic trend) to ensure that the model had no significant bias.

[0251] From multiple candidate nonlinear models, the model with the highest adjusted R², the lowest AIC / BIC, and residuals that meet the assumptions is selected as the final correlation model;

[0252] Deviation control and hyperbolic alignment module: Based on the correlation between systematic deviation amplitude and fusion accuracy deviation, and combined with the target fusion accuracy, determine the adjustment and alignment amount of systematic deviation amplitude, and combine the hyperbolic space alignment algorithm to align the functional-structural modes.

[0253] The process of determining the adjustment amount for the magnitude of the systematic deviation includes:

[0254] The method for setting the target fusion accuracy is determined by the clinical / engineering requirements of the target task;

[0255] Substituting the target fusion accuracy into the correlation regression equation, we can solve for the target systematic bias d. 目标 ;

[0256] The adjustment amount for systematic deviation is the difference between the current deviation magnitude and the target deviation.

[0257] In step five, the alignment process using the hyperbolic space alignment algorithm includes:

[0258] Based on the spatial coordinate differences between the functional and structural modal characteristic regions, a hyperbolic transformation model with anatomical landmarks as the reference is established:

[0259] The coordinates of the structural modal feature region are (x s ,y s The coordinates of the functional modal feature region are (x... f ,y f The deviation between the two is (∆x,∆y)=(x f -x s ,y f -y s ) .

[0260] The hyperbolic transformation maps functional coordinates to the structural modal coordinate system by introducing parameters k (curvature factor) and d (reference distance):

[0261] , The curvature factor k is determined by the deviation control amount. The larger the control amount, the closer k is to 1, and the larger the correction magnitude. The calculation method is: k = control amount / X 当前 The reference distance d is taken as 1 / 5 of the average diameter of the structural modal characteristic region.

[0262] Select multiple stable anatomical landmarks (such as bone edges and acupoints) from structural modalities (e.g., ultrasound images) and record their coordinates P. s ={(x s1 ,y s1 ),(x s2 ,y s2 ),...} .

[0263] Identify identical marker points from functional modes (such as infrared thermograms) and record their coordinates P. f ={(xf1,y f1 ),(x f2 ,y f2 ),...};

[0264] Calculate the initial deviation field: Based on the coordinate difference of the marker points, the global deviation field D(x,y)=(∆x(x,y),∆y(x,y)) of the functional mode and structural mode is generated by interpolation, which reflects the deviation distribution of the whole map.

[0265] For each pixel (x) of the functional modality feature region f ,y f ), mapped to structural modal coordinates (x) through the hyperbolic transformation formula. s ,y s ), generating aligned functional modal feature regions.

[0266] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A Chinese meridian detection method based on multi-modal data fusion, characterized in that: Comprise: Step one: deviation analysis of the spatial mapping of the functional and structural modalities of the meridians, and stability and direction consistency verification of the deviation, to determine whether there is systematic deviation in the spatial mapping of the functional and structural modalities; Step two: if there is, determine the magnitude of the systematic deviation according to the stability and direction consistency of the systematic deviation, and determine whether the functional and structural modalities need to be regulated and aligned through comparison and analysis of the magnitude of the systematic deviation; Step three: if needed, pre-fuse the functional and structural modality data, and calculate the pre-fusion accuracy, and determine the fusion accuracy deviation by comparing and analyzing the pre-fusion accuracy with the fusion accuracy without systematic deviation; Step four: correlation analysis of the magnitude of the systematic deviation and the fusion accuracy deviation to determine the correlation model of the magnitude of the systematic deviation and the fusion accuracy deviation; Step five: according to the correlation between the magnitude of the systematic deviation and the fusion accuracy deviation, and combining the target fusion accuracy, determine the amount of regulation and alignment of the magnitude of the systematic deviation, and align the functional-structural modalities using the hyperbolic space alignment algorithm; Wherein, the determination method of the correlation model of the magnitude of the systematic deviation and the fusion accuracy deviation is: The linear regression is used to linearly fit the systematic deviation and the fusion precision deviation, and a determination coefficient R is calculated 2 ; The decision coefficient R 2 is compared with a preset threshold value, if the decision coefficient R 2 is greater than or equal to the preset threshold value, the systematic deviation amplitude and the fusion precision deviation are in a linear relationship, otherwise, the systematic deviation amplitude and the fusion precision deviation are in a nonlinear relationship. If the magnitude of the systematic deviation and the fusion accuracy deviation are linearly related, the fitting equation obtained by linear fitting is the correlation model of the magnitude of the systematic deviation and the fusion accuracy deviation; If the magnitude of the systematic deviation and the fusion accuracy deviation are nonlinear, select candidate nonlinear models based on data trends, and solve the candidate nonlinear models using an iterative optimization algorithm; From the plurality of candidate nonlinear models, a decision coefficient R is selected 2 The highest nonlinear model is selected as the correlation model of the systematic deviation amplitude and the fusion accuracy deviation The process of alignment using the hyperbolic space alignment algorithm is: Coordinates of the structural modal feature region are (x s ,y s ), coordinates of the functional modal feature region are (x f ,y f ), and the deviation between the two is (Δx, Δy) = (x f -x s ,y f -y s ); Hyperbolic transformation maps the functional coordinates to the structural modality coordinate system by introducing a parameter curvature factor k and a reference distance d: , wherein the curvature factor k is the ratio of the systematic deviation amplitude control quantity and the current systematic deviation amplitude, and the reference distance d is 1 / 5 of the average diameter of the structural modal characteristic region. Selecting a plurality of anatomical landmark points from the structural mode, recording coordinates P s ={(x s1 ,y s1 ),(x s2 ,y s2 ),...} Identify the same landmark points from the functional modality, record the coordinates P f {(xf1,y f1 ),(x f2 ,y f2 ),...} Calculate the initial deviation field: based on the coordinate difference of the landmark points, generate the global deviation field D(x,y)=(∆x(x,y),∆y(x,y)) of the functional and structural modalities by interpolation method, reflecting the deviation distribution of the whole image; For each pixel (x f ,y f ) of the functional modality feature region, a hyperbolic transformation formula is used to map to the structural modality coordinate (x s ,y s ), and an aligned functional modality feature region is generated.

2. The meridian detection method based on multi-modality data fusion according to claim 1, characterized in that: The process of deviation analysis of the spatial mapping of the functional and structural modalities of the meridians is: Extract the functional modality feature region and the structural modality feature region from the functional modality and the structural modality original collection data of the same detected person and the same anatomical region to generate multiple samples; For any one sample: Generate an offset vector through the coordinate difference of the center points of the functional feature region and the structural feature region, the direction of the offset vector is the deviation direction, and the module length is the center distance; Obtain the pixel set of the functional feature region and the pixel set of the structural feature region from the binary image region obtained after feature extraction; Calculate the intersection and union of the pixel sets of the functional feature region and the structural feature region, and perform intersection and union ratio calculation on the pixels to obtain the intersection-union ratio.

3. The meridian detection method based on multi-modality data fusion according to claim 2, characterized in that: The judgment method of whether there is systematic deviation in the spatial mapping of the functional and structural modalities is: Calculate the variation coefficients of the center distance and the intersection-over-union of all samples, and if the variation coefficients of the center distance and the intersection-over-union are both less than a threshold value, the center distance and the intersection-over-union of all samples are stable; Statistically determine the category with the highest proportion of offset direction in the sample, and compare the proportion with a preset proportion, and if the proportion is greater than the preset proportion, it is determined that the sample offset direction is consistent; If the center distance and the intersection-over-union of the sample are both stable, and the offset direction is consistent, it is determined that the spatial mapping of the functional mode and the structural mode has a systematic deviation; If the center distance and the intersection-over-union of all samples are stable, and the proportion of samples with consistent offset direction is greater than a preset proportion, it is determined that the spatial mapping of the functional mode and the structural mode has a systematic deviation.

4. The traditional Chinese meridian detection method based on multi-modal data fusion according to claim 1, characterized in that: The way of determining whether the functional mode and the structural mode need to be regulated and aligned is: Calculate the mean of the center distance and the intersection-over-union of all samples as the center distance and the intersection-over-union of the systematic deviation; Subtract the intersection-over-union from the normalized center distance of the functional feature region and the structural feature region to obtain the systematic deviation amplitude; Compare the systematic deviation amplitude with a preset deviation, and if the systematic deviation amplitude is greater than the preset deviation, regulation and alignment are needed.

5. The traditional Chinese meridian detection method based on multi-modal data fusion according to claim 1, characterized in that: The calculation method of the pre-fusion accuracy is: Data cleaning, spatial / time registration, and feature standardization are performed on the multi-modal raw data; If the data dimensions of the functional mode and the structural mode are consistent, the functional mode feature matrix and the structural mode feature matrix are spliced according to the channel dimension to pre-fuse the data layers of the functional mode and the structural mode; If the data dimensions of the functional mode and the structural mode are inconsistent, core features are extracted from the functional mode and the structural mode respectively, and then the extracted feature vectors are spliced into a unified fusion feature vector to realize pre-fusion at the feature layer; Wherein, the core features of the functional mode and the structural mode refer to the peak value and the mean value of the time series signal extracted from the functional mode, and the geometric features such as volume and surface area extracted from the structural mode; The normalized mutual information of the fused functional mode features and the structural mode features is the pre-fusion accuracy.

6. The traditional Chinese meridian detection method based on multi-modal data fusion according to claim 5, characterized in that: The calculation method of the fusion accuracy deviation is: Stability analysis is performed on the normalized mutual information after fusion under the condition of no historical deviation; If stable, take the mean of the normalized mutual information as the reference value of the normalized mutual information, otherwise, if not stable, take the maximum of the historical normalized mutual information as the reference value of the normalized mutual information; The fusion accuracy deviation is the difference between the pre-fusion normalized mutual information and the reference value of the normalized mutual information.

7. The traditional Chinese meridian detection method based on multi-modal data fusion according to claim 1, characterized in that: The determination method of the regulation amount of the systematic deviation amplitude is: Set the accuracy target according to the clinical / engineering requirements of the target task, and substitute the accuracy target into the correlation regression equation of the systematic deviation amplitude and the fusion accuracy deviation to solve the target systematic deviation amplitude. The regulation amount of the systematic deviation amplitude is a difference between the current deviation amplitude and the target systematic deviation amplitude.

8. A traditional Chinese meridian detection system based on multi-modal data fusion, characterized in that: The system comprises the following modules: A systematic deviation determination module: performing deviation analysis on the spatial mapping of the functional mode and the structural mode of the meridian and verifying the stability and direction consistency of the deviation to determine whether the spatial mapping of the functional mode and the structural mode of the meridian has systematic deviation; A deviation quantification and alignment decision module: if there is, determining the systematic deviation amplitude according to the stability and direction consistency of the systematic deviation and determining whether the functional mode and the structural mode need to be regulated and aligned through comparison and analysis of the systematic deviation amplitude; A multi-modal pre-fusion and precision evaluation module: if needed, pre-fusing the functional mode and the structural mode data and calculating the pre-fusion precision, and determining the fusion precision deviation through comparison and analysis of the pre-fusion precision and the fusion precision without systematic deviation; A deviation-precision correlation construction module: performing correlation analysis on the systematic deviation amplitude and the fusion precision deviation to determine the correlation model of the systematic deviation amplitude and the fusion precision deviation; A deviation regulation and hyperbolic curve alignment module: determining the regulation and alignment amount of the systematic deviation amplitude according to the correlation between the systematic deviation amplitude and the fusion precision deviation and combining the target fusion precision, and aligning the functional-structural mode by combining the hyperbolic curve spatial alignment algorithm; The determination of the correlation model of the systematic deviation amplitude and the fusion precision deviation is as follows: The linear regression is used to linearly fit the systematic deviation and the fusion precision deviation, and a determination coefficient R is calculated 2 ; The decision coefficient R 2 is compared with a preset threshold value, if the decision coefficient R 2 is greater than or equal to the preset threshold value, the systematic deviation amplitude and the fusion precision deviation are in a linear relationship, otherwise, the relationship is nonlinear. If the systematic deviation amplitude and the fusion precision deviation are linearly related, the fitting equation obtained by linear fitting is the correlation model of the systematic deviation amplitude and the fusion precision deviation; If the systematic deviation amplitude and the fusion precision deviation are nonlinearly related, candidate nonlinear models are selected based on the data trend, and an iterative optimization algorithm is used to solve the candidate nonlinear models; From the plurality of candidate non-linear models, a decision coefficient R is selected 2 The highest non-linear model is selected as the correlation model of the systematic deviation amplitude and the fusion accuracy deviation The process of alignment by combining the hyperbolic curve spatial alignment algorithm is as follows: Coordinates of the structural modal feature region are (x s ,y s ), coordinates of the functional modal feature region are (x f ,y f ), and the deviation between the two is (Δx, Δy) = (x f -x s ,y f -y s ). Hyperbolic curve transformation maps the functional coordinates to the structural mode coordinate system by introducing a parameter curvature factor k and a reference distance d: , wherein the curvature factor k is the ratio of the systematic deviation amplitude control quantity and the current systematic deviation amplitude, and the reference distance d is 1 / 5 of the average diameter of the structural modal characteristic region. Selecting a plurality of anatomical landmark points from the structural mode, recording coordinates P s ={(x s1 ,y s1 ),(x s2 ,y s2 ),...} Identify identical marker points from functional modes and record their coordinates P. f ={(xf1,y f1 ),(x f2 ,y f2 ),...}; Calculate the initial deviation field: based on the coordinate difference of the landmark points, generate the global deviation field D(x,y)=(∆x(x,y),∆y(x,y)) of the functional mode and the structural mode by interpolation method to reflect the deviation distribution of the whole image; For each pixel (x f ,y f ) of the functional modality feature region, a hyperbolic transformation formula is used to map to the structural modality coordinate (x s ,y s ), and an aligned functional modality feature region is generated.

Images