A tumor recognition method based on multi-modal information cooperation

Abstract

The application provides a tumor recognition method based on multi-modal information cooperation, and relates to the technical field of tumor recognition. First, a multi-modal cooperative representation unit is constructed to synchronously receive medical images and electrical impedance time series data, and multi-domain coupled representation space technology is used to cooperatively mine tumor macroscopic morphological correlation features and microscopic difference features, and output a multi-modal cooperative feature vector. Then, an electrical impedance time series analysis unit is built to extract deep correlation features of electrical impedance signals through multi-order analysis of time series features, and output an electrical impedance deep feature vector. Next, a dynamic feature adaptive fusion unit is designed to adaptively fuse and dimensionally reorganize the multi-modal cooperative feature vector and the electrical impedance deep feature vector based on feature response degree adaptation technology, and output a stabilized fusion feature vector. Finally, a feature correlation judgment engine is constructed to quantify feature contribution based on structural causal models and multi-layer correlation verification technology, and output a tumor diagnosis result. The application effectively solves the problems of insufficient multi-modal feature cooperation and interpretive bias caused by black box mapping in the prior art, and improves the accuracy and logical robustness of tumor recognition.

Description

Technical Field

[0001] This invention relates to the field of tumor identification technology, and in particular to a tumor identification method based on multimodal information collaboration. Background Technology

[0002] Improving the clinical treatment effect of tumors and prolonging the survival of patients depends heavily on the early detection and identification of tumors. At present, clinical diagnosis mainly relies on medical imaging examinations and electrophysiological tests. Among the existing technologies, identification schemes based on a single modality have obvious limitations. Medical imaging can intuitively show the macroscopic anatomical morphology of tumors, but it is not sensitive enough to the complex electrophysiological metabolic abnormalities inside tumor tissues. While electrical impedance testing has extremely high sensitivity to functional changes in tissues, it is weak in spatial resolution and morphological localization.

[0003] To overcome the limitations of single-modality approaches, several multimodal fusion schemes have emerged in recent years, such as the Chinese patent CN115830017A, which discloses a tumor detection system based on image-text multimodal fusion. This system improves detection accuracy by fusing image feature vectors with text feature vectors from electronic medical records. However, such existing multimodal solutions still face the following difficulties in practical applications: Existing fusion technologies often focus on the simple cascading of independent features from different modalities, lacking analysis of interdomain coupling between macroscopic morphology of images and microscopic differences in electrical impedance. This makes it difficult to capture the deep correlation between the physical structure and physiological function of diseased tissues. Due to the significant differences in amplitude dimension and distribution characteristics between medical image data and electrical impedance time-series signals, conventional weight allocation methods can easily lead to the model over-reliance on a certain modality, resulting in an imbalance in fusion weights and affecting the model's generalization stability in multi-center and cross-device scenarios. Most existing AI recognition models are based on statistical correlation modeling, which is a black-box mapping. They cannot distinguish between true pathological features and spurious correlation interference from unobserved tissue background, resulting in poor interpretability of diagnostic suggestions and difficulty in meeting the causal logic requirements of the medical field.

[0004] In summary, existing technologies for deep fusion of medical images and electrical impedance signals still suffer from problems such as insufficient information collaboration, low analytical accuracy, large fusion deviation, and lack of logical interpretation. There is an urgent need for a tumor identification method that is more causal robust and can achieve multimodal deep collaboration. Summary of the Invention

[0005] The main objective of this invention is to provide a tumor identification method based on multimodal information collaboration, which solves the problems of insufficient multimodal fusion, strong spurious interference, and limited identification accuracy.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a tumor identification method based on multimodal information collaboration, the method comprising the following steps: S1. Construct a multimodal collaborative representation unit, simultaneously receive medical imaging and electrical impedance time-series data, and based on multi-domain coupled representation space technology, collaboratively mine the macroscopic morphological correlation features and microscopic difference features of tumors, and output a multimodal collaborative feature vector. .

[0007] S2. Construct a impedance timing analysis unit to process impedance timing data independently. Using multi-level timing feature analysis techniques, extract deep correlation features of the impedance signal and output a deep impedance feature vector. .

[0008] S3. Design a dynamic feature adaptation and fusion unit. Based on feature responsivity adaptation technology, adaptively fuse and reorganize the dual-channel features output from S1 and S2, and output a stable fused feature vector. .

[0009] S4. Construct a feature association determination engine, based on multi-layer association verification technology, quantify feature contribution, and output high-precision tumor diagnosis results.

[0010] In the preferred embodiment, step S1 specifically includes the following steps: S11. Use hardware trigger pulses to synchronously start medical imaging equipment and impedance acquisition device to obtain tumor tissue structure slices and electrical impedance sequences at the same time.

[0011] S12. Map the medical image pixel grid points to the image representation domain, and define the mapping relationship as follows: (1); in, It is an intra-domain mapping operator that realizes the transformation from raw pixel information to the physical properties of the characterization factor; Within the image representation domain Characterization factor at coordinates; coordinates Standardized baseline values ​​for the image; Output characterization factor This is used to characterize the associated information of tissue density at that location.

[0012] S13. Perform time-to-space domain mapping on the impedance-time signal $Z(t)$ to obtain the time-series representation vector. The formula is: (2); in, For dimension The time-series representation vector reflects the topological evolution of the impedance signal over time; for The original signal strength of the impedance at any given moment; The sampling time step is determined by the first minimum value of the signal correlation function within the domain, maintaining the independence of representation between adjacent points; The mapping order is determined through information integrity verification.

[0013] S14. The multimodal feature extraction process is viewed as a process of minimizing the system's potential energy. Feature extraction is transformed into a process of finding a physical steady state in an energy field composed of image and electrical impedance data. A self-created multi-domain coupling correlation function is used. The formula for integrating dual-domain information is as follows: (3); in, Let be the total coupling potential function of the system; The summation of the interaction energies of spatially adjacent grid point pairs describes the local consistency of the organization; The correlation coefficient is set to the intra-domain correlation coefficient. As an example, the suspected tumor area is set to a higher coefficient to enhance the correlation of lesions, and the normal tissue is set to a lower coefficient to suppress background interference. For grid points Characteristic factors at the location; This is a parameter for adjusting heterogeneity of the organization, used to balance the self-energy of the lattice points; This is a smoothing coefficient used to control the magnitude of abrupt changes in the image gradient; This represents the gradient (spatial rate of change) of the characterizing factor in space. This is the cross-domain coupling coefficient, used to adjust the constraint strength of the temporal domain of electrical impedance on the feature distribution of the image domain; As a time series transformation operator, it transforms the time series representation vector... Transformed into an energy distribution term that adapts to the image space; By introducing gradient terms and cross-domain coupling terms, mathematical mappings are given the ability to simulate physical evolution properties, enabling a more accurate description of the physical field interaction between tumor tissue and its surrounding environment.

[0014] S15. The stable representation state is solved using intra-domain evolution convergence, and the formula is: (4); in, This represents the stable state vector after the system evolves to a steady state. For evolution parameters A dynamic representation function of change; The rate of change of the characterizing function; The total energy of the system is minimized through iterative calculations. When the rate of change is less than a preset threshold, a steady-state field reflecting the physical properties of the tumor is output. .

[0015] S16. Using the spatial frequency domain operator decomposition method to... Extracting macro-level correlation features and microscopic differences The formula is: (5); (6); in, To use low-pass operators Macroscopic morphological features such as tumor boundaries, shape, and depth of invasion were extracted; To use the high-pass residual operator Functional anomalous features represented by localized minute perturbations captured in a steady-state field; It is a convolution operator.

[0016] S17. Employing the covariance-aligned weighted fusion method for fusion. Obtain multimodal collaborative feature vectors The formula is: (7); in, This represents a multimodal collaborative feature vector. To minimize The feature space alignment matrix obtained from mutual information; These are weighting coefficients determined based on feature entropy values; To achieve the reorganization of macro and micro information.

[0017] In the preferred embodiment, step S2 specifically includes the following steps: S21. Employ a multi-stage physical filtering algorithm to process the original signal. To perform purification, the first step is to correct baseline drift using the formula: (8); in, for Polynomial fitting of order one is used to subtract slow fluctuations caused by breathing, etc. Subsequently, power frequency interference filtering is performed using the following formula: (9); in, In response to or The set frequency response function of the notch filter.

[0018] Finally, normalization is performed, and the formula is: (10); in, The mean of the signal; Standard deviation; Output purified signal .

[0019] S22. Process using the same method as S13. The purified time-series representation vector is obtained. .

[0020] S23, using time-series trajectory analytical functions The formula for extracting evolutionary patterns is: (11); in, For trajectory analytical operators that include multi-order difference features; Final output As a relational matrix obtained through signal fitting optimization, it is used to uncover the deeper patterns of changes in electrical impedance with tissue metabolism.

[0021] S24. The evolution intensity of the signal is calculated using the multidimensional correlation integral method, with the following formula: (12); (13); in, For the correlation integral function; It is a step function; The radius is the measure. By introducing This allows its complex quantification to focus on the evolutionary trajectory identified by S23, strengthening the transmission correlation; Final output , serving as a comprehensive correlation strength parameter to characterize the electrophysiological complexity of tissues.

[0022] S25. Construct a temporal evolution trajectory structure representation model, and use continuous cohomology technology to capture the geometric topological features of electrical impedance temporal signals at multiple scales to identify subtle tissue malignancy signals. First, a series of nested simple complexes are constructed on the time-series point cloud data, and their sequence is given as: (14); Subsequently, the corresponding value for each complex is calculated. Conflict groups Record the generation and disappearance times of topological features such as connected components and holes; Next, record the birth points of all topological features. With vanishing point Build a persistent point set ; Finally, the difference between the current signal topology and the standard health model is quantified using the Wasserstein distance, as shown in the formula: (15); in, The Wasserstein distance between the two persistent graphs; These are the continuity graphs of the signal under test and the reference signal, respectively. For bijective matching between point sets; The distance order; This process suppresses random noise during impedance acquisition and accurately identifies early-stage electrophysiological microenvironmental mutations in tumors that are difficult to detect with conventional linear analysis.

[0023] S26. By fusing the comprehensive parameters of S24 and the topological features of S25 using a nonlinear projection fusion operator, the deep feature vector of electrical impedance is obtained. The formula is: (16); in, This represents the deep feature vector of electrical impedance; For characteristic cascade operators; For the set of topologically persistent feature points from S25; The kernel mapping projection function maps heterogeneous features to a high-dimensional consistent representation space.

[0024] In the preferred embodiment, step S3 specifically includes the following steps: S31. Based on the Transformer architecture, the multimodal collaborative feature $F_c$ and the impedance deep feature $F_r$ are dynamically recombined to achieve nonlinear fusion of the two types of modal features; First, a linear transformation is performed on the input features to obtain the Query, Key, and Value matrices, as shown in the formula: (17); (18); (19); in, This is a learnable weight parameter matrix; Then, the cross-attention matrix is ​​calculated and feature reorganization is completed, as shown in the formula: (20); in, This is the initial fusion feature vector after recombination; This is a scaling factor for the feature dimension, used to prevent gradient vanishing or over-divergence; By using the Cross-Attention mechanism, the evolution law of electrical impedance K and V is most diagnostically discriminative when Q-mode is presented, thus realizing dynamic information blinding between modes.

[0025] S32. Implemented after the cross-attention mechanism. Normalization eliminates network bias caused by different physical dimensions in data from different modalities. (twenty one); in, This is the final stabilized fused feature vector; The square norm of the eigenvectors; To prevent smoothing constraint values ​​with a denominator of 0; Cross-attention is responsible for determining weights at the topology level. Normalization projects all features onto a uniform unit hypersphere at the intensity level, stabilizing the weight allocation when calculating contribution and avoiding deviations due to excessively large amplitudes of a certain mode signal.

[0026] In the preferred embodiment, step S4 specifically includes the following steps: S41. Construct a relational network framework based on a structural causal model, clarifying the transmission logic between variables, including: The input variable set is: (twenty two); in, Represents multimodal collaborative feature vectors Relevant unobserved tissue background factors, Represents the deep feature vector of the corresponding electrical impedance. Relevant unobserved tissue background factors; The core variable set is: (twenty three); in, Corresponding to the fused feature vectors The cooperating components and impedance components in; The result variable is That is, the conclusion of a tumor diagnosis. benign, Malignant; Causal mapping rule orientation description: (twenty four); (25); By explicit modeling This enables the model to identify and eliminate spurious correlation interference.

[0027] S42, will Through built-in adaptation mapping operator Transform into core variables and The formula is: (26); in, These are the core variable nodes of the mapped causal network; These are the corresponding feature components; To fit the optimized mapping parameters and ensure the accuracy of information conversion; Complete the adaptation mapping between fusion features and network variables.

[0028] S43. Use do-calculus to perform multi-level causal relationship verification on core variables; First, an intervention baseline value is established, and the statistical mean of healthy organizations is selected as the standard baseline value. ; Subsequently, correlation intervention deduction was performed to fix [the issue]. ,shield The path, calculating the diagnostic predictive value after intervention. The formula is: (27); Similarly, calculate ; By comparing the deviations between actual observed values ​​and predicted values ​​after intervention, we can verify whether there is a robust causal relationship between input features and diagnostic conclusions.

[0029] S44. Calculate the pure contribution of each feature component using the contribution expectation operator. The formula is: (28); in, For mathematical expectation; By analyzing unobserved factors The probability integral of the distribution is used to eliminate background clutter; Finally, based on the high-contribution feature set, a high-precision determination result of tumor benignity or malignancy is output. .

[0030] This invention provides a tumor identification method based on multimodal information collaboration. By constructing a multi-domain coupled representation space, it synchronously and collaboratively processes the macroscopic morphological features of medical images and the microscopic differences in electrical impedance time-series data. Compared with traditional single-modal identification or simple feature stacking, it overcomes the limitations of single-modal identification in describing tumor heterogeneity and improves the feature mining depth for complex lesions. By performing multi-level analysis on electrical impedance time-series data, it extracts deep correlation features, effectively capturing the weak fluctuations and high-dimensional temporal evolution patterns of lesion tissue at the electrophysiological level, enhancing the model's sensitivity in identifying early-stage small tumors and borderline lesions. It employs a self-reactivity-based approach. By adapting to fusion technology and combining topological weighting and L2 normalization constraints of cross-attention, heterogeneous features are projected onto a unified unit hypersphere, solving the problem of weight allocation bias caused by excessive differences in the amplitude of multimodal signals. This ensures the stability of feature vectors during the recombination process and improves the model's generalization ability under different device data. Furthermore, an association judgment engine based on a structural causal model is introduced. By explicitly modeling unobserved tissue background factors, it is able to identify and eliminate spurious correlation interference. This quantifies the feature contribution from the perspective of causal logic, so that tumor identification results no longer rely solely on statistical correlation, thus improving the scientificity, accuracy, and clinical interpretability of diagnostic conclusions. In summary, this invention integrates the complementary advantages of medical imaging and electrical impedance information through a full-link technology synergy of multimodal collaborative characterization, temporal multi-level analysis, dynamic adaptation fusion, and causal correlation determination. This solves the technical problems of insufficient multimodal fusion, strong spurious correlation interference, and limited recognition accuracy in existing technologies, thereby improving the accuracy and clinical practical value of tumor benign and malignant identification. Attached Figure Description

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of a tumor identification method based on multimodal information collaboration according to the present invention; Detailed Implementation In the following embodiments, the medical imaging equipment used is a Siemens SOMATOM Force 64-slice spiral CT scanner to acquire lung tissue structure slices; the impedance acquisition device uses an ADI AD5941 impedance spectrum analyzer with an impedance detection electrode array; the main control chip uses the NVIDIA Jetson AGX Xavier edge computing platform, which is responsible for the synchronous processing, feature extraction and fusion calculation of multimodal data; the synchronization control module uses an Altera Cyclone V FPGA to realize the synchronous triggering of the CT equipment and the impedance acquisition device.

[0032] Example 1 like Figure 1As shown, a tumor identification method based on multimodal information collaboration is proposed, which includes the following steps: In the preferred embodiment, step S1 includes the following steps: In the preferred embodiment, step S11: The Siemens SOMATOM Force CT and ADI AD5941 impedance spectroscopy analyzer are synchronously started using a hardware trigger pulse generated by the FPGA module; the trigger pulse adopts a TTL level signal, with a high level of 3.3V and a low level of 0V, and a pulse width of 10μs. The clock is calibrated by the PLL phase-locked loop module of the FPGA to ensure synchronization. Since the macroscopic morphology and microscopic metabolic characteristics of small lung tumors are highly correlated in time, the error is controlled within ±0.5ms. This precision can accurately match the spatiotemporal correspondence between the two and avoid feature misalignment caused by synchronization delay. During lung CT scans, impedance sequences of the corresponding regions are acquired simultaneously. The impedance acquisition area is precisely aligned with the CT scan area using laser positioning, with an acquisition time of 10 seconds, obtaining lung tumor tissue structure slices and impedance sequences at the same time.

[0033] In the preferred embodiment, step S12: Map the pixel grid of the CT image to the image representation domain using formula (1); where, coordinates The value range is the index of the CT image pixel matrix, that is... , Standardized base value The calculation incorporates standard window width and level adjustments for lung CT diagnosis, with the window width set to 1500 HU and the window level set to -600 HU, to differentiate between lung tissue, tumor nodules, and air regions. More accurately reflecting differences in tissue density. The specific calculation method is as follows: ; in coordinates The original CT values ​​at the location, ranging from -1000 to 1000, correspond to the area from air to bone. After conversion using this formula... The value range is 0-255, which avoids numerical overflow and adapts to the calculation precision of subsequent feature extraction. Intradomain mapping operator The nonlinear Sigmoid transform is used, and the formula is: ; in , Its values ​​are obtained from clinical samples and can expand [the scope of the study]. The suspected tumor area and Characterization factors of normal tissue regions Differences enhance the differentiation of lesion areas; The final output characterization factor Correlation information used to characterize the density of lung tissue at this location, normal alveolar tissue Inflammatory nodules, concentrated between 20-50. Concentrated between 50-80, malignant tumors The number of organizations is concentrated between 80 and 200, achieving a preliminary distinction between organizational types.

[0034] In the preferred embodiment, step S13: For impedance timing signals Perform a temporal-to-spatial domain mapping and obtain the temporal representation vector according to formula (2). Among them, the original signal strength of the impedance. The unit is Ω, and the typical range of lung tissue impedance is 500-5000Ω, which is consistent with the measurement range of ADI AD5941; sampling time step The signal correlation function within the domain is determined by its first minimum value. The signal correlation function within the domain is defined as follows: ; in This represents the number of sampling points; Based on historical experience, when hour, Taking the first minimum value, where the representation independence of adjacent sampling points is good, it can better preserve temporal evolution information and avoid information redundancy or loss; mapping order The information integrity indicators, as determined through information integrity verification, are: ; when At that time, determine The time series representation vector is: ; It can cover most impedance signal evolution information; like The information coverage is too low, making it impossible to capture the complete laws of metabolic evolution; like Insufficient information gain and redundant feature dimensions will increase computational pressure. Therefore, the time-series representation vector It is a 10-dimensional vector used to reflect the evolution topology of the impedance signal within a 72ms time window.

[0035] In the preferred embodiment, step S14: Through a self-created formula (3), multi-domain coupling correlation functions Integrating image domain and electrical impedance time-series information, feature extraction is transformed into a process of minimizing system potential energy; among which, the intra-domain correlation coefficient An adaptive adjustment strategy is adopted, based on grid points. of The difference is set dynamically, and the specific formula is as follows: ; when and When the difference is less than 20, they belong to the same tissue type. Approaching 0.8 strengthens the local consistency in this region; when and A difference greater than 80 indicates different tissue types. Approaching 0.2, it suppresses interference between different tissues; This adaptive strategy is used to match the heterogeneous distribution of lung tissue and improve feature discrimination. Tissue heterogeneity regulation parameters Set as: ; The higher the tissue density, The larger the value, the less likely the characterization factor of a single grid point will excessively dominate the system potential energy; Smoothing coefficient Set to 0.08, determined based on the gradient characteristics of lung CT images, it is used to control abrupt changes in image gradients, filter noise, and preserve tumor boundary information. Cross-domain coupling coefficient A dynamic adjustment mechanism is adopted, based on the correlation between the characteristics of the image domain and the temporal domain. The setting is dynamic, and the calculation method is as follows: ; ; The value range is 0-1, when Strong correlation between macroscopic and microscopic features suggests a typical malignant tumor. Enlarging strengthens cross-domain coupling; when The weak correlation between macroscopic and microscopic features suggests early-stage, small tumors. Smaller size avoids excessive coupling that obscures key features; improves cross-domain collaboration. Timing Transformation Operator Wavelet transform was used, with the db4 wavelet basis selected and a decomposition level of 3. Low-frequency approximation coefficients were extracted as the energy distribution term. The specific calculation method is as follows: ; in For the first The low-frequency coefficients of layer wavelet decomposition transform the time-series signal into an energy distribution term that adapts to the image space, while preserving the evolution law of the time-series signal.

[0036] In the preferred scheme, step S15: the stable representation state is calculated using the domain evolution convergence iteration through formula (4); where the evolution parameters are... The initial value is set to 0.05, and the iteration step size adopts an adaptive strategy, that is, when When the step size is 0.02, the convergence speed is accelerated; when At this time, the step size is 0.005 to improve convergence accuracy; the number of iterations is controlled within 200 while ensuring convergence accuracy; the preset threshold is set to 5e-7, when the rate of change of the characterization function... When the total energy of the system approaches its minimum, the iteration stops, and the stable representation vector is output. This avoids wasting computing resources due to excessive iteration; A 512×512 dimensional vector is used to reflect the physical field interaction between lung tumor tissue and its surrounding environment, where the tumor region... The value is much higher than that of normal tissue areas, reflecting the physical characteristics of lung tumor tissue.

[0037] In the preferred scheme, step S16: using formulas (5) and (6), the spatial frequency domain operator decomposition method is used to extract macroscopic correlation features. and microscopic differences ; Among them, low-pass operator A Gaussian low-pass filter is used, with a standard deviation of Extracting macroscopic features such as tumor boundaries, shape, and invasion depth, filtering out pixel-level noise; High-pass residual operator. A Laplace high-pass filter with a 3×3 kernel size was used to capture local small perturbations in the steady field, corresponding to the functional abnormalities of tumor tissue, and to present the boundary contour, shape regularity and invasion depth of the tumor. Through convolution operations It is a macroscopic feature matrix with dimensions of 512×512. It is a 512×512 dimensional micro-feature matrix that captures minute functional abnormalities within the tumor, such as the differences in micro-features between the tumor core region and the peripheral region.

[0038] In the preferred scheme, step S17: use the covariance alignment weighted fusion method to fuse the components. and The multimodal collaborative feature vector is obtained through formula (7). Among them, the feature space alignment matrix By minimizing and The maximum mean difference is obtained, and the MMD threshold is set to 0.05. When the MMD is less than 0.05, the feature space alignment is considered complete. The dimension is 512×512, and through iterative optimization, it becomes... and The distribution of the weights tends to be uniform; and The Fisher discriminant ratio is dynamically determined based on features. The Fisher discriminant ratio is: ; Through the analysis of numerous benign and malignant characteristics, the following methods were adopted: , Therefore, we can conclude that: ; ; The classification ability of a feature is reflected; features with stronger classification ability are assigned higher weights, thereby improving the effectiveness of feature fusion. The final output is a 1×(512×512×2) dimensional multimodal collaborative feature vector. This aims to integrate macroscopic and microscopic features of tumors to provide information for subsequent feature fusion and diagnostic judgment.

[0039] In the preferred embodiment, step S2 includes the following steps: In the preferred embodiment, step S21: a multi-stage physical filtering algorithm is used to process the original impedance signal. The purification process involves sequentially performing baseline drift correction, power frequency interference filtering, and normalization. Baseline drift correction is performed using formula (8), with adaptive polynomial fitting, and the optimal polynomial order is determined by the Akaike information criterion. Since the baseline drift of pulmonary impedance signals is mainly caused by slow fluctuations in respiration and heart rate, the following selection... ,like It will overfit the effective components of the signal, if The drift cannot be completely eliminated, and the corrected signal is: ; Reduce baseline drift amplitude; Power frequency interference filtering is performed using formula (9). A notch filter is designed for 50Hz power frequency interference from the mains power supply. The frequency response function of the notch filter is... An infinite impulse response filter is used, with a notch bandwidth set to 49.5Hz-50.5Hz to filter power frequency interference, while avoiding affecting the effective frequency components of the impedance signal, thus preserving more of the filtered signal. Signal-to-noise ratio; Normalization is performed using formula (10), where the signal mean is... The average value and standard deviation of 1000 sampling points within a 10-second sampling period. To correspond to the standard deviation, the normalized signal is: ; The value range is from -3 to 3, which eliminates individual differences between different subjects, makes the signals comparable, and avoids deviations caused by excessive amplitude.

[0040] In the preferred embodiment, step S22: process using the same method as in S13. That is, sampling time step Mapping order The purified temporal representation vector is obtained as follows: ; With a dimension of 10, the temporal evolution information of the impedance signal is preserved, and signal noise is reduced.

[0041] In the preferred embodiment, step S23: using the time-series trajectory analytical function through formula (11). Extracting evolutionary patterns; time-series trajectory analysis operators It includes multi-order difference features, and the formula is: ; in, ; It is a first-order difference, reflecting the rate of change of the signal; ; It is a second-order difference, reflecting the acceleration of signal changes; ; It is a third-order difference, reflecting the signal variation characteristics; It can capture the nonlinear evolution of electrical impedance signals and, compared with single-order difference features, can capture more information about abnormal metabolism in tumor tissue. Relationship matrix For 10×3 dimensions, with Using the multi-order difference features as input and the multi-order difference features as output, construct a system of linear equations and solve for the matrix that minimizes the fitting error. This is used to explore the correlation between electrical impedance and changes in lung tissue metabolism.

[0042] In the preferred scheme, step S24: using formulas (12) and (13), the multidimensional correlation integral method is used to calculate the evolution intensity of the signal; based on the normalized signal value range, the measurement radius is... Set to 0.5 to cover the local fluctuation range of the signal while avoiding excessive inclusion of irrelevant information; Step function Set as: when hour, ;when hour, Used to determine the evolutionary correlation between two sampling points; Related integral functions The calculation window is 200ms, reflecting the dynamic changes in signal intensity over time. During the calculation process, The number of sampling points within the window. The value ranges from 0 to 1. The larger the value, the stronger the correlation between the evolution of the signal within the window. Comprehensive correlation strength parameters The evolutionary complexity of the electrical impedance signal is calculated using formula (13), providing a basis for determining the benign or malignant nature of tumors.

[0043] In the preferred scheme, step S25 involves constructing a temporal evolution trajectory structure representation model and using continuous cohomology technology to capture the geometric topological features of the impedance temporal signal at multiple scales. First, nested simplex sequences are constructed on the temporal data point cloud. The construction of the complex is based on the distance threshold of the temporal data point cloud. The distance threshold is set to It gradually captures the multi-scale topological features of the signal, from local small fluctuations to global evolution trends, covering the entire topological information; Subsequently, the 0-dimensional, 1-dimensional, and 2-dimensional homology groups corresponding to each complex are calculated. ( ),in: The 0-dimensional homology group reflects the connected components of the signal and corresponds to the continuous evolution region of the impedance signal. One-dimensional homology groups reflect signal holes, corresponding to local troughs in impedance signals, which may be areas of active tumor metabolism. Two-dimensional homology groups reflect the three-dimensional topological structure of a signal, corresponding to the complex evolution mode of impedance signals; Record the birth point of each topological feature With vanishing point Build a persistent point set In a persistent point set, the persistence of topological features is defined as follows: Topological features with a persistence greater than 0.3 are considered valid features corresponding to stable electrophysiological abnormalities in tumor tissue, while those with a persistence less than 0.3 are considered noise interference and are therefore removed; this is used to filter random noise and retain key topological features. Finally, the difference between the current signal topology and the standard health model is quantized using the Wasserstein distance in formula (15), and the distance order is determined. Reference signal The mean of the duration plot of the electrical impedance time series signal based on statistics of healthy tissue. The value ranges from 0 to 3, indicating benign tissue. A value less than 0.8 indicates malignant tissue. A value greater than 1.5 indicates early mutations in the electrophysiological microenvironment of tumors.

[0044] In the preferred embodiment, step S26 involves fusing the comprehensive parameters from S24 using a nonlinear projection fusion operator. Topological persistence feature point set of S25 The deep eigenvector of electrical impedance is obtained through formula (16). Among them, the characteristic cascade operator To directly splice, the scalar and Dimensional splicing into dimensions vector; kernel mapping projection function Kernel principal component analysis was used, with radial basis functions selected as the kernel function and kernel parameters... , used to handle nonlinear mapping of heterogeneous features; The final output is a 1×4096 dimension deep feature vector of electrical impedance. It includes the evolutionary complexity and topological characteristics of electrical impedance signals, reflecting the electrophysiological properties of lung tissue.

[0045] In the preferred scheme, step S3: In the preferred embodiment, step S31: Based on the complete Transformer architecture... and Dynamic recombination is performed to achieve nonlinear fusion of features from two modalities. The Transformer architecture adopts a 2-layer encoder and 1-layer decoder structure. The encoder includes a multi-head attention mechanism and a feedforward neural network, while the decoder is used for fine-tuning after feature recombination. First, the input features are linearly transformed using formulas (17), (18), and (19) to obtain the Query, Key, and Value matrices, and the weight parameter matrix. , , The initial values ​​are based on a Xavier normal distribution with a mean of 0 and a standard deviation of 0. ;in For the input feature dimension, To optimize the output matrix dimension, avoid training instability caused by initial weights being too large or too small; Then, the cross-attention matrix is ​​calculated using formula (20) and the feature reorganization is completed, along with the feature dimension scaling factor. Set to the same dimensions as the Query and Key matrices. The calculation of the cross-attention matrix introduces a masking mechanism to mask low-contribution features and retain only high-contribution features for attention calculation, thereby improving computational efficiency and fusion accuracy. After normalizing the cross-attention matrix using the Softmax function, multiplying it with the Value matrix yields the recombined initial fused feature vector, which integrates the core information of multimodal collaborative features and deep electrical impedance features. The dimension is 1×2048; Through the Cross-Attention mechanism, it automatically learns when presenting... (Q) In the form of Which evolutionary patterns of (K,V) possess diagnostic discriminative power, enabling dynamic information supplementation between modalities, such as for early-stage tumors with blurred boundaries? Topological features will be assigned higher attention weights.

[0046] In the preferred scheme, step S32: Implemented after the cross-attention mechanism using formula (21). Normalization is performed to eliminate network biases in data from different modalities; among these, smoothing constraint values ​​are used. This is used to prevent the denominator from being zero; after normalization, The value range is [-1, 1]. The features are projected onto a uniform unit hypersphere to balance the amplitude differences between different modes and avoid weight allocation deviations caused by large fluctuations in impedance signal amplitude or the wide range of medical image feature values, compared to the initial fused feature vector. Its distribution is more uniform and its anti-interference ability is stronger.

[0047] In the preferred embodiment, step S4: In the preferred scheme, step S41: Build a correlation network framework based on the structural causal model and explore the transmission logic between variables; input the variable set through formula (22). ,in: Represents multimodal collaborative feature vectors Related unobserved tissue background factors, including pulmonary inflammation, fibrosis, and emphysema, affect the tissue density characterization in medical imaging, thereby interfering with the imaging process. Non-tumor lesions for feature extraction; Represents the deep eigenvectors of electrical impedance. Related unobserved background factors, including fluctuations in electrode contact resistance, interference from the subject's respiratory movements, and differences in skin stratum corneum thickness, can affect the accuracy of impedance signal acquisition, thereby interfering with the acquisition process. Physical fluctuations in feature extraction; Through formula (23), the core variable set ,in: Corresponding fused feature vector From China Feature information; correspond From China Feature information; Outcome variable For the diagnosis of tumor, Indicates a benign lesion. Indicates a malignant tumor; Using formulas (24) and (25), the causal mapping rule orientation is described as follows: and ,in: and All were fitted using a multilayer perceptron, where: It contains two hidden layers with ReLU activation function, used to characterize unobserved factors. For core variables The impact; It contains 3 hidden layers, with GELU as the activation function, used to characterize the core variables. and unobserved factors Regarding the diagnostic conclusion The combined impact; By explicit modeling This enables the model to identify and eliminate spurious correlation interference, thus avoiding misdiagnosis.

[0048] In the preferred embodiment, step S42: using formula (26), Through built-in adaptation mapping operator Transform into core variables and Built-in adaptive mapping operator It is implemented through a deep neural network, which contains two hidden layers and uses GELU as the activation function. This activation function can alleviate the gradient vanishing problem and improve the mapping accuracy while maintaining the advantages of ReLU. Mapping parameters The Bayesian optimization algorithm is used to optimize the mapping error. The search space for Bayesian optimization is: The number of iterations is set to 50 to enable... and reflect The key information in the mapping error control is used to complete the adaptation mapping between the fusion features and network variables.

[0049] In the preferred scheme, step S43: use do-calculus to perform causal correlation verification on the core variables to verify the robust causal relationship between the input features and the diagnostic conclusion; First, intervention baseline values ​​were established, and historical data without lung lesions were selected as the baseline sample for collection. and The statistical mean is used as the standard benchmark. , ; Subsequently, the correlation intervention was deduced, and the intervention was implemented in three layers, including: First layer, fixed ,shield The path, calculating the diagnostic predictive value after intervention. , through formula (27). ,in The distribution is estimated through variational inference. Unobserved factors The probability distribution; The second layer is fixed. ,shield The path is calculated similarly. ; The third layer, simultaneously fixed. and ,shield and Path, calculation ; By comparing actual observed values The deviation from the predicted values ​​after three levels of intervention verifies the robustness of causal association: like and Then determine the core variable. Diagnostic conclusion There is a robust causal relationship; If the deviation is less than 0.05, it is determined that there is spurious correlation interference, and the feature extraction process is re-optimized.

[0050] In the preferred embodiment, step S44: calculate the pure contribution of each feature component using the contribution expectation operator through formula (28). Quantify the true contribution of each feature to the diagnostic conclusion, including: Mathematical expectation Monte Carlo integration is used, with the number of samples set to 1000 to improve integration accuracy while balancing computational efficiency and avoiding computational delays caused by oversampling. Pure contribution Introducing causal effect correction factor Corrected contribution By eliminating statistically relevant but causally unrelated feature contributions, the contribution quantification becomes more aligned with clinical pathology logic. The selection threshold for the high-contribution feature set was set to 0.4. The features constitute a high-contribution feature set, and based on this set, a high-precision determination result of tumor benignity or malignancy is output. .

[0051] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A tumor identification method based on multimodal information collaboration, characterized in that, The method includes the following steps: S1. Construct a multimodal collaborative representation unit to simultaneously receive medical images and electrical impedance time-series data. Through a synchronous triggering mechanism, acquire tumor tissue structure slices and electrical impedance sequences at the same time. Map image pixel grids to the image representation domain and analyze the electrical impedance time-series signals. Perform a temporal-to-spatial domain mapping to obtain the temporal representation vector. Using multi-domain coupling correlation functions Integrating dual-domain information, feature extraction is transformed into a process of finding the physical steady state in an energy field composed of imagery and electrical impedance data. Intra-domain evolution convergence is used to solve for the stable representation state, and iterative calculations are performed to minimize the total system energy, thus obtaining the steady-state field. Using the spatial frequency domain operator decomposition method, from the steady-state field Extracting macro-level correlation features and micro-difference characteristics The covariance alignment weighted fusion method is used to fuse macroscopic correlation features. and microscopic differences Output multimodal collaborative feature vector ; S2. Construct a impedance timing analysis unit to process impedance timing data, purify the impedance timing data, and perform time-to-space domain mapping to obtain the purified timing representation vector. It employs time-series trajectory analytical operators that include first-order, second-order, and third-order difference features. extract The evolutionary pattern was observed, and the relation matrix was obtained. Furthermore, by analyzing the impedance signal as described above, deep correlation features are extracted, and a deep feature vector of impedance is output. ; S3. Design a dynamic feature adaptation and fusion unit, based on feature responsiveness adaptation technology, to process the multimodal collaborative feature vector. With electrical impedance deep eigenvectors Perform adaptive fusion and dimension restructuring to output a stable fusion feature vector. ; S4. Construct a feature association determination engine based on a structural causal model, and stabilize and fuse feature vectors. By adapting the mapping operator Transform into core variables, using - The algorithm performs causal intervention on the core variables, calculates the diagnostic prediction value after intervention, quantifies the feature contribution based on the diagnostic prediction value, and outputs the tumor diagnosis result.

2. The tumor identification method based on multimodal information collaboration according to claim 1, characterized in that, the steps are as follows: In S1, the spatial frequency domain operator decomposition method is used to extract the steady-state field. Extracting macro-level correlation features and microscopic differences ,include: Use low pass operator For steady-state fields Extracting macro-level correlation features ; Using high-pass residual operators For steady-state fields Capturing microscopic differences .

3. The tumor identification method based on multimodal information collaboration according to claim 1, characterized in that, Step S2 specifically includes the following steps: S21. Employ a multi-stage physical filtering algorithm to process the original impedance timing signal. Purification is performed, and a purification signal is obtained. ; S22, Regarding the purification signal Perform a temporal-to-spatial domain mapping to obtain the purified temporal representation vector. ; S23. Employing time-series trajectory analytical operators Extract the evolutionary patterns to obtain the relation matrix. .

4. The tumor identification method based on multimodal information collaboration according to claim 3, characterized in that, In step S21: The purification method is as follows: S211. Perform baseline drift correction to obtain the corrected signal. ; S212, Using the frequency response function The filtered signal is obtained by performing power frequency interference filtering. ; S213, Normalized and filtered signal Output purification signal .

5. A tumor identification method based on multimodal information collaboration according to claim 3, characterized in that, Step S2 also includes the following steps: S24. The evolution intensity of the signal is calculated using the multidimensional correlation integral method. First, the correlation integral function is obtained. Then through the correlation integral function The comprehensive correlation strength parameters characterizing the electrophysiological complexity of tissues were calculated. ; S25. Construct a temporal evolution trajectory structure representation model, and build nested simplex complexes on the temporal data point cloud. Topological features are captured by calculating the homology group, and the persistent point set of the current signal topology is calculated. Compared with the standard model Between distance; S26. Fuse the comprehensive correlation strength parameters using a nonlinear projection fusion operator. With topological persistent feature point set The deep eigenvectors of electrical impedance are obtained. .

6. The tumor identification method based on multimodal information collaboration according to claim 1, characterized in that, Step S3 specifically includes the following steps: S31. Multimodal collaborative feature vectors based on Transformer architecture With electrical impedance deep eigenvectors Perform dynamic reorganization, according to , , Perform a linear transformation on the input features, where, , , These are the query matrix, key matrix, and value matrix, respectively. , , The weight parameter matrix is ​​then used to perform feature reorganization using a cross-attention mechanism, resulting in an initial fused feature vector. ; S32. Initial fusion feature vector conduct Normalization process yields a stable fused feature vector. .

7. The tumor identification method based on multimodal information collaboration according to claim 1, characterized in that, Step S4 Specifically, the following steps are included: S41. Construct a relational network framework based on a structural causal model; First, define a set of input variables that includes unobserved background factors. ; Next, define components that include cooperative elements. With impedance components core variable set ; Finally, the causal mapping rules are described in a directional manner; S42. Using the Adaptive Mapping Operator Stabilize the fused feature vectors Transform into core variable nodes .

8. A tumor identification method based on multimodal information collaboration according to claim 7, characterized in that, Step S4 also includes the following steps: S43. Use do-calculus to perform multi-level causal association verification on core variables and calculate the diagnostic prediction value after intervention. ,in, Core variables for healthy organizations The statistical mean was used to verify the robustness of the causal relationship between input features and diagnostic conclusions; S44. Calculate the pure contribution of each characteristic component using the contribution expectation operator. The final tumor diagnosis result is output based on the high-contribution feature set. .

Images