Tumor identification method based on multi-modal information coding

Abstract

The invention provides a tumor identification method based on multi-modal information coding, and aims to solve the problems of lack of space-time alignment, insufficient time sequence feature mining and weak semantic relevance in the prior art. The method comprises the following steps: firstly, synchronously collecting structure image data and electrical impedance time sequence data of a to-be-detected tissue, and realizing time-space synchronization of bimodal data through timestamp alignment and anatomical marker mapping; a dual-path adaptive enhancement path is constructed, targeted preprocessing is performed on the two types of data, and fusion weights are distributed based on data quality scores; then extracting multi-scale spatial features and multi-directional time sequence cyclic coding features through a dual-path modal adaptation coding architecture; and finally, integrating bimodal features based on a cross-modal attention fusion mechanism, and sequentially completing tumor existence judgment, benign and malignant qualitative analysis and malignant degree grading by means of a multi-stage progressive classifier. According to the method, the structural form and the electrophysiological metabolism information are deeply fused, the accuracy and robustness of tumor recognition are improved, and the clinical rapid screening and precise diagnosis and treatment requirements are met.

Description

Technical Field

[0001] This invention relates to the field of tumor identification technology, and in particular to a tumor identification method based on multimodal information encoding. Background Technology

[0002] The challenge in tumor identification lies in accurately capturing the morphology and physiological characteristics of lesions and effectively distinguishing between normal tissue and diseased areas. However, existing identification systems, due to limitations in information dimensions and a lack of fusion logic, struggle to achieve high-precision identification and fine-grained grading in clinical diagnosis.

[0003] Currently, tumor identification technology mainly relies on structural imaging examinations or electrophysiological index monitoring. Both of these single-modal methods have significant information loss problems: structural imaging modality captures visual information by extracting the spatial distribution and boundary features of lesions, but often cannot correlate with the biochemical metabolic characteristics of tissues; while functional modality-based methods such as electrical impedance can reflect abnormalities in tissue metabolic activity, their spatial resolution is low and they are sensitive to physiological noise, lacking spatial structural support and positioning accuracy for lesions.

[0004] Among existing multimodal fusion technologies, patent CN101984462A discloses a method for synthesizing functional electrical impedance tomography images and anatomical images. This method acquires electrical impedance imaging data and anatomical images, and uses coordinate mapping to achieve visual fusion and display of the two. However, in practical applications, this type of method still suffers from defects such as lack of spatiotemporal alignment calibration, insufficient mining of temporal features, and weak semantic correlation. It fails to fully utilize multi-directional encoding logic such as forward, reverse, interval, and segmentation to characterize the tumor-specific metabolic evolution and biomembrane capacitance characteristics. It also fails to introduce attention mechanisms and adaptive enhancement weights, resulting in key features being easily submerged by redundant noise; thus, it is difficult to meet the precise traceability requirements of clinical diagnosis.

[0005] Therefore, there is an urgent need to design a tumor identification method based on multimodal information encoding, which can deeply extract multi-dimensional temporal coding features on the basis of achieving high-precision spatiotemporal synchronization, and combine progressive classification logic to solve the problems existing in the above-mentioned technologies. Summary of the Invention

[0006] The main objective of this invention is to provide a tumor identification method based on multimodal information encoding, which solves the problems of missing spatiotemporal alignment calibration, insufficient temporal feature mining, and weak semantic correlation in tumor identification technology.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a tumor identification method based on multimodal information encoding, the method comprising the following steps: S1. Obtain structural imaging data using standard medical imaging equipment, and simultaneously acquire electrical impedance time-series data using a multi-electrode array system. Achieve spatiotemporal synchronization of the two modal data through timestamp alignment. S2. Construct a dual-path adaptive feature enhancement and filtering module to perform targeted preprocessing on structural image data and electrical impedance data respectively; S3. Design a dual-path modality adaptation coding architecture to extract deep features from the preprocessed dual-modality data. S4. Based on the cross-modal attention fusion mechanism, dual-path coding features are integrated, and a multi-level progressive classifier is used to achieve accurate tumor identification and grading.

[0008] In the preferred embodiment, step S1 specifically includes the following steps: A TTL level signal is sent to the impedance acquisition host computer at the start of image scanning via a hardware synchronization trigger and recorded as a reference timestamp. Each frame of impedance data sampling points is accompanied by a time offset. Corresponding to absolute time To achieve time alignment; Spatial synchronization was achieved by labeling anatomical markers that appeared under structural imaging on the subject's tissue.

[0009] In the preferred embodiment, step S2 specifically includes the following steps: S21. Construct an adaptive feature enhancement path for structural images and dynamically adjust the enhancement strategy based on local block features; S22. Construct a path for electrical impedance data filtering and feature extraction to eliminate noise interference and extract spatial electromagnetic features; S23. Dynamically allocate feature fusion weights based on data quality scores.

[0010] In the preferred embodiment, step S21 specifically includes the following steps: S211, Local Block Feature Quantization Original image Divided into Non-overlapping blocks of pixels For each block, calculate its contrast metric, noise level, and edge strength, where: Contrast measurement Calculated using the following formula: (1); in, These are the maximum and minimum values ​​of the pixel grayscale values ​​within the current block; It is a very small constant used to avoid numerical overflow caused by a denominator of zero; Reflects the distinguishability between soft tissues within a block; noise level Calculated using the following formula: (2); in, The gradient matrix of the block, i.e., the pixel value in First-order difference in direction; Take the median of the gradient Exclude interference from tissue edges and estimate the intensity of background Gaussian noise; Edge strength Calculated using the following formula: (3); in, This represents the total number of pixels in the block. Quantify the sharpness of lesion boundaries within a block.

[0011] S212, Dynamic Weight Allocation A dynamic weight allocation mechanism is designed based on the core feature indicators of the block. This mechanism limits the weights of the contrast-adaptive histogram equalization algorithm, the bilateral filtering denoising algorithm, the guided filtering sharpening algorithm, and the normalized weights of the enhancement algorithm through the output of the activation function. The weights of the contrast-limited adaptive histogram equalization algorithm are calculated using the following formula: (4); The weights of the bilateral filtering denoising algorithm are calculated using the following formula: (5); The weights of the guided filter sharpening algorithm are calculated using the following formula: (6); in, These are learnable weight parameters; For the corresponding bias term, for Activation function; And the above three types of weights satisfy .

[0012] S213, Weighted Fusion Weighted fusion data augmentation based on dynamic weights avoids over- or under-enhancement, and extracts weights from the contrast adaptive histogram equalization algorithm. bilateral filtering denoising algorithm weights Guided filtering sharpening algorithm weights The blocks are processed, and then stitched back into the original image according to their original coordinates to obtain a complete image with enhanced contrast and reduced noise. .

[0013] In the preferred embodiment, step S22 specifically includes the following steps: S221, Smoothing Processing The impedance time series data is preprocessed using a sliding window mean filter, with the following formula: (7); in, The window step size; Filter out spike noise caused by electrode contact resistance fluctuations and eliminate high-frequency noise interference.

[0014] S222, Mesh conductivity conversion Map the impedance acquisition area to A virtual grid; based on boundary voltage measurements. and input current The conductivity distribution is solved using a modified Newton-Raphson iterative algorithm, and the formula is: (8); in, The sensitivity matrix represents the partial derivatives of the boundary voltage with respect to changes in local conductivity. This is the regularization matrix; To adjust the parameters; Each grid cell represents the impedance characteristics of a specific anatomical region.

[0015] In the preferred embodiment, step S23 specifically includes the following steps: S231, Calculate structural image quality score The image quality score is calculated using a formula based on image contrast and noise level: (9); in, , These are the global maximum and minimum pixel intensity values ​​of the complete structure image, respectively; Based on step S211 The global noise standard deviation is calculated. This represents the maximum permissible noise standard deviation determined based on the model of the medical imaging equipment. Assess the quality of structural images.

[0016] S232, Calculation of electrical impedance data quality score Based on signal stability and the percentage of effective data, the impedance data quality score is calculated using the following formula: (10); in, The standard deviation of the smoothed impedance signal in step S221; The standard deviation of the signal is determined based on the impedance range of normal tissue. To satisfy the signal amplitude within Number of data points within the range; This represents the total number of impedance data points. Evaluate the quality of electrical impedance data.

[0017] In the preferred embodiment, step S3 specifically includes the following steps: S31. Perform multi-scale feature extraction on enhanced image data; S32. Perform multi-directional timing cyclic encoding on the impedance timing data.

[0018] In the preferred embodiment, S31 specifically includes the following steps: S311, Multi-receptive field spatial feature extraction By using parallel convolutional neural network layers, the visual logic of observing tumors from a local to a global perspective is simulated, with preprocessed images as input. It employs three sets of parallel convolutional kernels for processing, wherein: Fine-grained nuclei With a step size of 1, a fill size of 1, and 64 output channels, it is used to extract details such as burrs and microcalcifications at the edge of the tumor. Medium nucleus With a step size of 2, a fill size of 2, and 128 output channels, it is used to capture local morphological features such as the lobulation of tumors and the compression deformation of surrounding tissues. coarse-grained nuclei With a step size of 2, a padding size of 3, and 256 output channels, it is used to perceive the overall spatial relationship between the tumor and the background anatomical structure.

[0019] S312, Multi-scale Feature Aggregation and Attention Weighting Because lesion sizes vary in different cases, the weights of nuclei at different scales need to be adaptively adjusted. The calculation formula is as follows: (11); in, These are the weight coefficients learned through global average pooling; For the extracted feature map at a specific scale; (12); (13); in, The weights in the convolution kernel represent the sensitivity to a specific shape. In the image The grayscale values ​​of pixels in the location neighborhood; For bias terms; It is a linear rectification activation function; If the tumor has clear boundaries, the system automatically increases the weight of fine-grained analysis; if the tumor is large, the weight of coarse-grained analysis is increased.

[0020] S313, Feature Dimension Reduction and Standardized Output Spatial dimension is reduced by a global average pooling layer. Compress to 1, then execute Normalization, the formula is: (14); in, This is the original feature vector after aggregation; Let L be the L2 norm of the vector; To ensure a stable minimum value; This makes the image features comparable under different lighting or contrast conditions.

[0021] In the preferred embodiment, S32 specifically includes the following steps: S321, Four-directional temporal feature extraction Four independent time-series processing units—forward, reverse, interval, and segmented—are constructed to capture multi-dimensional temporal dependencies in the data, wherein: Forward coding in chronological order Input data to record the dynamic equilibrium process of impedance gradual polarization caused by electrolyte ion migration under continuous excitation current in the tumor; Reverse encoding in chronological order Reverse input, due to the memory effect of bioimpedance, reverse encoding can capture the relaxation characteristics at the end of the signal, which can be used to determine the water content inside the tumor; Interval encoding extracts subsequences according to a set interval step size. Because ordinary coding is easily affected by impedance pulsation caused by breathing and heartbeat, low-pass filtering is achieved in the time domain by using interval coding with a step size to directly filter out physiological noise and focus only on the slow conductivity drift caused by metabolism in tumor tissue. Segmented coding divides the complete time sequence into three physiological phases: the initial response phase, the plateau stabilization phase, and the decay and release phase. Because the cell membrane capacitance of malignant tumors is large, the impedance jump slope of malignant tissue is significantly different from that of benign tissue in the initial response phase. Segmented coding explicitly extracts such phase features through local attention, thus solving the gradient vanishing problem caused by long sequences.

[0022] S322, Multi-directional Feature Weight Fusion First, the importance score of the multi-directional encoded features at each time step is calculated based on their contribution to tumor identification. The formula is: (15); in, These are learnable attention parameters; These are the hidden state vectors extracted from the four directions; Subsequently, the importance scores of all time steps were normalized to obtain the weight coefficients for each time step. The formula is: (16); Finally, the multi-directional coding features at each time step are weighted and fused according to the weight coefficients, as shown in the formula: (17); Obtain the integrated feature vector that integrates key time-series information. .

[0023] S323, Dimensional Unification and Feature Representation Enhancement The feature vector is transformed by a two-layer fully connected neural network (MLP) and mapped to a fixed-dimensional feature vector. More discriminative features are then selected to obtain the final feature vector after electrical impedance encoding. The formula is: (18); in, This is the weight matrix of the fully connected layer; This is the bias term vector; For layer normalization, the formula is: (19); This ensures that impedance data collected in different batches have a consistent distribution in the feature space, reducing the impact of data fluctuations on the subsequent fusion process.

[0024] In the preferred embodiment, step S4 specifically includes the following steps: S41, Adaptive weighted cross-modal attention fusion; S42. Construct a three-level progressive classifier.

[0025] In the preferred embodiment, step S41 specifically includes the following steps: S411, Dimensional Alignment and Projection The feature transformation layer adjusts the feature vectors of the two modalities to the same dimension, ensuring the feasibility of subsequent fusion. For structural image coding features, their feature dimensions are mapped to the target dimension; For the electrical impedance coding features, their feature dimensions are also mapped to the target dimension; We obtain feature vectors with two identical dimensions.

[0026] S412, distribution is relatively accurate To address the issue of overly strong image features masking impedance characteristics, the overall distribution statistics of each feature vector are calculated, and each element in the feature vector is normalized and adjusted using the following formula: (20); (twenty one); in, These are the mean and standard deviation of the eigenvectors, respectively. By placing the two sets of features into the unit hypersphere space, the influence of different dimensions on the attention score is eliminated.

[0027] S413 Attention Weighting Scoring and Calculation The importance of the two standardized modal eigenvectors is evaluated separately, using the following formula: (twenty two); Where Q represents the spatially corresponding metabolic anomaly in the image features; K represents the metabolic distribution index of the impedance features; and V represents the impedance feature value. To define the feature dimension, prevent the dot product result from becoming too large and causing gradient saturation; Calculate the correlation between each morphological feature point in the image and each group of metabolic features in the impedance sequence; The two scores were then normalized to obtain the attention weights for each modality feature. The sum of the two weights is 1, which makes the weight allocation reasonable.

[0028] S414, Fusion Feature Optimization and Residual Enhancement The structural image weights calculated based on S413 With electrical impedance characteristic weight The standardized eigenvectors of the two modes are linearly weighted and summed, as shown in the formula: (twenty three); Initial fusion feature vectors of preliminary coupled morphological and electrophysiological information were obtained. ; Subsequently, a two-stage feature optimization process is used to improve the discriminative ability of the fused features. The first stage of optimization uses a fully connected projection layer to... Mapped to a higher intermediate-dimensional space and with the introduction of nonlinear excitation, the formula is: (twenty four); in, To optimize the weight matrix for the first stage, the features are mapped to a higher-dimensional manifold space through linear projection, making it easier to find the boundary between tumor tissue and healthy tissue in this space. This is a first-level bias term; Obtain the intermediate dimension fusion vector ; The second-level optimization compresses the feature dimension to the final classifier target dimension and eliminates numerical drift during deep network propagation through calibration operators. The formula is as follows: (25); in, The second-level optimization of the weight matrix compresses redundant intermediate features, retaining only the discriminative information that determines the presence, nature, and grade of tumors. This is a second-level bias term; The layer normalization operator standardizes the vector using the following formula: (26); The distribution is calibrated and stabilized to ensure effective output even when dealing with raw data exhibiting baseline differences. This enhances the robustness of the model in clinical, uncompared tissue testing.

[0029] In the preferred embodiment, step S42 specifically includes the following steps: S421. Primary Detection Classification: Determination of Tumor Presence The S414 output fused feature vector Input the first-level detection classifier; The classifier uses a fully connected linear layer combined with the Softmax normalization operator at its bottom layer, as shown in the formula: (27); in, This is a first-level detection weight matrix used to extract significant indicators related to tissue occupancy effect from the fused features; This is a first-level detection bias term used to correct detection offset caused by background noise; This represents the probability result of being determined to be tumor-free; This represents the probability of having a tumor. ; like If the system detects a suspected tumor tissue in the detection area, it will proceed to the next step. like If the tissue is normal, the system will output a conclusion that no tumor was found. in This is the preset threshold.

[0030] S422. Secondary Classification of Tumors: Qualitative Determination of Benign and Malignant Forms For suspected lesions detected by the primary classifier, further differentiation of their pathological nature is performed, and lesions with similar fusion characteristics are grouped together. Mapping to a higher-dimensional property discrimination space, the formula is: (28); in, This is a second-order property weight matrix, reflecting... The image edge and impedance phase angle features are more sensitive; This is a second-order property bias term; The probability of it being determined to be a benign tumor; The probability of being diagnosed as a malignant tumor; The probability of being classified as a borderline tumor; The sum of the three probability results is 1, and the category corresponding to the maximum probability is taken as the conclusion; If The probability is highest, leading to the conclusion of a benign tumor. If the situation is otherwise, proceed to the next step.

[0031] S423. Assessment of Tumor Malignancy in Three-Tier Classification Extracting deeper representations of metabolic rate, cell density, and angiogenesis from the fusion features, and inputting them into a four-class probabilistic model, the model formula is as follows: (29); in, It is a three-level weight matrix used to characterize the small shifts of the feature vectors in the manifold space as the degree of malignancy increases; This is a three-level hierarchical bias term; These correspond to the probability results of four different levels, from low malignancy G1 to extremely high malignancy G4. The output result with the highest probability value is taken as the final tumor malignancy grade conclusion, thus achieving prediction of the entire tumor life cycle.

[0032] This invention provides a tumor identification method based on multimodal information encoding. Through timestamp alignment and coordinate mapping based on anatomical reference points, it ensures spatial synchronization between static anatomical images and dynamic metabolic impedance time-series data at a physical level, resolving feature bias caused by cross-modal data mismatch. Utilizing dual-path adaptive feature enhancement, through block-based index quantization and dynamic weight allocation, it filters out background noise while preserving detailed tumor edge information. The introduced forward, reverse, interval, and segmented multi-directional temporal encoding, particularly through interval sampling, effectively captures long-range metabolic fluctuations across cycles and filters out physiological noise such as respiration and heartbeat, achieving high identification accuracy without the need for self-controlled comparisons and enhancing the system's robustness in blind clinical testing scenarios. Employing a two-level feature dimension optimization and distribution calibration stabilization mechanism, it solves the feature overload problem caused by cross-modal data dimensional redundancy and dimensional inconsistencies, strengthening the semantic coupling of morphological and electrophysiological features. It achieves interpretability and refined evaluation of auxiliary diagnostic logic, meeting the needs of rapid clinical screening and precise diagnosis. Attached Figure Description The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of a tumor identification method based on multimodal information encoding according to the present invention; Detailed Implementation This invention provides a tumor identification method based on multimodal information encoding. This method solves the problems of missing multimodal feature calibration, insufficient semantic association, and poor clinical interpretability in the prior art by deeply integrating high spatial resolution structural imaging data with high sensitivity electrophysiological metabolic time series data.

[0033] Example 1 This embodiment mainly implements the core steps described in the independent claims, aiming to provide a tumor screening solution with rapid response capabilities that can run on basic edge computing devices.

[0034] like Figure 1 As shown, a tumor identification method based on multimodal information encoding includes the following steps: In step S1, universally applicable technologies such as ultrasound are selected as structural imaging acquisition devices, and multi-electrode bioelectrical impedance sensors are used in conjunction with them. While the subject is undergoing image scanning, the sensor records local impedance changes at a sampling frequency of 50Hz. When the image is captured, the starting pointer of the impedance data stream is recorded simultaneously to achieve preliminary time alignment.

[0035] In step S2, global grayscale stretching and Gaussian denoising are performed on the structural image to improve the signal-to-noise ratio of the image. For impedance time series data, power frequency interference during the acquisition process is removed by moving average filtering. Subsequently, global features such as the mean and standard deviation of the image and the basic electrical parameters of the impedance sequence are calculated to provide preprocessed data support for subsequent encoding.

[0036] In step S3, the basic coding architecture is constructed; For structural images, the system uses a three-layer convolutional neural network to extract single-scale spatial feature maps; For impedance data, the system uses a standard unidirectional long short-term memory network to compress and encode the timing signal; The two sets of features are mapped to the same 128-dimensional space through a fully connected layer, forming a modality-adapted feature vector.

[0037] In step S4, basic weighted fusion is performed; The two sets of feature vectors are linearly added using preset fixed weight coefficients; the fused features are then input into a Softmax classification layer, which outputs the probability of tumor presence. When the probability exceeds the preset value, the system outputs a positive alarm.

[0038] Example 2 This implementation adds various optimization mechanisms described in the invention to the basis of Example 1, aiming to achieve precise clinical auxiliary diagnosis and grading of soft tissue tumors such as breast tumors; its hardware includes a Siemens Magnetom Prisma 3.0T MRI system and a 32-electrode high-precision bioelectrical impedance analyzer, and the main control platform adopts a computing workstation equipped with an NVIDIA RTX4090 GPU to support high-dimensional attention mechanism operations.

[0039] In the preferred embodiment, in step S1: A 50ms TTL signal is sent to the impedance acquisition host computer at the instant the MRI pulse sequence starts using a hardware synchronization trigger. After receiving the signal, the host computer records the reference timestamp. And based on the offset generated by the high-precision crystal oscillator Calculate absolute time This enables millisecond-level spatiotemporal alignment. Meanwhile, four imaging markers were attached to the subject's skin surface, which appeared as bright isolated points in MRI images; Operators can automatically lock the pixel coordinates of these markers through the system interface and record the coordinate position of the electrode array in physical space at this time, which serves as a physical reference for subsequent spatial mapping.

[0040] In the preferred embodiment, in step S2: In step S21: First, image processing involves dividing the original 256×256 pixel image into 32×32 pixel non-overlapping blocks, and then calculating the contrast metric, noise level, and edge strength of each block in parallel. The contrast measure is calculated using formula (1). The constant term in this formula The aim is to prevent numerical calculation anomalies caused by completely black background areas and to reflect the distinguishability of local tissues; The noise level is calculated using formula (2). The block gradient matrix for The first-order difference in three directions is used to extract the median gradient to exclude interference from tissue edges and estimate the intensity of background Gaussian noise. The gradient calculation is implemented using the Sobel operator. Calculate edge strength using formula (3) Quantify the spiculation characteristics of tumor margins; Subsequently, the dynamic weight allocation stage begins. Based on the core feature indicators C, N, and E of the block, a dynamic weight allocation mechanism is designed. The weights of the contrast-adaptive histogram equalization algorithm are limited by the output of the sigmoid activation function. bilateral filtering denoising algorithm weights , weights of the guided filter sharpening algorithm The three weights satisfy ; Through training with 5000 labeled breast MRI image samples, the learnable weight parameters in formulas (4), (5), and (6) were set as follows: It can accurately reflect the weights of different feature indicators on the algorithm's performance, where: Since noise has the most significant impact on the selection of denoising algorithms, the noise level N has the highest weight in the bilateral filtering denoising algorithm. Since edge sharpness directly determines the necessity and intensity of sharpening, the edge intensity E has the highest weight in the guided filter sharpening algorithm; The sigmoid activation function restricts the weight output to the [0,1] interval, making the weight distribution reasonable; Finally, the weighted fusion stage is entered. Based on the dynamically allocated weights, CLAHE enhancement, bilateral filtering denoising, and guided filtering sharpening are performed on each block respectively. The clipLimit of CLAHE is set to 2.0, and the grid size is 8×8. The spatial standard deviation of bilateral filtering is set to 5, and the standard deviation of grayscale value is set to 10. The radius of guided filtering is set to 3, and the regularization parameter is set to 0.01. After processing, all blocks are stitched back to the original image according to their original coordinates, resulting in a complete image with enhanced contrast and reduced noise. While preserving key details such as tumor edge burrs and microcalcifications, it also filters out equipment noise and tissue background noise.

[0041] In step S22: First, smoothing is performed. A sliding window mean filter is used to preprocess the impedance time series data. The window step size in formula (7) is... Set to 5, this window size is based on the human respiratory cycle and heartbeat cycle, which can effectively filter out high-frequency physiological noise caused by breathing and heartbeat, as well as spike noise caused by electrode contact resistance jitter, while retaining low-frequency impedance changes related to tumor tissue metabolism. Subsequently, grid conductivity conversion was performed, mapping the impedance acquisition area to a 4×4 virtual grid. Each grid cell corresponds to a 2.5cm×2.5cm anatomical region, ensuring that the grid size matches the spatial resolution of the MRI images; based on boundary voltage measurements... and input current The conductivity distribution is solved using the modified Newton-Raphson iterative algorithm, and the regularization matrix in formula (8) is used. Set as identity matrix, Setting it to 0.01 ensures the accuracy of conductivity calculation while avoiding numerical oscillations during the iteration process, so that each grid cell accurately represents the impedance characteristics of the corresponding anatomical region.

[0042] In step S23: First, the structural image quality score is calculated using formula (9). , Based on step S211 The global noise standard deviation is calculated. The maximum permissible noise standard deviation is determined based on the technical parameters of the Siemens Magnetom Prisma 3.0T MRI system, with a contrast ratio of 0.6 and a noise suppression ratio of 0.4. Subsequently, the impedance data quality score was calculated using formula (10). ,in The standard deviation of the smoothed impedance signal in step S221. The signal standard deviation is determined based on the impedance range of normal breast tissue. To satisfy the signal amplitude within Number of data points within the range The total number of impedance data points is calculated with 0.5 for signal stability and 0.5 for valid data, ensuring the reliability and validity of the impedance data. Ultimately, the feature fusion weights are determined based on... and The normalization result is determined, that is This allows modalities with higher data quality to receive higher weights in subsequent fusion, thereby improving the discriminative power of the fused features.

[0043] In the preferred embodiment, in step S3: In step S31: First, multi-receptive-field spatial feature extraction is performed, and the preprocessed enhanced image is input. It employs three sets of parallel 3D convolutional kernels for processing, wherein: The fine-grained kernel K1 (3×3×3) has a stride of 1, a padding of 1, and 64 output channels. This convolutional kernel is small in size and is used to extract fine-grained details such as burrs and microcalcifications at the edge of the tumor. With a medium K2 (5×5×5), a stride of 2, a padding of 2, and 128 output channels, this convolution kernel is of moderate size and is used to capture local morphological features of tumors such as lobulation, pleural traction, and compression deformation of surrounding tissues. The coarse-grained kernel K3 (7×7×7) has a stride of 2, a padding of 3, and 256 output channels. This convolutional kernel is relatively large and is used to perceive the overall spatial relationship between the tumor and background anatomical structures such as the chest wall and nipple. The outputs of the three sets of convolutional kernels are as follows: ; Subsequently, the multi-scale feature aggregation and attention weighting stage begins. Due to the significant differences in tumor size among different cases, the weights of kernels at different scales need to be adaptively adjusted. The weight coefficients are calculated using formulas (11) and (12). The global average pooling operation in formula (12) compresses each feature map into a 1-dimensional vector, and the weight coefficients are obtained through the exp function and normalization. ,satisfy ; For tumors with well-defined borders and small diameters (≤3cm), increase the fine-grained nuclear weight. Highlighting fine-grained details; For large (diameter ≥7cm) and irregularly shaped tumors, increase the coarse-grained kernel weight. Highlighting the overall spatial relationship characteristics; For medium-sized tumors (3-7cm), It has the highest weight, balancing the relationship between local form and the whole; The feature map is obtained by formula (13). The ReLU activation function is used to introduce nonlinearity and enhance the model's feature representation ability. Finally, the feature dimension reduction and output standardization stage is entered. The spatial dimension of the three aggregated feature maps is compressed to 1 by a global average pooling layer to obtain the original feature vector v. Then, L2 normalization is performed by formula (14) to make the image features under different illumination and scanning parameters comparable, and finally the structured image coding features are obtained. .

[0044] In step S32: First, four-way temporal feature extraction is performed. Based on the gated recurrent unit (GRU), four independent temporal processing units are constructed: forward, reverse, interval, and segmented. Forward coding in chronological order Input data to record the dynamic equilibrium process of impedance polarization caused by electrolyte ion migration under continuous excitation current, reflecting the metabolic activity of tumor tissue; Reverse encoding in chronological order By inputting data in reverse order, the memory effect of bioimpedance is used to capture the relaxation characteristics at the end of the signal, which are related to the water content inside the tumor. Interval encoding is based on a set interval step size. Extracting subsequences Sampling points are selected to form subsequences. Low-pass filtering is achieved in the time domain through interval coding with a step size to filter out physiological noise such as breathing and heartbeat, and only focus on the slow conductivity drift caused by the metabolism of tumor tissue. Segmented encoding divides the complete time sequence into three physiological phases: the initial response phase, the plateau stabilization phase, and the decay release phase. Features of each phase are explicitly extracted through a local attention mechanism. Since the cell membrane capacitance of malignant tumor cells is large, the absolute value of the jump slope is significantly greater than that of benign tumors. Therefore, the impedance jump slope of the initial response phase can be effectively used to distinguish the key features of benign and malignant tumors, while solving the gradient vanishing problem caused by long sequences. Subsequently, the multi-directional feature weight fusion stage begins, where the importance score of the multi-directional encoded features at each time step is calculated using formula (15). The parameters are obtained through training with a large amount of labeled impedance time series data; then the importance scores of all time steps are normalized by the Softmax function of formula (16) to obtain the weight coefficients of each time step. The sum of the weight coefficients is 1; the multi-directional coding features of each time step are weighted and fused using formula (17) to obtain a comprehensive feature vector integrating key time-series information. ; Finally, we move on to the dimensionality unification and feature representation enhancement stage, which involves transformation using a two-layer fully connected neural network. The weight matrix of the first fully connected layer... The dimension is 256×448, with bias terms. The dimension is 448; the weight matrix of the second fully connected layer. The dimension is 448×448, with bias terms. The dimension is 448; the formula (18) is used to... The impedance data is mapped to a fixed-dimensional feature vector, where the LayerNorm layer performs normalization using formula (19) to ensure that the impedance data collected in different batches have a consistent distribution in the feature space, reducing the impact of data fluctuations on the subsequent fusion process, and finally obtaining the final feature vector after impedance encoding. .

[0045] In the preferred embodiment, in step S4: In step S41: First, the structural image is encoded with features through a feature transformation layer. and electrical impedance coding features All are mapped to a target dimension of 256. Subsequently, each element in the feature vector is normalized and adjusted using formulas (20) and (21). After normalization, the two sets of features enter the unit hypersphere space, eliminating the influence of different dimensions on the attention score, so that morphological features and electrophysiological features have equal weight competition opportunities when fused. Next, the importance of the two standardized modal eigenvectors is evaluated using formula (22), where... This refers to the spatially corresponding metabolic abnormality region features in the image features, i.e., the sub-feature vectors related to tumor metabolism. This is the metabolic distribution index of impedance characteristics, i.e., the global feature vector. The impedance eigenvalues ​​are used to calculate the correlation between each morphological feature point in the image and each group of metabolic features in the impedance sequence, thus obtaining the attention weights for each of the two modalities. and ; Finally, based on attention weights and By using formula (23) to linearly weight and sum the standardized feature vectors of the two modalities, the initial fusion feature vector of the preliminary coupling morphological and electrophysiological information is obtained. ; The first-level optimization uses a fully connected projection layer to... Mapped to a 512-dimensional intermediate space, and with the introduction of ReLU nonlinear excitation, the intermediate-dimensional fusion vector is obtained through formula (24). This involves mapping features to a higher-dimensional manifold space to find the boundary between tumor tissue and healthy tissue. The second-level optimization compresses the feature dimension to 256 dimensions through a fully connected layer, and eliminates numerical drift during deep network propagation through a LayerNorm layer, resulting in the final fused feature vector. ; By enhancing discriminative features through two-stage optimization, redundant noise was suppressed, and the robustness of the model in clinical non-contrast tissue detection was improved.

[0046] In step S42: The primary detection category is the tumor presence determination stage, which will... The input is a first-level detection classifier, which uses a linear fully connected layer with a Softmax normalization operator at the bottom layer. The tumor-free probability is calculated using formula (27). and the probability of having a tumor Preset threshold ROC curve analysis based on clinical samples confirmed that it can reduce the missed diagnosis rate while controlling the misdiagnosis rate within a clinically acceptable range; if If the system detects a suspected tumor tissue in the area, it will classify it as a secondary tumor; if If the result is negative, the system will determine that it is normal tissue, output the conclusion that no tumor was found, and generate a preliminary report including image quality score, impedance data quality score, and key feature weight distribution. The secondary classification is the stage for determining the benign or malignant nature of tumors. For suspected lesions detected by the primary classifier, [the classification will be performed on these lesions]. Input the secondary property classifier and calculate the probability of benign tumor using formula (28). Probability of malignant tumors Probability of borderline tumors The weight matrix of the classifier Trained using benign and malignant labeled samples, it is more sensitive to image edge features and impedance phase angle features; the conclusion is drawn based on the category corresponding to the maximum probability: if... The highest probability indicates a benign tumor; if... The highest probability indicates a preliminary diagnosis of malignancy; if... The highest probability result is a borderline tumor diagnosis; the latter two cases are classified into a three-tiered classification.

[0047] The three-tier classification is the stage for assessing the malignancy of tumors, and extracts... The deep characterization of metabolic rate, cell density, and angiogenesis is input into a three-level classification classifier, and the probabilities of low malignancy G1, low-to-medium malignancy G2, medium-to-high malignancy G3, and extremely high malignancy G4 are calculated using formula (29). The weight matrix of the classifier It can characterize the small shifts of feature vectors in manifold space as the degree of malignancy increases; it outputs the grade with the highest probability value as the final conclusion of tumor malignancy classification, realizes the prediction of the entire life cycle of tumor, and generates a detailed classification report.

[0048] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A tumor identification method based on multimodal information encoding, characterized in that, Includes the following steps: S1. Synchronously acquire structural image data and electrical impedance time-series data of the tissue under test, and achieve spatiotemporal synchronization of dual-modal data through timestamp alignment and spatial marker mapping; S2. Construct a dual-path adaptive enhancement path, perform block-based dynamic enhancement processing on the structural image data, perform filtering and conductivity conversion on the electrical impedance time series data, and assign fusion weights based on the data quality score. S3. Design a dual-path modal adaptation coding architecture to extract multi-scale spatial features from the enhanced image data, perform multi-directional temporal cyclic coding on the preprocessed impedance time-series data, and output modal adaptation features. S4. Based on the cross-modal attention fusion mechanism, dual-path coding features are integrated, and tumor presence determination, benign or malignant characterization, and malignancy assessment are performed sequentially through a multi-level progressive classifier.

2. The tumor identification method based on multimodal information encoding according to claim 1, characterized in that, Step S1 specifically includes: A base timestamp is recorded using a hardware synchronization signal. And combined with the time offset of each frame sampling point Calculate absolute time Achieve time alignment; Spatial synchronization is achieved by marking anatomical markers on the subject's tissues and locking the correspondence between the markers' coordinates in the image and their physical space.

3. The tumor identification method based on multimodal information encoding according to claim 1, characterized in that, The processing of structural image data in step S2 includes: The image is divided into non-overlapping blocks, and a contrast metric is calculated. Noise level Edge strength ; Based on feature indicators and learnable weight parameters Bias terms Calculate the weights of the contrast adaptive histogram equalization algorithm respectively. bilateral filtering denoising algorithm weights and the weights of the guided filter sharpening algorithm And perform weighted fusion.

4. The tumor identification method based on multimodal information encoding according to claim 1, characterized in that, The processing of impedance timing data in step S2 includes: Using the set window step size The original signal is filtered to obtain the output signal; By constructing a virtual grid, based on boundary voltage measurements and input current Iterative solution of conductivity distribution .

5. The tumor identification method based on multimodal information encoding according to claim 1, characterized in that, Step S2 also includes: Calculate structural image quality score and electrical impedance data quality score ; Using quality rating and Normalization determines the fusion weights of each modality feature.

6. The tumor identification method based on multimodal information encoding according to claim 1, characterized in that, Step S3, multi-scale spatial feature extraction, includes: Fine-grained, medium-grained, and coarse-grained feature maps are extracted in parallel using convolutional kernels with different receptive fields. ; Adaptive adjustment of scale weights ; The spatial dimension is reduced by a global average pooling layer, and then execution is performed. Normalization yields structural image coding features .

7. The tumor identification method based on multimodal information encoding according to claim 1, characterized in that, Step S3, multi-directional timing cyclic coding, includes: Construct forward coding units to record the dynamic polarization equilibrium process; Constructing inverse coding units to capture end-slack characteristics; Construct interval coding units to filter out physiological pulsations; The phase features of the initial response segment, the plateau stabilization segment, and the decay release segment are explicitly extracted by constructing segmented coding units.

8. The tumor identification method based on multimodal information encoding according to claim 7, characterized in that, The multi-directional time-series cyclic coding further includes: Calculate the hidden state vector of the multi-directional encoded features Importance rating ; Calculate the weight coefficients for each step And generate a comprehensive feature vector. ; The feature vector is transformed by a fully connected neural network, mapped to a fixed-dimensional feature vector, and then more discriminative feature information is selected to obtain the final feature vector after electrical impedance encoding. .

9. The tumor identification method based on multimodal information encoding according to claim 1, characterized in that, Step S4 specifically includes: By calculating the overall distribution statistics of each eigenvector, and Perform distributed calibration; Metabolic abnormalities through imaging features Impedance Characteristic Metabolic Index and impedance characteristic value The importance of the two standardized modal feature vectors is evaluated separately, and cross-modal attention weights are calculated. and ; The initial fused features are obtained by linearly weighting and summing the standardized feature vectors of the two modalities. The final fused features are obtained by performing two-level feature optimization. .

10. The tumor identification method based on multimodal information encoding according to claim 1, characterized in that, Step S4 also includes: A linear fully connected layer combined with a normalization operator is used to determine the probability of tumor presence. ; For lesions that have been identified, those with the same fusion characteristics will be considered. Mapping to a higher-dimensional property discrimination space to compute benign properties Malignant and boundary probability; Extract deep representations from the fusion features and input them into a four-class classification probability model to calculate the probability of malignancy level grading. .

Images