Bioelectrical impedance tumor identification method based on multi-modal fusion

Abstract

The invention discloses a bioelectrical impedance tumor identification method based on multi-modal fusion. The method comprises the following steps: S1, acquiring electrical impedance signal data and tissue slice image data of a tumor sample; the tissue slice image data are pathological slice images corresponding to tumor tissues; s2, inputting the electrical impedance signal data and the tissue slice image data into a multi-modal feature coding module for asymmetric feature coding; s3, performing splicing and weighted fusion on the global semantic feature vector and the image semantic feature vector to form a multi-modal feature vector; and S4, inputting the multi-modal feature vector into a decoding network for classification prediction, and outputting a tumor identification result. According to the method, an asymmetric coding structure and a weighted fusion mechanism are constructed by fusing the electric signal modality and the image modality, so that the utilization efficiency and the feature discrimination capability of multi-modal information can be effectively improved, and the accuracy and the reliability of tumor recognition are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of tumor recognition technology, and in particular to a bioelectrical impedance tumor recognition method based on multimodal fusion. Background Technology

[0002] Cancer, especially solid tumors such as breast cancer, lung cancer, and liver cancer, is one of the major diseases threatening human health and life. Early detection and accurate identification of tumors are of great significance for improving patients' cure rates and quality of life. Traditional tumor detection methods include tissue biopsy, X-ray, CT, and MRI. While these methods have some diagnostic value, they generally suffer from problems such as complex operation, high cost, long imaging time, and even some invasiveness.

[0003] In recent years, bioelectrical impedance analysis (BIA) technology has been increasingly used in tumor detection due to its non-invasive, rapid, and low-cost characteristics. Bioelectrical impedance reflects the electrical response of tissues to current stimulation at different frequencies, revealing differences in internal tissue structure and lesion characteristics. Therefore, bioelectrical impedance-based tumor detection methods have become a research hotspot. Meanwhile, the rapid development of deep learning technology has made data-driven automatic identification possible, especially demonstrating extremely high accuracy in feature extraction and classification of medical images and biological signals. Deep learning models can extract complex features from large amounts of data, achieving efficient discrimination of tumor signals or images. However, most existing methods focus on single-modal feature extraction, such as using only bioelectrical impedance signals or pathological image information, lacking the fusion and utilization of multimodal information, resulting in limited discriminative ability and robustness of the models.

[0004] Chinese patent CN115018820A discloses a "Multi-classification method for breast cancer based on texture enhancement". This method enhances the extraction of texture information in breast cancer pathological images, increases the model's attention to the texture level of breast cancer tissue pathological images, enhances the model's extraction and recognition of features of breast cancer tissue pathological images, improves classification performance, and alleviates the problem of image heterogeneity.

[0005] However, the above methods rely only on image texture features and cannot utilize heterogeneous modal data such as electrical impedance, so their recognition capabilities are still limited. Therefore, it is urgent to propose a bioelectrical impedance tumor recognition method based on multimodal fusion to solve the problems existing in the above-mentioned technologies. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a bioelectrical impedance analysis (BIA) method for tumor detection based on multimodal fusion. By combining bioelectrical impedance signals with pathological image information and constructing an asymmetric modal encoder and a weighted fusion mechanism, the complementary information between multimodal data is effectively mined, thereby improving the accuracy, robustness, and physical interpretability of tumor identification.

[0007] To achieve the above objectives, according to one aspect of the present invention, a bioelectrical impedance analysis method for tumor identification based on multimodal fusion is provided, the method comprising the following steps: Step S1: Obtain the electrical impedance signal data and tissue section image data of the tumor sample; the tissue section image data is the pathological section image of the corresponding tumor tissue; Step S2: Input the electrical impedance signal data and tissue slice image data into the multimodal feature encoding module for asymmetric feature encoding, wherein the electrical impedance signal data is subjected to temporal feature encoding to generate a global semantic feature vector; and the tissue slice image data is subjected to spatial feature encoding to generate an image semantic feature vector. Step S3: Input the global semantic feature vector and the image semantic feature vector into the multimodal weighted fusion module for concatenation to form a fused input feature vector. Dynamically generate fusion weights based on the fused input feature vector. Perform weighted fusion on the global semantic feature vector and the image semantic feature vector according to the fusion weights to obtain a multimodal feature vector. Step S4: Input the multimodal feature vector into the decoding network for classification and prediction, and output the tumor identification result.

[0008] As an embodiment of this application, step S2, which involves time-series feature encoding of the impedance signal data to generate a global semantic feature vector, specifically includes: S211: Construct the impedance signal input matrix ,in, This represents the length of the impedance signal in the time dimension. Indicates the number of multi-frequency channels collected; S212: Input the impedance signal into the matrix The input is fed into a one-dimensional convolutional neural network for local temporal feature extraction. The calculation formula for the one-dimensional convolution operation is as follows:

[0009] in, Indicates the first The convolution output value at each position. Indicates the first convolution kernel Weight parameters for each position, Indicates the size of the convolution kernel. Represents the input matrix of the impedance signal In the time dimension The input vector at the location, Represents the bias term for the convolution operation; S213: Convolution output value Activation function processing and batch normalization are performed to obtain the initial temporal feature tensor. The formula is as follows:

[0010] in, , These represent the mean and variance of each feature channel within the batch, respectively. , This represents the learnable scaling factor and offset. Represents a small constant. This represents the activation function. This indicates the length of the time series obtained after convolution and downsampling. Indicates the dimension of the output feature; S214: For the initial time-domain feature tensor Perform linear transformations to generate query matrices respectively. Key matrix Value matrix The calculation formula is as follows:

[0011] in, , , These represent the linear transformation weight matrices for the query, key, and value, respectively. S215: Perform attention calculations for each attention head to obtain the single-head attention output. The calculation formula is as follows:

[0012] in, Represents the query matrix. This represents the transpose of the key matrix. This represents the dimension of the key vector, used as a scaling factor. This indicates the similarity calculation between the query and the key. This indicates a normalization operation. Represents a value matrix; S216: Concatenate the calculation results of multiple attention heads and generate the output of the multi-head attention module through linear transformation. The calculation formula is as follows:

[0013] in, Indicates the first to the second The output of each attention head, The number of heads representing multi-head attention. This indicates a splicing operation at the channel level. This indicates the output linear transformation weight matrix; S217: Output to multi-head attention module Perform residual connection and layer normalization operations to generate global semantic feature vectors of impedance signal modes. The calculation formula is as follows:

[0014] in, Presentation layer normalization operation, This indicates the output of the multi-head attention module. This represents the initial time-domain feature tensor of the input. This indicates that residual joins are achieved by adding elements one by one.

[0015] As an embodiment of this application, step S2, which involves spatial feature encoding of the tissue slice image data to generate an image semantic feature vector, specifically includes: S221: Construct the image input tensor The tumor tissue slice images were adjusted to a uniform size, among which, Represents the input image tensor. Indicates the height of the image. Indicates the width of the image. Indicates the number of image channels; S232: Convert the input image tensor The input is fed into a convolutional network based on a residual network structure, and spatial hierarchical features are extracted sequentially through multiple convolutional modules. The output calculation formula for each residual module is as follows:

[0016] in, Indicates the first The output feature map of each residual module This indicates the output of the previous layer. , These represent the weight matrices of the first and second convolutional kernels in the residual block, respectively. This represents the convolution operation. This indicates a batch normalization operation. Represents a non-linear activation function. This represents a residual connection, enabling feature backpropagation and gradient stabilization. S233: Repeatedly stacked residual modules extract spatial structural features of the image at different semantic levels to obtain the intermediate representation tensor of the image modality. ,in, , These represent the height and width of the feature map after convolutional downsampling, respectively. Indicates the number of channels; S234: Perform global average pooling on the output tensor to obtain the image modal semantic feature vector. The calculation formula is as follows:

[0017] in, The final semantic feature vector representing the image modality. Indicates the first Line number Column feature map pixel values, This represents the total number of pixels in the feature map. This indicates the process of calculating the global average.

[0018] As an embodiment of this application, the step S3 of forming the fused input feature vector specifically includes: concatenating the features of the global semantic feature vector and the image semantic feature vector along the channel dimension to obtain the fused input feature vector, and the calculation formula is as follows:

[0019] in, A global semantic feature vector representing the modes of a resistive impedance signal; Semantic feature vectors representing image modalities; This represents the fused input feature vector.

[0020] As an embodiment of this application, the calculation formula for the fusion weight in step S3 is as follows:

[0021] in, The fusion weight represents the impedance mode, and its value ranges from [0,1]. express Activation function This represents the weight matrix of the fused weight network. This indicates the fusion of input feature vectors. This indicates the bias term.

[0022] As an embodiment of this application, the calculation formula for weighted fusion in step S3 is as follows:

[0023] in, This represents the final fused multimodal feature vector. The fusion weights represent the impedance modes. Represents the fusion weights of image modalities. This indicates element-wise multiplication. A global semantic feature vector representing the modes of a resistive impedance signal; A semantic feature vector representing an image modality.

[0024] As an embodiment of this application, the decoding network includes a skip connection mechanism, a multi-layer mapping module, and a Softmax classifier. Step S4 specifically includes: S41: The initial time-domain feature tensor of electrical impedance is concatenated with the multimodal feature vector to obtain the enhanced feature; S42: Input the enhanced features into the multi-layer mapping module for pre-classification processing and output the final feature vector; S43: Input the final feature vector into the Softmax classifier, calculate the predicted probability of each type of tumor, and output the tumor identification result.

[0025] As an embodiment of this application, the calculation formula for feature splicing in step S41 is as follows:

[0026] in, Represents the enhanced feature vector. This represents the fused multimodal features. The initial time-domain characteristic tensor representing electrical impedance. This indicates a splicing operation at the channel dimension.

[0027] As an embodiment of this application, the multi-layer mapping module in step S42 includes at least two fully connected sub-layers. Each fully connected sub-layer includes linear mapping, batch normalization, and an activation function. The mapping process of the two fully connected sub-layers is as follows:

[0028]

[0029] in, This represents the output vector mapped from the first fully connected sublayer. This represents the final output vector mapped from the second fully connected sublayer. , These represent the weight matrices of the first and second fully connected layers, respectively. , Indicates the bias term. This indicates a batch normalization operation. Indicates the activation function; As an embodiment of this application, the calculation formula for the Softmax classifier in step S43 is as follows:

[0030] in, Indicates the first Predictive probability of tumor-like structures This indicates that the final eigenvector is at the th... Activation value on class, This represents the class index variable in the Softmax classifier, used to sum the index scores for all classes. Indicates the total number of categories. It represents the base of the natural logarithm.

[0031] The beneficial effects of this invention are as follows: (1) This invention integrates the frequency response features of electrical signals and the spatial morphological features of images by performing multimodal joint recognition and modeling of bioelectrical impedance signals and tumor slice images. This overcomes the problem of insufficient expression of single modality information and greatly improves the discriminability and robustness of tumor recognition. It is particularly suitable for actual detection scenarios with complex tissue morphology, blurred boundaries or poor signal quality.

[0032] (2) The present invention inputs the electrical impedance signal data and tissue slice image data into a multimodal feature encoding module for asymmetric feature encoding. Based on the differences in structure, dimension and semantic level between the electrical impedance signal and the image mode, a dedicated feature encoder is designed. The electrical impedance mode uses a one-dimensional convolutional neural network to extract temporal features of sequence properties, and the image mode uses a two-dimensional convolutional residual network to extract spatial hierarchical structure features. This realizes the equivalent expression and deep collaboration of cross-modal features, and significantly enhances the model's ability to perceive and fuse heterogeneous information.

[0033] (3) This invention introduces a skip connection mechanism and a multi-layer mapping module through a decoding network, which effectively alleviates the gradient vanishing and semantic degradation problems in the deep fusion process, enabling shallow local features and deep global representations to be modeled in parallel. While maintaining feature integrity, it improves the detection sensitivity of tumors that are difficult to identify, such as those with small volume and blurred boundaries.

[0034] (4) The present invention is based on the fusion input feature vector formed by splicing the global semantic feature vector and the image semantic feature vector, dynamically generating fusion weights, and performing weighted fusion of the global semantic feature vector and the image semantic feature vector according to the fusion weights. It can adaptively learn the complementary relationship and information contribution of each modality under different semantic dimensions, realize the explicit enhancement of highly correlated features between modalities and the suppression of redundant features, and effectively improve the discriminative power and robustness of the fusion features. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating a bioelectrical impedance analysis method for tumor identification based on multimodal fusion provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a multimodal feature encoding module for a bioelectrical impedance analysis method for tumor identification based on multimodal fusion provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of a multimodal weighted fusion module for a bioelectrical impedance analysis method for tumor identification based on multimodal fusion provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the decoding network structure of a bioelectrical impedance analysis method for tumor identification based on multimodal fusion provided in an embodiment of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0037] Reference Figures 1-4 The first aspect of this invention provides a bioelectrical impedance analysis method for tumor identification based on multimodal fusion, the method comprising the following steps: Step S1: Obtain the electrical impedance signal data and tissue section image data of the tumor sample; the tissue section image data is the pathological section image of the corresponding tumor tissue; Specifically, this invention first uses a multi-frequency bioelectrical impedance analysis device to non-invasively scan tumor tissue, acquiring its electrical impedance characteristics data at multiple frequency points. Furthermore, combined with pathological image data acquisition, tumor tissue slice images corresponding to the electrical signals are obtained, providing an input basis for subsequent construction of image modal channels. These images, being tissue slices, reflect the morphology, structure, boundaries, and distribution patterns of tumor cells, laying the foundation for multimodal feature modeling.

[0038] Step S2: Input the electrical impedance signal data and tissue slice image data into the multimodal feature encoding module for asymmetric feature encoding, wherein the electrical impedance signal data is subjected to temporal feature encoding to generate a global semantic feature vector; and the tissue slice image data is subjected to spatial feature encoding to generate an image semantic feature vector. Step S3: Input the global semantic feature vector and the image semantic feature vector into the multimodal weighted fusion module for concatenation to form a fused input feature vector. Dynamically generate fusion weights based on the fused input feature vector. Perform weighted fusion on the global semantic feature vector and the image semantic feature vector according to the fusion weights to obtain a multimodal feature vector. Step S4: Input the multimodal feature vector into the decoding network for classification and prediction, and output the tumor identification result.

[0039] As an embodiment of this application, step S2, which involves time-series feature encoding of the impedance signal data to generate a global semantic feature vector, specifically includes: S211: Construct the impedance signal input matrix ,in, This represents the length of the impedance signal in the time dimension. Indicates the number of multi-frequency channels collected; S212: Input the impedance signal into the matrix The input is fed into a one-dimensional convolutional neural network for local temporal feature extraction. The calculation formula for the one-dimensional convolution operation is as follows:

[0040] in, Indicates the first The convolution output value at each position. Indicates the first convolution kernel Weight parameters for each position, Indicates the size of the convolution kernel. Represents the input matrix of the impedance signal In the time dimension The input vector at the location, Represents the bias term for the convolution operation; S213: Convolution output value Activation function processing and batch normalization are performed to obtain the initial temporal feature tensor. The formula is as follows:

[0041] in, , These represent the mean and variance of each feature channel within the batch, respectively. , This represents the learnable scaling factor and offset. Represents a small constant. This represents the activation function. This indicates the length of the time series obtained after convolution and downsampling. Indicates the dimension of the output feature; Specifically, this invention first constructs the original impedance signal as a two-dimensional input matrix, where the row dimension represents the sampling length in the time dimension, and the column dimension represents the number of frequency channels acquired. A one-dimensional convolutional neural network is then used to perform a local sliding window operation on this matrix to extract local variation patterns of the impedance signal in the time domain, such as impedance fluctuations and signal amplitude changes. Each convolutional kernel can sense local frequency band changes of a fixed width, thereby effectively characterizing the electrical properties of the microstructure. Furthermore, the activation function and batch normalization operation enhance the nonlinear expressive power of the features and improve the stability during training.

[0042] S214: For the initial time-domain feature tensor Perform linear transformations to generate query matrices respectively. Key matrix Value matrix The calculation formula is as follows:

[0043] in, , , These represent the linear transformation weight matrices for the query, key, and value, respectively. S215: Perform attention calculations for each attention head to obtain the single-head attention output. The calculation formula is as follows:

[0044] in, Represents the query matrix. This represents the transpose of the key matrix. This represents the dimension of the key vector, used as a scaling factor. This indicates the similarity calculation between the query and the key. This indicates a normalization operation. Represents a value matrix; S216: Concatenate the calculation results of multiple attention heads and generate the output of the multi-head attention module through linear transformation. The calculation formula is as follows:

[0045] in, Indicates the first to the second The output of each attention head, The number of heads representing multi-head attention. This indicates a splicing operation at the channel level. This indicates the output linear transformation weight matrix; S217: Output to multi-head attention module Perform residual connection and layer normalization operations to generate global semantic feature vectors of impedance signal modes. The calculation formula is as follows:

[0046] in, Presentation layer normalization operation, This indicates the output of the multi-head attention module. This represents the initial time-domain feature tensor of the input. This indicates that residual joins are achieved by adding elements one by one.

[0047] Specifically, a multi-head attention mechanism is introduced to model the temporal features extracted by convolution. The multi-head attention module learns linear mappings of queries, keys, and values ​​separately, capturing long-distance dependencies within the signal and enhancing the model's ability to focus on key frequency regions. Attention scores measure the semantic relevance between different frequency points. After parallel computation by multiple attention heads, the information is integrated through concatenation and linear transformation. Finally, residual connections and layer normalization operations are used to obtain a semantically consistent and stable global semantic vector for the impedance signal, effectively improving the signal mode's ability to represent complex organizational structures.

[0048] As an embodiment of this application, step S2, which involves spatial feature encoding of the tissue slice image data to generate an image semantic feature vector, specifically includes: S221: Construct the image input tensor The tumor tissue slice images were adjusted to a uniform size, among which, Represents the input image tensor. Indicates the height of the image. Indicates the width of the image. This indicates the number of image channels, typically 3 (RGB image). S232: Convert the input image tensor The input is fed into a convolutional network based on a residual network structure, and spatial hierarchical features are extracted sequentially through multiple convolutional modules. The output calculation formula for each residual module is as follows:

[0049] in, Indicates the first The output feature map of each residual module This indicates the output of the previous layer. , These represent the weight matrices of the first and second convolutional kernels in the residual block, respectively. This represents the convolution operation. This indicates a batch normalization operation. To represent a non-linear activation function, this application preferentially uses the ReLU function. This represents a residual connection, enabling feature backpropagation and gradient stabilization. S233: Repeatedly stacked residual modules extract spatial structural features of the image at different semantic levels to obtain the intermediate representation tensor of the image modality. ,in, , These represent the height and width of the feature map after convolutional downsampling, respectively. Indicates the number of channels; S234: Perform global average pooling on the output tensor to obtain the final semantic feature vector of the image modality. The calculation formula is as follows:

[0050] in, The final semantic feature vector representing the image modality. Indicates the first Line number Column feature map pixel values, This represents the total number of pixels in the feature map. This represents the global averaging process. At this point, feature extraction of the image modalities is complete, resulting in a vector representation in a unified semantic space, which can be used for fusion with electrical impedance signals.

[0051] Specifically, this invention uniformly adjusts tumor tissue slice images to a specified size to form the input tensor. The image encoding path based on the residual network results includes multiple stacked residual blocks, each module consisting of consecutive convolutions, normalization, and activation functions. Residual connections alleviate the gradient vanishing problem in deep network training. This design allows the network to gradually extract higher-level semantic features, such as the arrangement pattern of tumor cells, edge morphology, and density changes, while preserving low-level structural information. Finally, through global average pooling, the spatial dimension feature mapping is compressed into a global semantic vector of the image modality, providing a structural foundation for subsequent multimodal fusion.

[0052] This invention constructs two asymmetric coding paths for electrical impedance signal data and tissue slice image data: For electrical impedance signal data, a one-dimensional convolutional neural network is used to extract temporal features of sequence properties, while for tissue slice image data, a two-dimensional convolutional residual network is used to extract spatial hierarchical structural features. For the electrical impedance signal, multi-layer one-dimensional convolutional operations are used to model the response changes between different frequency points, preserving the continuity and abrupt changes of the bioelectrical response in the frequency domain; simultaneously, an attention mechanism is used to further mine the characteristic response intensity of key frequency bands, obtaining a weighted global semantic feature vector. The image modality coding path is based on a residual network structure, relying on multi-layer residual units to extract spatial representation information such as texture patterns and cell distribution characteristics of pathological images step by step, effectively modeling the differences in tumor structural expression at different scales.

[0053] As an embodiment of this application, the step S3 of forming the fused input feature vector specifically includes: concatenating the features of the global semantic feature vector and the image semantic feature vector along the channel dimension to obtain the fused input feature vector, and the calculation formula is as follows:

[0054] in, A global semantic feature vector representing the modes of a resistive impedance signal; Semantic feature vectors representing image modalities; This represents the fused input feature vector.

[0055] As an embodiment of this application, the calculation formula for the fusion weight in step S3 is as follows:

[0056] in, The fusion weight represents the impedance mode, and its value ranges from [0,1]. express Activation function This represents the weight matrix of the fused weight network. This indicates the fusion of input feature vectors. This indicates the bias term.

[0057] Specifically, this invention concatenates feature vectors from the electrical impedance mode and the image mode, and automatically assigns an importance weight to each mode in the fusion process based on a fusion weight generation mechanism. This design avoids the subjectivity of manually setting weights, making the fusion process more intelligent and flexible. It can dynamically adjust the modal contribution according to the feature differences of different input samples, effectively improving the robustness and adaptability of recognition.

[0058] As an embodiment of this application, the calculation formula for weighted fusion in step S3 is as follows:

[0059] in, This represents the final fused multimodal feature vector. The fusion weights represent the impedance modes. Represents the fusion weights of image modalities. This indicates element-wise multiplication. A global semantic feature vector representing the modes of a resistive impedance signal; A semantic feature vector representing an image modality.

[0060] Specifically, this application performs weighted fusion of features from the electrical impedance mode and the image mode based on fusion weights. This process not only preserves the discriminative information of each mode but also emphasizes the modal content that is more critical to the current recognition task. Thus, while retaining multi-source information, it effectively suppresses redundant interference and improves the discriminative power of the final fused features. This weight-controlled information fusion method enhances the model's ability to recognize complex tumor features and improves classification accuracy and the model's generalization performance.

[0061] As an embodiment of this application, the decoding network includes a skip connection mechanism, a multi-layer mapping module, and a Softmax classifier. Step S4 specifically includes: S41: Enhanced features are obtained by concatenating the initial temporal feature tensor of electrical impedance with the multimodal feature vector through a skip connection mechanism; S42: Input the enhanced features into the multi-layer mapping module for pre-classification processing and output the final feature vector; S43: Input the final feature vector into the Softmax classifier, calculate the predicted probability of each type of tumor, and output the tumor identification result.

[0062] As an embodiment of this application, the calculation formula for feature splicing in step S41 is as follows:

[0063] in, Represents the enhanced feature vector. This represents the fused multimodal features. The initial time-domain characteristic tensor representing electrical impedance. This indicates a splicing operation at the channel dimension.

[0064] As an embodiment of this application, the multi-layer mapping module in step S42 includes at least two fully connected sub-layers. Each fully connected sub-layer includes linear mapping, batch normalization, and an activation function. The mapping process of the two fully connected sub-layers is as follows:

[0065]

[0066] in, This represents the output vector mapped from the first fully connected sublayer. This represents the final output vector mapped from the second fully connected sublayer. , These represent the weight matrices of the first and second fully connected layers, respectively. , Indicates the bias term. This indicates a batch normalization operation. Indicates the activation function; Specifically, the method designs a decoding network with a fusion enhancement mechanism. This decoding network introduces a skip connection mechanism to concatenate the initial electrical impedance temporal features with the fused multimodal features, effectively preserving low-level local details and early electrical signal features, thereby enhancing the model's discriminative ability before classification. Simultaneously, the subsequent multi-layer mapping module achieves high-dimensional reconstruction and semantic abstraction of the feature space through continuous feature mapping and nonlinear transformation, which helps to uncover potential high-order discriminative information and provides stronger semantic support for the final classification.

[0067] As an embodiment of this application, the calculation formula for the Softmax classifier in step S43 is as follows:

[0068] in, Indicates the first Predictive probability of tumor-like structures This indicates that the final eigenvector is at the th... Activation value on class, This represents the class index variable in the Softmax classifier, used to sum the index scores for all classes. Indicates the total number of categories. It represents the base of the natural logarithm.

[0069] Specifically, this invention feeds the final fused enhanced features into a Softmax classifier to predict the probability distribution of tumor categories and outputs recognition results with clear classification labels. The Softmax classifier has a stable and interpretable output mechanism, ensuring the reliability and consistency of classification decisions. By combining a structured feature enhancement path with a flexible classification decision mechanism, this invention significantly improves the accuracy of tumor recognition under multimodal fusion, especially in complex samples with blurred boundaries or indistinct features, demonstrating stronger classification capabilities.

[0070] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A bioelectrical impedance analysis method for tumor identification based on multimodal fusion, characterized in that, The method includes the following steps: Step S1: Obtain the electrical impedance signal data and tissue section image data of the tumor sample; the tissue section image data is the pathological section image of the corresponding tumor tissue; Step S2: Input the electrical impedance signal data and tissue slice image data into the multimodal feature encoding module for asymmetric feature encoding, wherein the electrical impedance signal data is subjected to temporal feature encoding to generate a global semantic feature vector; and the tissue slice image data is subjected to spatial feature encoding to generate an image semantic feature vector. Step S3: Input the global semantic feature vector and the image semantic feature vector into the multimodal weighted fusion module for concatenation to form a fused input feature vector. Dynamically generate fusion weights based on the fused input feature vector. Perform weighted fusion on the global semantic feature vector and the image semantic feature vector according to the fusion weights to obtain a multimodal feature vector. Step S4: Input the multimodal feature vector into the decoding network for classification and prediction, and output the tumor identification result.

2. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 1, characterized in that, Step S2, which involves temporal feature encoding of the impedance signal data to generate a global semantic feature vector, specifically includes: S211: Construct the impedance signal input matrix ,in, This represents the length of the impedance signal in the time dimension. Indicates the number of multi-frequency channels collected; S212: Input the impedance signal into the matrix The input is fed into a one-dimensional convolutional neural network for local temporal feature extraction. The calculation formula for the one-dimensional convolution operation is as follows: in, Indicates the first The convolution output value at each position. Indicates the first convolution kernel Weight parameters for each position, Indicates the size of the convolution kernel. Represents the input matrix of the impedance signal In the time dimension The input vector at the location, Represents the bias term for the convolution operation; S213: Convolution output value Activation function processing and batch normalization are performed to obtain the initial temporal feature tensor. The formula is as follows: in, , These represent the mean and variance of each feature channel within the batch, respectively. , This represents the learnable scaling factor and offset. Represents a small constant. This represents the activation function. This indicates the length of the time series obtained after convolution and downsampling. Indicates the dimension of the output feature; S214: For the initial time-domain feature tensor Perform linear transformations to generate query matrices respectively. Key matrix Value matrix The calculation formula is as follows: in, , , These represent the linear transformation weight matrices for the query, key, and value, respectively. S215: Perform attention calculations for each attention head to obtain the single-head attention output. The calculation formula is as follows: in, Represents the query matrix. This represents the transpose of the key matrix. This represents the dimension of the key vector, used as a scaling factor. This indicates the similarity calculation between the query and the key. This indicates a normalization operation. Represents a value matrix; S216: Concatenate the calculation results of multiple attention heads and generate the output of the multi-head attention module through linear transformation. The calculation formula is as follows: in, Indicates the first to the second The output of each attention head, The number of heads representing multi-head attention. This indicates a splicing operation at the channel level. This indicates the output linear transformation weight matrix; S217: Output to multi-head attention module Perform residual connection and layer normalization operations to generate global semantic feature vectors of impedance signal modes. The calculation formula is as follows: in, Presentation layer normalization operation, This indicates the output of the multi-head attention module. This represents the initial time-domain feature tensor of the input. This indicates that residual joins are achieved by adding elements one by one.

3. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 1, characterized in that, Step S2, which involves spatial feature encoding of the tissue slice image data to generate an image semantic feature vector, specifically includes: S221: Construct the image input tensor The tumor tissue slice images were adjusted to a uniform size, among which, Represents the input image tensor. Indicates the height of the image. Indicates the width of the image. Indicates the number of image channels; S232: Convert the input image tensor The input is fed into a convolutional network based on a residual network structure, and spatial hierarchical features are extracted sequentially through multiple convolutional modules. The output calculation formula for each residual module is as follows: in, Indicates the first The output feature map of each residual module This indicates the output of the previous layer. , These represent the weight matrices of the first and second convolutional kernels in the residual block, respectively. This represents the convolution operation. This indicates a batch normalization operation. Represents a non-linear activation function. This represents a residual connection, enabling feature backpropagation and gradient stabilization. S233: Repeatedly stacked residual modules extract spatial structural features of the image at different semantic levels to obtain the intermediate representation tensor of the image modality. ,in, , These represent the height and width of the feature map after convolutional downsampling, respectively. Indicates the number of channels; S234: Perform global average pooling on the output tensor to obtain the image modal semantic feature vector. The calculation formula is as follows: in, The final semantic feature vector representing the image modality. Indicates the first Line number Column feature map pixel values, This represents the total number of pixels in the feature map. This indicates the process of calculating the global average.

4. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 1, characterized in that, The step S3 of forming the fused input feature vector specifically includes: concatenating the features of the global semantic feature vector and the image semantic feature vector along the channel dimension to obtain the fused input feature vector, and the calculation formula is as follows: in, A global semantic feature vector representing the modes of a resistive impedance signal; Semantic feature vectors representing image modalities; This represents the fused input feature vector.

5. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 1, characterized in that, The formula for calculating the fusion weight in step S3 is as follows: in, The fusion weight represents the impedance mode, and its value ranges from [0,1]. express Activation function This represents the weight matrix of the fused weight network. This indicates the fusion of input feature vectors. This indicates the bias term.

6. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 1, characterized in that, The calculation formula for weighted fusion in step S3 is as follows: in, This represents the final fused multimodal feature vector. The fusion weights represent the impedance modes. Represents the fusion weights of image modalities. This indicates element-wise multiplication. A global semantic feature vector representing the modes of a resistive impedance signal; A semantic feature vector representing an image modality.

7. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 1, characterized in that, The decoding network includes a skip connection mechanism, a multi-layer mapping module, and a Softmax classifier. Step S4 specifically includes: S41: The initial time-domain feature tensor of electrical impedance is concatenated with the multimodal feature vector to obtain the enhanced feature; S42: Input the enhanced features into the multi-layer mapping module for pre-classification processing and output the final feature vector; S43: Input the final feature vector into the Softmax classifier, calculate the predicted probability of each type of tumor, and output the tumor identification result.

8. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 7, characterized in that, The calculation formula for feature splicing in step S41 is as follows: in, Represents the enhanced feature vector. This represents the fused multimodal features. The initial time-domain characteristic tensor representing electrical impedance. This indicates a splicing operation at the channel dimension.

9. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 7, characterized in that, The multi-layer mapping module in step S42 includes at least two fully connected sub-layers. Each fully connected sub-layer includes linear mapping, batch normalization, and an activation function. The mapping process of the two fully connected sub-layers is as follows: in, This represents the output vector mapped from the first fully connected sublayer. This represents the final output vector mapped from the second fully connected sublayer. , These represent the weight matrices of the first and second fully connected layers, respectively. , Indicates the bias term. This indicates a batch normalization operation. This represents the activation function.

10. The bioelectrical impedance analysis method for tumor identification based on multimodal fusion as described in claim 7, characterized in that, The calculation formula for the Softmax classifier in step S43 is as follows: in, Indicates the first Predictive probability of tumor-like structures This indicates that the final eigenvector is at the th... Activation value on class, This represents the class index variable in the Softmax classifier, used to sum the index scores for all classes. Indicates the total number of categories. It represents the base of the natural logarithm.

Images