Individual treatment effect estimation method based on multi-modal data fusion
By using multimodal data fusion and deep full-space cross-network optimization model, the problems of complex nonlinear feature interactions and sample importance differences in the ITE estimation method are solved, achieving high-precision prediction of individual treatment effects and model generalization.
Patent Information
- Application Number
- CN202512009637.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-07
AI Technical Summary
Existing ITE estimation methods struggle to effectively handle complex nonlinear feature interactions in multimodal data, neglect differences in sample importance, and the simple cross-structure of deep learning models limits estimation accuracy and model expressive power.
By acquiring multimodal data, performing preprocessing and feature mapping, using an attention mechanism for weighted fusion, and using a deep full-space cross-network optimization model for analysis, multi-layer cross-networks and attention mechanisms are introduced to enhance feature interaction capabilities.
It improves the accuracy of individual treatment effect estimation and model generalization ability, dynamically adjusts the weights of different data sources, effectively integrates heterogeneous information, and enhances prediction accuracy and robustness.
Smart Images
Figure CN121812155A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of machine learning, in particular to an individual treatment effect estimation method based on multi-modal data fusion. BACKGROUND
[0002] In recent years, with the rapid development of big data and machine learning technology, individual treatment effect estimation (ITE) has been increasingly applied in medical, financial, e-commerce and other fields. ITE aims to infer the potential response of individuals under different intervention conditions, such as predicting the efficacy of a treatment plan for a specific patient in a medical scenario, or evaluating the impact of a promotion activity on individual user purchasing behavior in an e-commerce scenario. With the increasing richness of multi-modal data, deep learning-based ITE methods have gradually become a research hotspot. Such methods can effectively model complex nonlinear relationships and high-dimensional feature interactions through end-to-end training and feature interaction learning, thereby improving prediction performance.
[0003] Currently, ITE estimation methods can be mainly divided into two categories: one is based on traditional statistical model assumptions, and the other is based on modern deep learning methods. Traditional methods such as propensity score matching (PSM) and inverse probability weighting (IPW) usually estimate individual effects by grouping samples and estimating the average effect of each group. However, such methods often have difficulty in describing complex nonlinear dependence relationships between features, and their performance is limited in high-dimensional or multi-modal data scenarios, and they are easily affected by sample imbalance and feature redundancy problems.
[0004] With the progress of deep learning technology, neural network-based ITE estimation methods have gradually emerged, especially by constructing deep cross-networks to capture high-order interactions between features. However, existing methods still face the following key challenges: first, most methods can handle multi-source data, but they do not fully consider the differences between different data modalities; second, traditional methods and existing deep learning models usually assume that all samples contribute equally to model training, ignoring the impact of sample importance differences on estimation results; third, although existing deep learning models have certain feature interaction modeling capabilities, their cross-structure is often simple, making it difficult to effectively capture high-order, complex nonlinear feature interaction relationships, limiting the model's expressive power and estimation accuracy. SUMMARY
[0005] To solve the above problems, the present application provides an individual treatment effect estimation method based on multi-modal data fusion. The method comprises:
[0006] Acquire multimodal data related to individual treatment effects and preprocess the multimodal data, which includes user initial features, treatment information features, and treatment record information features;
[0007] Feature mapping is performed on the preprocessed multimodal data to obtain a low-dimensional representation vector of the same dimension for different modalities;
[0008] An attention mechanism is used to weight and fuse low-dimensional representation vectors of different modalities with a unified dimension to obtain a fused feature set;
[0009] The fused feature set is input into the trained deep full-space cross-network optimization model for analysis and processing to obtain individualized effect prediction results for different treatment methods. The deep full-space cross-network optimization model includes multiple cross-networks, and an attention mechanism is introduced between each cross-network layer and between each cross-network layer.
[0010] Optionally, the step of performing feature mapping on the preprocessed multimodal data to obtain a low-dimensional representation vector of a unified dimension for different modalities includes:
[0011] For each modality, the individual treatment effect data is one-hot encoded to obtain encoded features. The encoded features are then input into a pre-constructed corresponding modality embedding layer. The transformation function of the embedding layer maps the individual treatment effect data of each modality into a low-dimensional representation vector of the same dimension.
[0012] Optionally, the attention mechanism is used to perform weighted fusion of low-dimensional representation vectors of different modalities with a unified dimension to obtain fused features, satisfying the following formula:
[0013] The attention weights for each modality are determined using an attention mechanism, satisfying the following formula:
[0014]
[0015] in, It is the first Attention weights for each modality This represents a learnable weight vector. This represents the transpose of the learnable weight vector. For the first A low-dimensional representation vector of each modality. For the first Low-dimensional representation vectors of each modality;
[0016] The low-dimensional representation vectors of all modalities are weighted and fused according to the attention weights corresponding to each modality to obtain the fused features.
[0017] Optionally, the step of inputting the fused feature set into the trained deep full-space cross-network optimization model for analysis and processing to obtain individualized effect prediction results for different treatment methods includes:
[0018] The fused feature set is input into the first-layer cross network for feature enhancement, and the output of the first-layer cross network is obtained.
[0019] The output of the first nonlinear cross layer is sequentially passed through multiple cross networks for feature enhancement, resulting in the output of the last cross network. The feature enhancement process of each cross network layer satisfies the following formula:
[0020]
[0021] in, Indicates the first The output of the layer cross network, Indicates the first The output of the layer cross network, Indicates the first The weight matrix of the layer cross network, It is the first Bias terms in layer cross-networks, It is the first Nonlinear activation functions for layer cross-networks;
[0022] The output of the last cross-network layer is passed through a fully connected layer for feature mapping to obtain individualized effect prediction results for different treatment methods.
[0023] Optionally, the step of inputting the fused feature set into the trained deep full-space cross-network optimization model for analysis and processing further includes: calculating the attention weight of each pair of feature interactions in the fused feature set in each layer of the cross-network, satisfying the following formula:
[0024]
[0025] in, To represent features and Attention weights in interactions This represents a learnable weight vector. This represents the transpose of the learnable weight vector. This represents element-wise multiplication.
[0026] Optionally, the step of inputting the fused feature set into the trained deep full-space cross-network optimization model for analysis and processing further includes:
[0027] The first The output of the layer cross-connect network and the output of the layer above cross-connect network Feature interaction is performed, satisfying the following formula:
[0028]
[0029] in, Indicates the first The output of the layer, Indicates the first The output of the layer, This represents the weight matrix of the layer. For bias terms, It is a non-linear activation function.
[0030] Optionally, the training process of the deep full-space cross-network optimization model includes:
[0031] The sample weight is determined based on the predicted and actual values of each sample. The cross-entropy loss is then weighted according to the sample weights to obtain the weighted cross-entropy loss.
[0032] Construct the target loss function based on the weighted cross-entropy loss and L2 regularization loss;
[0033] The deep full-space cross-network optimization model is trained using a preset optimizer based on the objective loss function, and the hyperparameters are optimized through cross-validation to obtain the trained deep full-space cross-network optimization model.
[0034] This application provides a method for estimating individual treatment effects based on multimodal data fusion. The method acquires multimodal data related to individual treatment effects and preprocesses this data, including user initial features, treatment information features, and treatment record information features. Feature mapping is performed on the preprocessed multimodal data to obtain low-dimensional representation vectors of different modalities with a unified dimension. An attention mechanism is used to weightedly fuse these low-dimensional representation vectors of different modalities to obtain a fused feature set. This fused feature set is then input into a trained deep full-space cross-network optimization model for analysis and processing to obtain individualized effect prediction results for different treatment methods. The deep full-space cross-network optimization model includes multiple layers of cross-networks, and an attention mechanism is introduced between each layer and between the layers. This method improves the accuracy of individual treatment effect estimation and the model's generalization ability. Attached Figure Description
[0035] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0036] Figure 1 This is a flowchart illustrating the individual treatment effect estimation method based on multimodal data fusion provided in an embodiment of this application.
[0037] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0039] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.
[0040] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0041] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0042] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0043] Figure 1 This is a flowchart illustrating the individual treatment effect estimation method based on multimodal data fusion provided in an embodiment of this application.Figure 1 As shown in this embodiment, the individual treatment effect estimation method based on multimodal data fusion includes:
[0044] S1: Acquire multimodal data related to individual treatment effects and preprocess the multimodal data.
[0045] The multimodal data includes user initial features, treatment information features, and treatment record information features.
[0046] For example, data preprocessing includes converting time data into timestamps and performing category conversion on category data; initial user features should include basic information such as user ID, gender, and age. Treatment record information features should include user ID, time, whether treatment was selected, treatment variable name, and whether treatment was received, where the time field should be converted to timestamp format and the treatment received field should be categorized.
[0047] Understandably, preprocessing multimodal data ensures data quality and consistency.
[0048] S2: Perform feature mapping on the preprocessed multimodal data to obtain a low-dimensional representation vector of the same dimension for different modalities.
[0049] Specifically, one-hot encoding is performed on the individual treatment effect data for each modality to obtain encoded features. The encoded features are then input into the pre-constructed corresponding modality embedding layer. The transformation function of the embedding layer maps the individual treatment effect data for each modality into a low-dimensional representation vector of the same dimension.
[0050] For example, the data is first converted into a format suitable for neural network input using one-hot encoding. Then, the data is mapped to a unified low-dimensional representation space:
[0051]
[0052] in, It is the first Data for each modality, This indicates that the modal data is transformed through the embedding layer's transformation function. The transformed low-dimensional representation can effectively capture relevant information about the modality features.
[0053] S3: An attention mechanism is used to weight and fuse the low-dimensional representation vectors of different modalities with a unified dimension to obtain a fused feature set.
[0054] Understandably, after executing step S2, a low-dimensional representation of the data mapping is obtained. To effectively combine information from different data sources, a weighted attention mechanism is introduced, with weights assigned to each modality. Dynamic learning through an attention mechanism reflects the impact of each modality on the final prediction. Specifically, for each modality, weights are calculated... as follows:
[0055]
[0056] in, It is the first Attention weights for each modality This represents a learnable weight vector. This represents the transpose of the learnable weight vector. For the first A low-dimensional representation vector of each modality. For the first A low-dimensional representation vector of each modality. Attention weights can automatically adjust the importance of each modality based on its contribution to the final prediction.
[0057] The weighted modal features are then fused to obtain the final feature representation. :
[0058]
[0059] in, It is a fused feature obtained by weighting and summing all modal features, and this feature is ultimately used as the input to the subsequent model.
[0060] Understandably, by introducing an attention mechanism, the model can dynamically evaluate and adjust the weights of different data sources (such as medical records, e-commerce data, environmental factors, etc.), effectively integrate heterogeneous information, avoid the information loss or redundancy problems common in traditional methods, and significantly improve the efficiency and prediction accuracy of multimodal data fusion.
[0061] S4: Input the fused feature set into the trained deep full-space cross-network optimization model for analysis and processing to obtain the individualized effect prediction results of different treatment methods.
[0062] The deep full-space cross-network optimization model includes multiple cross-networks and introduces an attention mechanism between each cross-network layer and between each cross-network layer.
[0063] Understandably, the Deep Entire Space Cross Network (DESCN) is an innovative deep learning architecture that aims to improve model performance by jointly learning processing and response functions through a full-space network, capturing comprehensive information through cross-networks, and introducing pseudo-treatment effect predictions to alleviate imbalances. This invention, based on the original DESCN model, designs nonlinear cross layers and enhances the expressive power of feature interactions by introducing nonlinear activation functions in each layer.
[0064] Specifically, the feature set will be fused , It is the fused feature representation, which is input into the first-layer cross network for feature enhancement, and the output of the first-layer cross network is obtained.
[0065] The output of the first nonlinear cross layer is sequentially passed through multiple cross networks for feature enhancement, resulting in the output of the last cross network. The feature enhancement process of each cross network layer satisfies the following formula:
[0066]
[0067] in, Indicates the first The output of the layer cross network, Indicates the first The output of the layer cross network, Indicates the first The weight matrix of the layer cross network, It is the first Bias terms in layer cross-networks, It is the first Nonlinear activation functions for layer cross-networks.
[0068] In an optional embodiment, in each layer of the cross-network, the attention weights for each pair of feature interactions in the fused feature set are computed. , indicating features and The interaction strength between them is determined by the following formula:
[0069]
[0070] in, To represent features and Attention weights in interactions This represents a learnable weight vector. This represents the transpose of the learnable weight vector. This represents element-wise multiplication.
[0071] Understandably, incorporating attention mechanisms into the feature interaction learning process enables the model to automatically identify key samples and important features, dynamically adjust their weight allocation, improve the model's sensitivity to key information, effectively suppress overfitting, and enhance the model's robustness and generalization performance.
[0072] In an optional embodiment, a multi-layer cross-neural network is designed, which not only performs feature crossing within the same layer, but also continuously performs cross-layer feature crossing between different layers. Specifically, assume that the first... The output of the layer is , will the The output of the layer and the output of the layer above By crossing over, new feature interactions are formed:
[0073]
[0074] in, Indicates the first The output of the layer, Indicates the first The output of the layer, This represents the weight matrix of the layer. For bias terms, It is a non-linear activation function.
[0075] The output of the last cross-connection layer is passed through a fully connected layer for feature mapping to obtain individualized effect predictions for different treatment methods. Specifically, this is represented as follows:
[0076]
[0077] in, The output of the last cross-network layer, It is an activation function. For the weights of the output layer, This is the bias term for the output layer.
[0078] In an optional embodiment, the training process of the deep full-space cross-network optimization model includes: obtaining a training set and a validation set for model training; determining the sample weight based on the predicted and true values of each sample; and weighting the cross-entropy loss according to the sample weights to obtain a weighted cross-entropy loss, the weighted loss function being as follows:
[0079]
[0080] in, This represents the total number of samples. Indicates the first The weights of each sample, Indicates the first The predicted value for each sample, Indicates the first The true value of each sample This represents the cross-entropy loss function. Dynamically calculate the importance of the samples , It is a small constant, so division by zero should be avoided.
[0081] Furthermore, the objective loss function is constructed based on the weighted cross-entropy loss and L2 regularization loss, and is expressed by the following formula:
[0082]
[0083] in, Represents the target loss function. This represents the weighted loss function. The L2 regularization loss is represented by the following calculation method: , Indicates the first One parameter, It is the square of the L2 norm. This is the regularization coefficient, used to control the strength of regularization. The L2 regularization term is introduced to prevent overfitting of the model.
[0084] Furthermore, a pre-defined optimizer is used to train the deep full-space cross-network optimization model based on the objective loss function, and the hyperparameters are optimized through cross-validation to obtain the trained deep full-space cross-network optimization model.
[0085] For example, a mini-batch gradient descent algorithm (such as stochastic gradient descent (SGD), Adam optimization algorithm, etc.) is used for parameter updates, with a small learning rate, and a certain number of iterations are trained until the target loss function converges or a preset stopping condition is met. During model training, the hyperparameters of the model are adjusted and optimized by observing the performance metrics on the validation set.
[0086] The individual treatment effect estimation method based on multimodal data fusion provided in this application involves acquiring multimodal data related to individual treatment effects and preprocessing the multimodal data; performing feature mapping on the preprocessed multimodal data to obtain low-dimensional representation vectors of different modalities with a unified dimension; employing an attention mechanism to weightedly fuse the low-dimensional representation vectors of different modalities to obtain a fused feature set; and inputting the fused feature set into a trained deep full-space cross-network optimization model for analysis and processing to obtain individualized effect prediction results for different treatment methods. The deep full-space cross-network optimization model includes multiple layers of cross-networks, and an attention mechanism is introduced between each layer of cross-networks. This method improves the accuracy of individual treatment effect estimation and the model's generalization ability.
[0087] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0088] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method for estimating individual treatment effects based on multimodal data fusion, characterized in that, The method includes: Acquire multimodal data related to individual treatment effects and preprocess the multimodal data, which includes user initial features, treatment information features, and treatment record information features; Feature mapping is performed on the preprocessed multimodal data to obtain low-dimensional representation vectors of different modalities with a unified dimension; An attention mechanism is used to weight and fuse low-dimensional representation vectors of different modalities with a unified dimension to obtain a fused feature set; The fused feature set is input into the trained deep full-space cross-network optimization model for analysis and processing to obtain individualized effect prediction results for different treatment methods. The deep full-space cross-network optimization model includes multiple cross-networks, and an attention mechanism is introduced between each cross-network layer and between each cross-network layer.
2. The method according to claim 1, characterized in that, The step of performing feature mapping on the preprocessed multimodal data to obtain a low-dimensional representation vector of a unified dimension for different modalities includes: For each modality, the individual treatment effect data is one-hot encoded to obtain encoded features. The encoded features are then input into a pre-constructed corresponding modality embedding layer. The transformation function of the embedding layer maps the individual treatment effect data of each modality into a low-dimensional representation vector of the same dimension.
3. The method according to claim 1, characterized in that, The attention mechanism is used to weight and fuse low-dimensional representation vectors of different modalities with a unified dimension to obtain fused features, which satisfy the following formula: The attention weights for each modality are determined using an attention mechanism, satisfying the following formula: ； in, It is the first Attention weights for each modality This represents a learnable weight vector. This represents the transpose of the learnable weight vector. For the first A low-dimensional representation vector of each modality. For the first Low-dimensional representation vectors of each modality; The low-dimensional representation vectors of all modalities are weighted and fused according to the attention weights corresponding to each modality to obtain the fused features.
4. The method according to claim 1, characterized in that, The process involves inputting the fused feature set into a trained deep full-space cross-network optimization model for analysis and processing to obtain individualized effect prediction results for different treatment methods, including: The fused feature set is input into the first-layer cross network for feature enhancement, and the output of the first-layer cross network is obtained. The output of the first nonlinear cross layer is sequentially passed through multiple cross networks for feature enhancement, resulting in the output of the last cross network. The feature enhancement process of each cross network layer satisfies the following formula: ； in, Indicates the first The output of the layer cross network, Indicates the first The output of the layer cross network, Indicates the first The weight matrix of the layer cross network, It is the first Bias terms in layer cross-networks, It is the first Nonlinear activation functions for layer cross-networks; The output of the last cross-network layer is passed through a fully connected layer for feature mapping to obtain individualized effect prediction results for different treatment methods.
5. The method according to claim 4, characterized in that, The step of inputting the fused feature set into the trained deep full-space cross-network optimization model for analysis and processing further includes: in each layer of the cross-network, calculating the attention weight of each pair of feature interactions in the fused feature set, satisfying the following formula: ； in, To represent features and Attention weights in interactions This represents a learnable weight vector. This represents the transpose of the learnable weight vector. This represents element-wise multiplication.
6. The method according to claim 4, characterized in that, The step of inputting the fused feature set into the trained deep full-space cross-network optimization model for analysis and processing also includes: The first The output of the layer cross-connect network and the output of the layer above cross-connect network Feature interaction is performed, satisfying the following formula: ； in, Indicates the first The output of the layer, Indicates the first The output of the layer, This represents the weight matrix of the layer. For bias terms, It is a non-linear activation function.
7. The method according to claim 1, characterized in that, The training process of the deep full-space cross-network optimization model includes: The sample weights are determined based on the predicted and actual values of each sample. The cross-entropy loss is then weighted according to the sample weights to obtain the weighted cross-entropy loss. Construct the target loss function based on the weighted cross-entropy loss and L2 regularization loss; The deep full-space cross-network optimization model is trained using a preset optimizer based on the objective loss function, and the hyperparameters are optimized through cross-validation to obtain the trained deep full-space cross-network optimization model.