Reinforced learning optimization channel weight double-attribution advertisement effect analysis system and method

Abstract

The invention belongs to the technical field of advertisement media, and particularly relates to a reinforcement learning optimization channel weight double-attribution advertisement effect analysis system and method, and the method comprises the steps: collecting multi-source heterogeneous data, and carrying out the preprocessing, so as to obtain standardized data; a reinforcement learning algorithm is designed, weights are distributed according to dynamic time sequence attention, a double-weight coordination action space is constructed, and a multi-modal coding state and action requirements are obtained; multi-modal data is coded, text, time sequence and structured data are coded respectively, and feature vectors matched with the model are output through modal attention fusion; designing a reward function, and generating a reward signal for real-time optimization; and a real-time optimization mechanism is constructed, parameters are updated in a layered manner, gray scale verification is performed, multi-source heterogeneous data preprocessing and enhancement algorithm design are promoted, and a closed loop is formed. According to the method, the effectiveness and accuracy of advertisement effect analysis are effectively improved, and the adaptability to multi-source heterogeneous data and dynamic scenes is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of advertising and media technology, and in particular relates to a reinforcement learning-based dual-attribution advertising effectiveness analysis system and method for optimizing channel weights. Background Technology

[0002] Digital marketing has entered a multi-channel integration phase. While the "social media + search + short video + e-commerce" campaign model has expanded its coverage, the core bottleneck is the challenge of attributing channel weights—quantifying the true contribution of each channel to conversion directly affects budget efficiency and ROI.

[0003] Traditional attribution falls into two categories: single-touchpoint attribution (e.g., last / first click) attributes only to a single touchpoint, ignoring the value of secondary channels; multi-touchpoint attribution (e.g., linear / time decay) distributes credit equally according to fixed rules, failing to identify key channels. To address this, dual attribution technology emerged, constructing a two-dimensional system through "key touchpoint identification (determining key weights) + global contribution allocation (determining secondary weights)." However, existing technologies suffer from architectural limitations, making them difficult to adapt to dynamic delivery requirements. The main shortcomings of existing dual attribution technologies are: (1) The weight allocation is subjective and lacks dynamic adjustment. Relying on expert experience (such as the Analytic Hierarchy Process (AHP) and the Delphi method) and fixed proportions (such as 60% for key weights and 40% for overall weights) presents two major problems: First, there is significant subjective bias, with different experts scoring the importance of channels differently, resulting in unstable weighting results; second, updates are lagging, and when channel conversion efficiency changes abruptly, the weights cannot be adjusted in time, leading to budget misallocation and low attribution accuracy.

[0004] (2) Weak data processing capabilities, unable to adapt to heterogeneous data Multi-channel data is characterized by heterogeneity (structured / semi-structured / time-series), sparseness, and time-series characteristics, which traditional statistical methods (linear / logistic regression) are inadequate for processing: heterogeneous data needs to be standardized, and time-series correlations are lost (such as the conversion trend after users continuously watch short videos); sparse data (such as long-tail channels) cannot support model calculations, and its auxiliary value is ignored.

[0005] (3) Insufficient real-time performance, making it difficult to match dynamic environments. Advertising placement is influenced in real time by user preferences, competitor strategies, and market events, but current technology uses "offline batch processing," with weight updates occurring every 1-2 weeks. During major e-commerce promotions, channel conversion efficiency fluctuates wildly, and lagging weights cannot be updated in time, leading to missed conversion peaks or wasted budgets.

[0006] In summary, the core deficiency of existing dual attribution methods lies in the lack of capabilities for "dynamic interaction, sequence modeling, and multi-objective optimization." Reinforcement learning (RL), through real-time interaction between the agent, environment, and reward, can dynamically adjust weights; through temporal decision modeling, it captures behavioral sequence correlations and, combined with a custom reward function, effectively improves the real-time performance of attribution. Therefore, introducing RL becomes a key path to address existing deficiencies and meet the needs of multi-channel real-time optimization.

[0007] Existing technology CN120634641A discloses an online advertising analysis method and system based on artificial intelligence. By setting up an advertising set and its standard records, it clarifies the various elements of advertising data collection, solving the problems of inconsistent data sources and data chaos. The advertising set includes text descriptions, image data, video data, and display timestamps. Finally, the numerical values ​​of text, image, and video data are summed to obtain the comprehensive advertising description value, achieving unified processing of multimodal data and improving data integration and cross-modal analysis. However, this invention's intelligent solution only achieves unified processing of data from different modalities and does not address the problem of insufficient dynamic adaptation capability in existing fixed attribution methods based on expert prior experience, resulting in relatively low attribution accuracy. The prior art CN119151609A discloses a method, device, and medium for advertising effectiveness analysis based on standardized data processing. This includes creating a unified media cost table, tracking data table, media ID dimension table, and media business relationship table; collecting data and recording it in the corresponding tables; storing the media cost table, tracking data table, media ID dimension table, and media business relationship table in a MySQL database; obtaining user access log data and storing it in a local file; obtaining service log data and storing it in a server-side program; writing data from the MySQL database, local file, and server-side program into a real-time streaming Kafka message queue; storing the data in the Kafka message queue in real-time into ClickHouse; generating BI reports and performing statistical analysis based on the data in ClickHouse; and analyzing the advertising effectiveness based on the statistical results of the BI reports. While this invention can achieve real-time tracking of advertising effectiveness, it relies on manual data analysis, depends on expert experience, has a low level of intelligence, weak data processing capabilities, and cannot adapt to heterogeneous data. Summary of the Invention

[0008] To address the aforementioned problems, this invention proposes a reinforcement learning-based dual-attribution advertising effectiveness analysis system and method for optimizing channel weights. By constructing an agent-environment-reward system, it achieves real-time interaction with the environment and autonomously and dynamically adjusts weights. Simultaneously, by combining temporal decision modeling and designing a custom reward function, it achieves efficient and accurate dual-attribution advertising effectiveness analysis based on channel weights. To implement the above solution, this invention provides the following technical solution: The reinforcement learning-based dual-attribution advertising effectiveness analysis method for optimizing channel weights includes the following steps: S1: Multi-source heterogeneous data collection and preprocessing, classifying and acquiring advertising placement, user behavior, and channel feedback data, and obtaining standardized data through format normalization, noise filtering, and spatiotemporal alignment; S2: Construct a reinforcement learning algorithm, building a dynamic temporal attention deep temporal algorithm based on the preprocessed data from step S1. Q The DTA-DTQN network allocates weights based on dynamic temporal attention, constructs a dual-weighted collaborative action space, and obtains multimodal encoding states and action requirements. S3: Multimodal data encoding. Based on the multimodal encoding state and action requirements obtained in step S2, text, time series, and structured data are encoded respectively. Through modal attention fusion, feature vectors adapted to DTA-DTQN are output. S4: Design a reward function, calculate short-term reward, long-term reward and cost reward based on the feature vector encoded in step S3, and then add a causal inference bias correction term to generate a reward signal for real-time optimization; S5: Construct a real-time optimization mechanism, based on the reward signal generated in step S4 and its incremental update model, update parameters in layers and perform grayscale verification.

[0009] Further, step S1 includes: S10: Collect data from the advertising delivery end and pull channel operation data in real time through the channel open API; S11: Collect user behavior data by using the event tracking SDK, local caching, and batch upload strategies to collect user behavior interaction data. S12: Collect data from channel feedback, obtain unstructured text data through compliant and authorized crawling tools or channel feedback interfaces, and process it in two steps: real-time crawling and offline parsing. S13: Data format normalization converts structured data into JSON format, unstructured data into UTF-8 encoded plain text data format, and time series data is normalized according to timestamp and numerical key-value pair format to eliminate format heterogeneity; S14: Filter noise by using a dynamic threshold method to filter out outliers and remove data that exceeds the normal range; for data with missing values, structured data is filled with the average value of the same channel during the same period, and time series data is filled with linear interpolation. S15: Spatiotemporal data alignment, aligning all data to time granularity according to a unified time scale, and associating them in a two-dimensional relationship based on user ID and channel ID; S16: Perform Z-Score normalization on the structured data after spatiotemporal alignment in step S15.

[0010] Further, step S2 includes: S20: Construct DTA-DTQN, with the channel weight optimizer as the decision-making body and the optimization goal being to adjust the dual attribution weights to maximize the campaign ROI; S21: Define channel conversion rate and channel volatility; S22: Construct time-series feature fusion logic based on the channel conversion rate and channel volatility defined in step S21, including calculating the channel time-series contribution, allocating dynamic attention coefficients, and fusing time-series state vectors; S23: Extract user-dimensional features from the user behavior data collected in step S11, and perform vector encoding through the embedding layer of DTA-DTQN; S24: Extract channel-dimensional features from the advertising delivery data collected in step S10, and form a multi-dimensional encoding vector through batch normalization and a fully connected layer; S25: Extract scene dimension features from the channel feedback data collected in step S12, and generate multi-dimensional vectors through One-Hot encoding and cross-feature processing; S26: Three-dimensional feature fusion. The features extracted in steps S23 to S25 are weighted and fused to calculate the influence of the transformation, and the final multi-dimensional state vector is output.

[0011] Furthermore, step S2 also includes: S27: Define the action dimension; the actions for each channel include key weight adjustments. Global weight adjustment and Coefficient of Synergy Three dimensions; S28: Define dual-weighted collaborative constraints and set collaborative coefficients according to channel type. Establish constraint relationships ; S29: Adaptive control of action amplitude, setting the maximum adjustment amplitude based on the channel volatility defined in step S21.

[0012] Further, step S3 includes: S30: Text data encoding, constructing a lightweight BERT neural network model DistilBERT, using a public corpus of advertising industry to perform secondary training on DistilBERT, and adjusting the word embedding layer parameters; adding a brand relevance prediction branch to the output layer of the DistilBERT model, sharing the encoder with the sentiment classification task, inputting unstructured text data from channel feedback, and outputting the sentiment tendency and brand relevance of the text, and encoding it into a multi-dimensional vector; S32: Time series data encoding. The time series data of user behavior is sorted by timestamp and then input into Bi-LSTM. A time attention layer is added to Bi-LSTM to calculate the time distance weight between the hidden state and the conversion event at each time step. The weighted data is then output as a multi-dimensional vector. S33: Cross-feature enhancement realizes the structured data encoding of the advertising delivery end. The original structured data is Z-Score normalized as described in step S16 to obtain the basic feature vector; the cross-feature capture collaborative information feature vector of conversion efficiency-cost and exposure-channel type is designed; after concatenating the basic feature vector and the cross-feature, a multi-dimensional vector is output through a fully connected layer. S34: The multidimensional vectors obtained in steps S30 to S33 are fused through a modal attention mechanism and aligned with the multidimensional state vector format output in step S26 before being directly used as the input of DTA-DTQN.

[0013] Further, step S4 includes: S40: Design a three-dimensional basic reward framework based on short-term, long-term, and cost, calculating short-term conversion rewards, long-term revenue rewards, and cost control rewards; short-term conversion rewards are calculated based on the current hourly channel conversion rate, and long-term revenue rewards are calculated based on... m Intraday LTV forecast calculation; cost control rewards are calculated based on single conversion cost. m These are variables that can be determined independently; S41: Calculate the true contribution weight using a dual / inverse probability weighted causal model, taking channel reach as the processing variable, user conversion as the outcome variable, and user profile and historical behavior as confounding variables to calculate the true contribution weight of each channel. S42: Design a deviation correction term, which forces the algorithm to adjust the weights to be closer to the true contribution weights calculated in step S41 by penalizing the deviation between the output weights and the true contribution weights. S43: Total reward fusion, which weights and fuses the three-dimensional basic reward framework calculated in step S40 with the deviation correction term obtained in step S42 to obtain the total reward function.

[0014] Further, step S5 includes: S50: Collect state-action-reward-next state samples according to a fixed sampling period and store them in an experience replay pool with a lifecycle; assign high priority to high reward samples and high bias samples, and extract them according to priority weights during sampling. S51: Based on the real-time updated sampling data from step S50, freeze the parameters of the DistilBERT and Bi-LSTM neural network models, and only update the parameters of the DTA-DTQN model. Q The network and attention layer parameters are configured using the objective function of minimizing temporal difference error. S52: Gray-scale verification. Apply the updated model from step S51 to a portion of the advertising traffic data and compare its performance with the model before the update. If the performance meets the standard, deploy it to the entire system; if it does not meet the standard, return to the model before the update and recalibrate the reward function weights.

[0015] On the other hand, a reinforcement learning-optimized channel weight dual-attribution advertising effectiveness analysis system is provided, applied to any of the reinforcement learning-optimized channel weight dual-attribution advertising effectiveness analysis methods described above. The reinforcement learning-optimized channel weight dual-attribution advertising effectiveness analysis system includes: The data acquisition and processing module consists of an industrial-grade data server and a multi-terminal tracking SDK. It acquires multi-source data such as advertising placement, user behavior, and channel feedback. After format normalization, noise filtering, and spatiotemporal alignment preprocessing, it outputs standardized data. The enhanced computing power support module includes high-performance GPU servers, providing the computing power support required for neural network model deployment, text and time-series data encoding, and feature vector generation. The performance output feedback module includes a cloud server and a visualization terminal, which outputs the weight adjustment results and the campaign performance report, while collecting real-time performance data.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. By using Dynamic Temporal Attention (DTQN) and a dual-weight collaborative mechanism, we solve the problem of insufficient dynamic adaptation in traditional dual-attribution, and improve the accuracy of weight adjustment and attribution by combining causal correction rewards.

[0017] 2. The core algorithm connects multi-source data preprocessing and multimodal coding modules, with layered incremental updates ensuring real-time performance, supporting method implementation, and achieving a dynamic balance between advertising effectiveness and cost. Attached Figure Description

[0018] Figure 1 This is a flowchart of the reinforcement learning-based optimization method for dual-attribution advertising effectiveness analysis.

[0019] Figure 2 This is a structural diagram of the reinforcement learning-optimized channel weight dual-attribution advertising effectiveness analysis system of the present invention.

[0020] Figure 3 This is a structural diagram of the DTA-DTQN network model of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0023] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0024] In this embodiment of the invention, a reinforcement learning-based method for optimizing channel weights and performing dual-attribution advertising effectiveness analysis is described; see appendix for details. Figure 1 Specifically, it includes: S1: Multi-source heterogeneous data collection and preprocessing, classifying and acquiring advertising placement, user behavior, and channel feedback data, and obtaining standardized data through format normalization, noise filtering, and spatiotemporal alignment; S2: Design a reinforcement learning algorithm to construct a dynamic temporal attention deep temporal learning algorithm based on the preprocessed data in step S1. Q The DTA-DTQN network allocates weights based on dynamic temporal attention, constructs a dual-weighted collaborative action space, and obtains multimodal encoding states and action requirements. S3: Multimodal data encoding. Based on the multimodal encoding state and action requirements obtained in step S2, text, time series, and structured data are encoded respectively. Through modal attention fusion, feature vectors adapted to DTA-DTQN are output. S4: Design a reward function, calculate short-term reward, long-term reward and cost reward based on the feature vector encoded in step S3, and then add a causal inference bias correction term to generate a reward signal for real-time optimization; S5: Construct a real-time optimization mechanism. Based on the reward signal generated in step S4 and its incremental update model, update parameters hierarchically and perform grayscale verification. Optimize the preprocessing and reinforcement algorithm design of multi-source heterogeneous data in reverse to form a closed loop.

[0025] See attached document Figure 2 The present invention also provides a reinforcement learning-optimized channel weight dual-attribution advertising effect analysis system, including a data acquisition and processing module, a reinforcement computing power support module, and an effect output feedback module.

[0026] The data acquisition and processing module consists of an industrial-grade data server and a multi-terminal tracking SDK. It acquires multi-source data such as advertising placement, user behavior, and channel feedback. After format normalization, noise filtering, and spatiotemporal alignment preprocessing, it outputs standardized data. The enhanced computing power support module includes high-performance GPU servers, providing the computing power support required for neural network model deployment, text and time-series data encoding, and feature vector generation; The performance output feedback module includes a cloud server and a visualization terminal, outputting weight adjustment results and campaign performance reports, while also collecting real-time performance data.

[0027] In a specific embodiment, to address the issues of subjective weight allocation, lack of dynamic adjustment capability, difficulty in adapting to heterogeneous data, and insufficient real-time performance in existing advertising effectiveness analysis methods, advertising effectiveness analysis engineers propose a reinforcement learning-optimized channel weight dual-attribution advertising effectiveness analysis method. First, multi-source heterogeneous data from ad placement, user behavior, and channel feedback are categorized and acquired, and standardized data is obtained after preprocessing. Second, a reinforcement learning algorithm is designed to construct a dynamic temporal attention deep temporal series... Q The network DTA-DTQN is constructed, and weights are allocated according to dynamic temporal attention to build a dual-weight collaborative action space to obtain multimodal encoding states and action requirements. Then, based on the multimodal encoding states and action requirements, text, temporal, and structured data are encoded respectively, and feature vectors adapted to DTA-DTQN are output through modal attention fusion. Next, a reward function is designed to calculate short-term, long-term, and cost rewards respectively, and a causal inference bias correction term is superimposed to generate a reward signal for real-time optimization. Finally, a real-time optimization mechanism is constructed, which updates the model incrementally based on the generated reward signal, updates parameters hierarchically and performs grayscale verification, and reverse-optimizes the preprocessing and reinforcement algorithm design of multi-source heterogeneous data to form a closed loop.

[0028] Example 1 In one embodiment, as shown in the appendix Figure 1As shown, to address the problem of current advertising effectiveness analysis methods being unable to adapt to heterogeneous data, advertising effectiveness analysis engineers propose a multi-source heterogeneous data collection and preprocessing method, specifically including: S10: Collect data from the advertising platform and pull channel operation data in real time through channel open APIs, such as the marketing API of short video platforms and the backend API of search ads. This includes the advertising budget, impressions and clicks of each channel. The sampling frequency is set to 1 minute / time, and the latency is controlled within 50ms. S11: Collect user behavior data. Collect user behavior interaction data through the event tracking SDK, local caching, and batch upload strategies, including user channel reach path, dwell time, and conversion behavior data. S12: Collect data from channel feedback terminals. Obtain unstructured text data, including user comments, content likes and reposts, and channel quality score data, through compliant and authorized crawling tools or channel feedback interfaces. The data is then processed in two steps: real-time crawling and offline parsing. S13: Data format normalization converts structured data into JSON format, unstructured data into UTF-8 encoded plain text data format, and time series data is normalized according to timestamp and numerical key-value pair format to eliminate format heterogeneity; S14: Noise filtering. A dynamic threshold method is used to filter out outliers. The mean and standard deviation are calculated based on the data distribution over the past hour, and values ​​exceeding the normal range are removed. The data, among which This is the average of data from the past hour. This represents the standard deviation of data from the past hour; this normal range was determined by combining statistical principles with the actual fluctuation patterns of advertising data. Statistically, within data that conforms to a normal distribution, It can cover approximately 95.4% of normal data while avoiding... An overly wide range can cause minor outliers to go unfiltered. An excessively narrow timeframe can cause normal fluctuations in advertising revenue to be misjudged as anomalies. At the advertising data level, the main source of abnormal noise is systematic noise, which typically deviates from the mean. above, It can effectively filter data; for data with missing values, structured data is filled with the average value of the same channel and the same period, and time series data is filled with linear interpolation. S15: Spatiotemporal data alignment, aligning all data to hourly time granularity and associating them by a dual dimension of user ID and channel ID; S16: Perform Z-Score normalization on the structured data after spatiotemporal alignment in step S15 to eliminate dimensional differences. The specific calculation formula is as follows:

[0029] in, z These are the normalized eigenvalues. x This is the global mean of the structured feature. The global mean of this structured feature. This is the global standard deviation of the structured feature.

[0030] Example 2 In one embodiment, as shown in the appendix Figure 1 As shown, to address the issues of subjective weight allocation and lack of dynamic adjustment in current advertising effectiveness analysis methods, advertising effectiveness analysis engineers designed a reinforcement learning algorithm, specifically including: S20: Constructing Dynamic Temporal Attention Deep Temporal Series Q The structure of the DTA-DTQN network is shown in the attached figure. Figure 3 As shown, the decision-making step size is set to 1 hour, the decision-making body is the channel weight optimizer, and the optimization goal is to adjust the two attribution weights, key weight and global weight, to maximize the ROI of the campaign. S21: Define Channel Conversion Rate (CR). Define it as a channel... t The ratio of users converted at any given moment to users reached is calculated from user behavior data after preprocessing from multiple sources, and is denoted as follows: ,in i Channel number; define channel volatility (Vol), which is the standard deviation of a channel's conversion rate over the past 6 hours, reflecting the stability of performance. The calculation expression is:

[0031] in, This represents the average conversion rate over the past 6 hours. S22: Construct time-series feature fusion logic based on the channel conversion rate and channel volatility defined in step S21, including calculating the channel time-series contribution, allocating dynamic attention coefficients, and fusing time-series state vectors; Channel time-series contribution rate over the past 12 hours, statistically analyzed for each hour. k Total conversion contribution across all channels The calculation expression is:

[0032] The data comes from pre-processed ad placement and user behavior data; Attention weight is allocated per hour based on contribution percentage. The calculation expression is:

[0033] in, k =0 corresponds to t time, k=11 corresponds to t -11 o'clock, ensure that high-contribution periods receive more attention; The underlying state vector, weighted by the attention coefficient, has been used for nearly 12 hours. Output the final timing state The calculation expression is:

[0034] S23: Extract user-dimensional features. From the user behavior data collected in step S11, extract user profiles (age, spending power), historical conversion frequency, and high-frequency contact channel features in the past 3 days. Encode these features into a 128-dimensional vector using the DTA-DTQN embedding layer. ; S24: Extract channel-dimensional features, and extract channel conversion rate from the advertising delivery data collected in step S10. The features of cost per conversion (CPC) and inter-channel collaborative conversion efficiency (the conversion rate of a channel reaching other channels together) are batch normalized and encoded into a 64-dimensional vector through a fully connected layer. ; S25: Extract scenario-dimensional features. Extract the ad placement time (weekday / weekend), industry promotional periods, and competitor ad placement share from the channel feedback data collected in step S12. Generate a 32-dimensional vector through One-Hot encoding and time-promotion cross-feature processing. ; S26: 3D Feature Fusion. The 3D features extracted in steps S23 to S25 are weighted and fused according to their influence on the transformation. The weight values ​​for the 3D features are as follows: Output a 224-dimensional state vector The calculation expression is:

[0035] S27: Define action dimensions for each channel i Actions include key weight adjustments Global weight adjustment and Coefficient of Synergy Three dimensions, the overall action space is , n Number of channels; S28: Define dual-weighted collaborative constraints and set collaborative coefficients according to channel type. ; Value of conversion-triggered channel coefficient =0.3, the coefficient value for auxiliary reach channels =0.7, establish constraint relationship ; S29: Adaptive control of motion amplitude, based on the channel volatility defined in step S21. Set the maximum adjustment range To ensure operational safety, considering that 89% of normal advertising channel fluctuations are concentrated between 0.05 and 0.15 under normal advertising conditions, and only 7% of channels have fluctuations > 0.15, 0.15 was chosen as the cutoff point: like ≥0.15 indicates high volatility; take... =0.05, make small adjustments; like ≤0.15, this is considered low volatility, take =0.15, a significant optimization.

[0036] Example 3 In one embodiment, as shown in the appendix Figure 1 As shown, to address the problems of current advertising effectiveness analysis methods, such as difficulty in adapting to heterogeneous data, low analytical accuracy, and reliance on prior human experience, advertising effectiveness analysis engineers have designed multimodal data coding, specifically including: S30: Text data encoding, constructing a lightweight BERT neural network model (DistilBERT), and using a public advertising industry corpus to pre-train DistilBERT. This public advertising industry corpus contains 100,000 advertising reviews and 50,000 brand promotional texts. The public advertising industry corpus is divided into training set, test set and validation set in a 7:2:1 ratio. DistilBERT is continuously trained on the training set, with pre-trained model weights loaded. The AdamW optimizer is used, with the learning rate set to 5e-5 and the number of iterations set to 10. At the same time, the word embedding layer parameters were adjusted, the top-level parameters of the DistilBERT encoder were fixed, and only the word embedding layer and the next two encoder layers were unfrozen. The MLM task and NSP task consistent with the basic pre-training were adopted to enable the model to recognize industry terms such as "planting grass" and "conversion" and improve the semantic understanding accuracy. A brand association prediction branch is added to the output layer of the DistilBERT model, sharing the encoder with the sentiment classification task. It takes unstructured text data from channel feedback as input and outputs the sentiment tendency and brand association of the text, encoded as a 4-dimensional vector. The sentiment of the text is divided into positive and negative, which is a 3-dimensional vector; the brand relevance value is 0 to 1, which is a 1-dimensional vector. S32: Time series data encoding. The user behavior time series and viewing duration series data are sorted by timestamp and then input into a bidirectional long short-term memory network (Bi-LSTM). The forward LSTM captures the trend of continuous 3 hours of access, and the backward LSTM captures the prediction of the conversion probability 1 hour after the current access. The output is a 256-dimensional hidden state. A temporal attention layer is added to the Bi-LSTM to calculate the temporal distance weight between the hidden state and the conversion event at each time step. Based on the reach-conversion contribution statistics of 100,000 user conversion paths in the advertising industry, the weight is set to 0.3 for the hour before conversion and 0.05 for the six hours before conversion. The weighted output is a 64-dimensional vector. The logical expression for weight calculation is:

[0037] in, For the first j Time distance weights for each time step This is the weight decay coefficient. t j For the first j The timestamp of each time step For the timestamp of the transformation event, m This represents the total number of time steps in the time series data. S33: Cross-feature enhancement enables structured data encoding for ad delivery. The ad delivery impressions and CPC are normalized using the Z-Score method described in step S16 to obtain basic feature vectors. Cross-features of conversion efficiency-cost and impressions-channel type are designed and captured using the Hadamard product to extract collaborative information feature vectors. The Hadamard product formula is as follows:

[0038] in, C For cross feature vectors, A , B For two basic feature vectors with the same dimension, This is the Hadamard product operator; After concatenating the basic feature vector with the cross features, the output is a 32-dimensional vector after passing through two fully connected layers and a ReLU activation function. ; S34: The multidimensional vectors obtained in steps S30 to S33 are fused using a modal attention mechanism. The contribution of each multidimensional vector to the state objective is calculated, and fusion weights are assigned according to the contribution. A 224-dimensional vector is output, which is then aligned with the multidimensional state vector format output in step S26 and directly used as the input to DTA-DTQN. The expression for calculating the fusion weights is as follows:

[0039] in, For the fusion weights of a certain modality, M Modal type, T The state objective is the transformation and prediction objective. For modality M With the goal T The similarity.

[0040] Example 4 In one embodiment, as shown in the appendix Figure 1 As shown, to address the problems of current advertising effectiveness analysis methods, such as difficulty in adapting to heterogeneous data, low analytical accuracy, and reliance on prior human experience, advertising effectiveness analysis engineers have designed a novel reward function, which specifically includes: S40: Design a three-dimensional basic reward framework of short-term-long-term-cost, calculate short-term conversion reward, long-term revenue reward and cost control reward, and determine the value range of each reward based on the numerical relationship between the three rewards. Short-term conversion rewards are calculated based on the channel conversion rate for the current hour, and the calculation formula is as follows:

[0041] in, This is a short-term conversion reward, with a value range of [0, 10]. After weight adjustment t Conversion rate at time +1 CR t Before weight adjustment t The conversion rate at any given time; if the conversion rate increases after adjustment, the reward is positive and positively correlated with the increase; if it decreases, the reward is 0, guiding the algorithm to prioritize short-term results; Long-term revenue rewards are calculated based on the predicted LTV over 30 days. The XGBoost model is used to predict the adjusted LTV, and features include user reach channels and conversion frequency. The calculation expression is as follows:

[0042] in, This is a long-term reward, with a value range of [0, 8]. After weight adjustment t The 30-day customer lifetime value forecast at time +1 LTV t Before weight adjustment t 30-day customer lifetime value at any given moment; Cost control rewards are based on the cost per conversion. CPC Calculation, introducing the cost elasticity coefficient High-cost channels are set based on the differences in the cost attributes of the channels themselves. Low-cost channels The calculation expression is:

[0043] in, This is a cost control incentive, with a value range of [0,5]. After weight adjustment t The cost of a single conversion at time +1 Before weight adjustment t The cost per conversion at any given moment; S41: Calculate the true contribution weight using a dual / inverse probability weighted (IPW) causal model, with channel reach as the processing variable, user conversion as the outcome variable, and user profile (age, spending power) and historical behavior (number of channel reach in the past 7 days, historical conversion records) as confounding variables to calculate the true contribution weight of each channel. The core calculation expression for IPW is:

[0044] in, The average processing effect reached through the channel, i.e., the actual contribution. N Total number of users; For user conversion results, =1 indicates that it has been converted. =0 indicates no conversion; For users j Channel reach identifiers =1 indicates that the target has been reached. =0 indicates no reach; For users j Confounding variables include user profiles and historical behavior; For users j The predicted probability of being reached by the channel; based on Allocate the weight of actual contributions from each channel:

[0045] in, For the first i Average processing efficiency of the channel; In a certain advertising scenario, select N =5 user samples, with confounding variables being historical conversion count and spending power divided into three levels (1-3). The actual contribution (ATE) of channel A is calculated using the IPW formula. The user-specific calculation items are as follows: User 1: ; User 2: ; User 3: ; User 4: ; User 5: ; Summing yields: 1.25 + 0 + 0 - 1.25 + 1.43 = 1.43; Calculating ATE yields: That is, the actual contribution of channel A is approximately 28.6%.

[0046] If both Channel A and Channel B are used simultaneously, and the actual contribution of Channel A is 0.286 while that of Channel B is 0.214, then: Channel A's actual contribution weight: ; Channel A's actual contribution weight: S42: Design a deviation correction term. By penalizing the deviation between the output weight and the actual contribution weight, the algorithm is forced to adjust the weights to align with the actual contribution weight calculated in step S41. The expression for the deviation correction term is:

[0047] in, As the attribution bias correction term, the calculation expression shows that its value range is [-2, 0]; n This represents the total number of current distribution channels. For the first i channel t The model output weights at time 1. For the first i channel t The weight of the actual contribution at any given moment; This represents the sum of weighted deviations across all channels. The smaller the deviation between the model's output weights and the true weights... The closer to 0, the weaker the penalty; the greater the deviation, The closer to -2, the stronger the penalty; S43: Total Reward Fusion. This involves a weighted fusion of the three-dimensional basic reward framework calculated in step S40 and the bias correction term obtained in step S42 to achieve a multi-objective balance between "effectiveness, cost, and attribution accuracy." The calculation formula is as follows:

[0048] in, for t Total reward at +1 moment These are the weighting coefficients for short-term, long-term, and cost-reward activities, respectively.

[0049] Example 5 In one embodiment, as shown in the appendix Figure 1 As shown, to address the shortcomings of current advertising performance analysis methods in terms of real-time performance and difficulty in adapting to dynamic environments, advertising performance analysis engineers have designed a real-time optimization mechanism, which specifically includes: S50: Collect status - action - reward - next status every hour. The samples are stored in an experience replay pool with a lifecycle, and the samples are retained for 48 hours to avoid interference from old samples. right High reward samples >1 and High-biased samples are given high priority, and sampling is performed according to priority weights to ensure that the algorithm focuses on learning effective samples. The priority weight calculation formula is as follows:

[0050] in, For the sample The sampling probability; For the sample Priority scoring, with higher-reward / high-bias samples receiving higher scores; For smoothing terms; This refers to the priority weighting coefficient; This is the sum of the priorities of all samples in the replay pool; After sampling, the sample weights are corrected through importance sampling to avoid sampling bias affecting model updates.

[0051] in, Weights are assigned to samples based on their importance. N The total number of samples in the replay pool. The sampling probability of the sample; S51: Based on the real-time updated sampling data from step S50, freeze the parameters of the DistilBERT and Bi-LSTM neural network models, and only update the parameters of the DTA-DTQN model. Q The network and attention layer parameters are updated using the objective function of minimizing temporal difference (TD) error, with the following expression:

[0052] in, for t +1 moment Q Network parameters, for t time Q Network parameters; For learning rate, To Q Network parameters Find the gradient. B =256 represents the batch sample set. For parameter-based status s ,action a of Q value, r For the sample; Dynamic discount factor, high conversion cycle products =0.95, Fast-moving consumer goods =0.85; Based on target network parameters status ,action Q value; The target network parameters adopt a soft update strategy to avoid parameter mutations caused by hard updates.

[0053] in, For the updated target Q Network parameters, This is the soft update coefficient. For the present Q Network parameters, These are the retention coefficients for the target network parameters; An update is triggered when the sample pool accumulates 1,000 valid samples. The time taken for a single update is controlled within 3 minutes, ensuring that at least one update is completed every hour to meet real-time requirements. S52: Gray-scale verification. Apply the updated model from step S51 to 20% of the ad delivery traffic data and compare the performance of CR, CPC, and LTV metrics with the model before the update. If CR increases by ≥10%, CPC decreases by ≥8%, and LTV increases by ≥12%, then deploy the model to the full extent. If the targets are not met, return to the model before the update and recalibrate the reward function weights.

[0054] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0055] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for analyzing the effect of double-attribution advertising optimized by reinforcement learning of channel weights, characterized by, Comprise the following steps: S1: multi-source heterogeneous data collection and preprocessing, classified acquisition of advertising, user behavior, channel feedback data, after format normalization, noise filtering, space-time alignment processing, get standardized data; S2: constructing a reinforcement learning algorithm, constructing a dynamic timing attention deep timing based on the preprocessed data of step S1 Q network DTA-DTQN, according to the dynamic timing attention weight distribution, construct a double weight collaborative action space, obtain multi-modal encoding state and action demand; S3: multi-modal data encoding, based on the multi-modal encoding state and action demand obtained in step S2, encode text, time series, structured data respectively, through modal attention fusion, output the feature vector suitable for DTA-DTQN; S4: design reward function, calculate short-term reward, long-term reward and cost reward according to the feature vector coded in step S3, then superimpose the causal inference bias correction term to generate the reward signal for real-time optimization; S5: build real-time optimization mechanism, update the model according to the reward signal generated in step S4 and its increment, update the parameters layer by layer and verify by gray scale. 2.The method of claim 1, wherein, The step S1 comprises: S10: collect advertising end data, pull channel operation data in real time through channel open API; S11: collect user behavior end data, collect user behavior interaction data through the method of embedding point SDK, local cache and batch upload strategy; S12: collect channel feedback end data, obtain unstructured text data through the crawler tool or channel feedback interface after compliance authorization, and adopt the method of real-time grabbing-offline analysis; S13: data format normalization, convert structured data into JSON format, convert unstructured data into pure text data format with UTF-8 encoding, and format time series data according to time stamp and numerical key value pair; S14: filter noise, filter abnormal values by dynamic threshold method, and remove data beyond the normal value range; for data with missing values, fill in the mean value of the same channel and the same period for structured data, and fill in the linear interpolation for time series data; S15: space-time data alignment, align the time granularity of all data according to the unified time scale, and associate them according to user ID-channel ID; S16: Z-Score normalization is performed on the structured data after space-time alignment in step S15. 3.The method of claim 1, wherein, The step S2 comprises: S20: build DTA-DTQN, the decision subject is channel weight optimizer, and the optimization goal is to adjust the double attribution weight to maximize the ROI of advertising; S21: define channel conversion rate and channel fluctuation degree; S22: build time series feature fusion logic based on the channel conversion rate and channel fluctuation degree defined in step S21, including calculating channel time series contribution degree, assigning dynamic attention coefficient and fusing time series state vector; S23: extract user dimension features from the user end behavior data collected in step S11, and perform vector coding through the Embedding layer of DTA-DTQN; S24: extract channel dimension features from the advertising end data collected in step S10, form multi-dimensional coding vector through batch normalization and full connection layer; S25: extract scene dimension features from the channel feedback end data collected in step S12, generate multi-dimensional vector through One-Hot encoding and cross feature processing; S26: three-dimensional feature fusion, weight and fuse the influence weight of the features extracted in steps S23 to S25 on conversion, and output the final multi-dimensional state vector. 4.The method of claim 1, wherein, Step S2 further includes: S27: define action dimension, each channel action includes key weight adjustment amount , global weight adjustment amount , and synergy coefficient three dimensions; S28: define double weight synergy constraint, set synergy coefficient according to channel type , establish constraint relationship ; S29: Adaptive control of action amplitude, setting the maximum adjustment amplitude based on the channel volatility defined in step S21.

5. The method of claim 1, wherein, Step S3 includes: S30: Text data encoding, constructing a lightweight BERT neural network model DistilBERT, using a public corpus of advertising industry to perform secondary training on DistilBERT, and adjusting the word embedding layer parameters; adding a brand relevance prediction branch to the output layer of the DistilBERT model, sharing the encoder with the sentiment classification task, inputting unstructured text data from channel feedback, and outputting the sentiment tendency and brand relevance of the text, and encoding it into a multi-dimensional vector; S32: Time series data encoding. The time series data of user behavior is sorted by timestamp and then input into Bi-LSTM. A time attention layer is added to Bi-LSTM to calculate the time distance weight between the hidden state and the conversion event at each time step. The weighted data is then output as a multi-dimensional vector. S33: Cross-feature enhancement realizes the structured data encoding of the advertising delivery end. The original structured data is Z-Score normalized as described in step S16 to obtain the basic feature vector; the cross-feature capture collaborative information feature vector of conversion efficiency-cost and exposure-channel type is designed; after concatenating the basic feature vector and the cross-feature, a multi-dimensional vector is output through a fully connected layer. S34: The multidimensional vectors obtained in steps S30 to S33 are fused through a modal attention mechanism and aligned with the multidimensional state vector format output in step S26 before being directly used as the input of DTA-DTQN.

6. The method of claim 1, wherein, Step S4 includes: S40: design a short-term-long-term-cost three-dimensional basic reward framework, calculate the short-term conversion reward, long-term benefit reward and cost control reward; the short-term conversion reward is calculated based on the channel conversion rate of the current hour, the long-term benefit reward is calculated based on the LTV prediction value of the day, and the cost control reward is calculated based on the single conversion cost m , which are self-determined variables; m ​ S41: Calculate the true contribution weight using a dual / inverse probability weighted causal model, taking channel reach as the processing variable, user conversion as the outcome variable, and user profile and historical behavior as confounding variables to calculate the true contribution weight of each channel. S42: Design deviation correction term, which penalizes the deviation between the output weight and the actual contribution weight; S43: Total reward fusion, which weights and fuses the three-dimensional basic reward framework calculated in step S40 with the deviation correction term obtained in step S42 to obtain the total reward function.

7. The method of claim 1, wherein, Step S5 includes: S50: Collect state-action-reward-next state samples according to a fixed sampling period and store them in an experience replay pool with a lifecycle; assign high priority to high reward samples and high bias samples, and extract them according to priority weights during sampling. S51: Freeze the DistilBERT, Bi-LSTM neural network model parameters based on the real-time updated sampling data in step S50, and only update the network and attention layer parameters of the DTA-DTQN using the time difference error minimization objective function. Q network and attention layer parameters, using a time difference error minimization objective function; S52: Gray-scale verification. Apply the updated model from step S51 to a portion of the advertising traffic data and compare its performance with the model before the update. If the performance meets the standard, deploy it to the entire system; if it does not meet the standard, return to the model before the update and recalibrate the reward function weights.

8. A system for reinforcement learning optimization of channel weight double attribution advertisement effect analysis, characterized in that, The reinforcement learning-based channel weight dual-attribution advertising effectiveness analysis method according to any one of claims 1-7, wherein the reinforcement learning-based channel weight dual-attribution advertising effectiveness analysis system comprises: The data acquisition and processing module consists of an industrial-grade data server and a multi-terminal tracking SDK. It acquires multi-source data such as advertising placement, user behavior, and channel feedback. After format normalization, noise filtering, and spatiotemporal alignment preprocessing, it outputs standardized data. The reinforcement computing power support module includes a large computing power GPU server, and provides required computing power support for neural network model running deployment, text and time series data coding, and feature vector generation; The effect output feedback module includes a cloud server and a visual terminal, outputs weight adjustment results and delivery effect reports, and collects real-time effect data.

Images