A multi-modal user behavior driven marketing reach opportunity decision method

Abstract

The application discloses a kind of multi-modal user behavior driven marketing touch opportunity decision-making methods, comprising the following steps: multi-modal behavior data acquisition step, behavior feature extraction step, modality weight distribution step, real-time state evaluation step, touch condition determination step, touch intensity calculation step, feedback updating step.The application can comprehensively understand the real-time state and intention of users from multiple dimensions by collecting the behavior data of users in multiple modalities such as text interaction, image browsing, voice interaction and conducting joint analysis, and overcomes the defect of insufficient information of single modal data;By constructing a dynamic weight distribution mechanism and introducing a real-time feedback loop, the contribution of each modality feature can be adjusted adaptively according to the changes in user behavior, making the judgment of touch opportunity more accurate and timely, effectively improving the conversion rate and user satisfaction of marketing touch.

Description

Technical Field

[0001] This invention relates to the field of digital marketing technology, and more specifically to a multimodal user behavior-driven marketing outreach timing decision-making method. Background Technology

[0002] With the widespread adoption of the internet and mobile internet, digital marketing has become a primary means for businesses to reach users and promote their products and services. In digital marketing practice, the timing of marketing outreach directly impacts users' willingness to accept marketing messages, click-through rates, and ultimately, sales conversion rates.

[0003] Traditional marketing outreach timing decisions mainly rely on fixed-time strategies (such as daily push notifications at fixed times) or simple rule-based judgments based on single-dimensional user behavior data (such as browsing history and click history). These methods lack a comprehensive capture and dynamic understanding of the multi-dimensional characteristics of user behavior.

[0004] In recent years, some technical solutions have begun to attempt to predict users' active or preferred time periods by collecting historical user behavior data. However, these solutions typically only utilize single-modal behavioral data (such as browsing behavior sequences) and do not fully consider the rich semantic information of user behavior across different modalities (such as text interaction, image browsing, and voice interaction), resulting in inaccurate characterization of users' real-time intentions and emotional states. Moreover, most existing methods use static rules or offline-trained models to determine the timing of outreach, making it difficult to dynamically adjust decisions based on real-time changes in user behavior. This prevents marketing outreach from being implemented at the appropriate time and with the appropriate intensity, thus affecting the effectiveness of marketing outreach and user experience. Summary of the Invention

[0005] To address this issue, the present invention provides a multimodal user behavior-driven marketing outreach timing decision-making method. This method solves the problem that most existing technologies rely on static rules or offline-trained models to determine outreach timing, making it difficult to dynamically adjust decisions based on real-time changes in user behavior. Consequently, marketing outreach cannot be implemented at the appropriate time and with the appropriate intensity, thus affecting the effectiveness of marketing outreach and user experience.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A multimodal user behavior-driven marketing outreach timing decision-making method includes the following steps:

[0008] The multimodal behavior data acquisition step involves collecting multimodal behavior data of the target user within a preset time window. The multimodal behavior data includes text interaction data, image browsing data, and voice interaction data. Each modal behavior data is preprocessed to obtain standardized text modal feature sequences, image modal feature sequences, and voice modal feature sequences.

[0009] The behavioral feature extraction step involves semantically encoding the text modal feature sequence to obtain a text behavioral feature vector, visually encoding the image modal feature sequence to obtain an image behavioral feature vector, acoustically encoding the speech modal feature sequence to obtain a speech behavioral feature vector, and concatenating the text behavioral feature vector, image behavioral feature vector, and speech behavioral feature vector according to the acquisition time into a multimodal joint feature vector.

[0010] The modal weight allocation step involves calculating the initial weight coefficient for each modality based on the proportion of data volume and data update frequency of each modality's behavioral data within the preset time window, and then calculating the weight adjustment factor for each modality according to the following formula:

[0011] W_i=α_i×(1+β×log(1+f_i / F))

[0012] Where W_i is the weight adjustment factor of the i-th mode, α_i is the initial weight coefficient of the i-th mode, β is the preset adjustment coefficient, f_i is the data update frequency of the i-th mode within the preset time window, and F is the average data update frequency of all modes.

[0013] The real-time state assessment step involves weighting and fusing the multimodal joint feature vector with the weight adjustment factors of each modality to generate a user real-time state representation vector. The user real-time state representation vector is then input into a pre-trained intent classification model, which outputs the user's current intent category and intent confidence.

[0014] The reach condition determination step involves obtaining a preset reach condition set, which includes multiple reach conditions and a reach timing threshold corresponding to each reach condition. It then determines whether the intent confidence level reaches the reach timing threshold. If it does, the reach condition is determined to be met at the current moment; otherwise, monitoring continues.

[0015] The reach intensity calculation step involves, when it is determined that the reach conditions are met at the current moment, calculating the reach intensity value based on the user's real-time state representation vector and the intent confidence, and performing marketing outreach according to the reach intensity value and the corresponding reach method.

[0016] The feedback update step involves collecting user feedback behavior data after the marketing outreach is executed, generating a feedback evaluation score based on the user feedback behavior data, and using the feedback evaluation score to update the parameters of the intent classification model and the outreach timing threshold in the outreach condition set.

[0017] Preferably, in the multimodal behavior data acquisition step, the text interaction data includes text messages input by the user, text content browsed by the user, and records of the user's text input behavior on the interactive interface; the image browsing data includes image content browsed by the user, the duration of the user's stay on the image, and records of the user's zooming and dragging operations on the image; the voice interaction data includes voice commands input by the user, the duration of the user's listening to voice content, and the speech rate and tone features during the voice interaction process; the preprocessing includes word segmentation and stop word removal of the text interaction data, size normalization and pixel value standardization of the image browsing data, and pre-emphasis and frame-segmentation windowing of the voice interaction data.

[0018] Preferably, in the behavioral feature extraction step, semantic encoding of the text modality feature sequence employs a pre-trained language model based on a transformer, mapping the text modality feature sequence to a fixed-dimensional text semantic vector; visual encoding of the image modality feature sequence employs a convolutional neural network model, mapping the image modality feature sequence to a fixed-dimensional image visual vector; acoustic encoding of the speech modality feature sequence employs a recurrent neural network model, mapping the speech modality feature sequence to a fixed-dimensional speech acoustic vector; and alignment according to acquisition time refers to aligning the feature vectors of the three modalities to the same time scale in the time dimension using the smallest acquisition time granularity as the unit, and filling the feature vectors of missing modalities with zero vectors.

[0019] Preferably, in the modal weight allocation step, the data volume ratio of each modality refers to the proportion of the data volume of each modality's behavioral data within the preset time window to the total data volume; the data update frequency of each modality refers to the number of new data entries of each modality's behavioral data per unit time; the initial weight coefficient is positively correlated with the data volume ratio; the preset adjustment coefficient β ranges from 0.1 to 0.5; after calculating the weight adjustment factor of each modality, the weight adjustment factor of each modality is normalized so that the sum of the weight adjustment factors of all modalities equals 1; the preset time window is from 5 minutes to 30 minutes.

[0020] Preferably, in the real-time state assessment step, the weighted fusion of the multimodal joint feature vector and the weight adjustment factors of each modality is calculated according to the following formula:

[0021] V=Σ(W_i'×H_i)

[0022] Wherein, V is the user's real-time state representation vector, W_i' is the weight adjustment factor after normalization of the i-th modality, and H_i is the behavioral feature vector of the i-th modality; the intent classification model is a multi-classification model based on a softmax classifier, and the intent categories include purchase intent, browsing intent, consultation intent, and exit intent; the intent confidence is the maximum value among the probabilities of each category output by the intent classification model.

[0023] Preferably, in the reach condition determination step, each reach condition in the reach condition set corresponds to an intent category, and the reach timing threshold includes a first threshold and a second threshold, wherein the first threshold is lower than the second threshold; when the intent confidence reaches the first threshold and is lower than the second threshold, it is determined that the current moment meets the first-level reach condition; when the intent confidence reaches the second threshold, it is determined that the current moment meets the second-level reach condition; the reach condition set also includes a time window constraint corresponding to each reach condition, wherein the time window constraint is used to limit the same user to triggering the same-level reach condition only once within a preset time length, and if the same-level reach condition has been triggered within the preset time length, the current moment is not determined to meet the reach condition again.

[0024] Preferably, in the reach intensity calculation step, the reach intensity value is calculated based on the user's real-time state representation vector and the intent confidence level according to the following formula:

[0025] S=C×(1+γ×max(0,P-T_first) / (T_second-T_first))

[0026] Wherein, S is the reach intensity value, C is the normalized value of the modulus of the user's real-time state representation vector, γ is a preset intensity scaling factor, P is the intent confidence, T_first is the first threshold, and T_second is the second threshold; when the intent confidence P is lower than or equal to the first threshold T_first, the reach intensity value is C; when the intent confidence P is higher than or equal to the second threshold T_second, the reach intensity value is C multiplied by (1+γ); when the intent confidence P is between the first threshold and the second threshold, the reach intensity value is calculated by linear interpolation; the reach method includes a first-level reach method and a second-level reach method, the reach intensity of the first-level reach method is lower than the reach intensity of the second-level reach method, the first-level reach method is matched when the reach intensity value is less than or equal to the preset first intensity threshold, and the second-level reach method is matched when the reach intensity value is greater than the first intensity threshold.

[0027] Preferably, the first-level reach method includes displaying a non-intrusive marketing information card or popping up a lightweight prompt window on the user's current browsing interface, and the second-level reach method includes displaying a full-screen marketing page or pushing a marketing notification with sound prompts on the user's current browsing interface; the reach duration of the first-level reach method is less than the reach duration of the second-level reach method; the reach intensity calculation step also calculates the reach duration based on the reach intensity value, and the reach duration is positively correlated with the reach intensity value.

[0028] Preferably, in the feedback update step, the user feedback behavior data includes the user's click behavior, close behavior, browsing duration, and subsequent conversion behavior in response to the marketing outreach; the feedback evaluation score is calculated as follows: if the user performs a click behavior or conversion behavior within a preset response time, a positive feedback score is assigned; if the user performs a close behavior within a preset response time, a negative feedback score is assigned; if the user does not perform any interaction behavior within a preset response time, a zero feedback score is assigned; updating the parameters of the intent classification model using the feedback evaluation score refers to using the feedback evaluation score as a reward signal and employing a reinforcement learning algorithm to perform gradient updates on the model parameters of the intent classification model; updating the outreach timing threshold in the outreach condition set using the feedback evaluation score refers to dynamically adjusting the values ​​of the first threshold and the second threshold based on the statistical distribution of the accumulated feedback evaluation scores.

[0029] This invention has the following advantages: By collecting and jointly analyzing user behavior data across multiple modalities such as text interaction, image browsing, and voice interaction, it can comprehensively understand the user's real-time state and intent from multiple dimensions, overcoming the shortcomings of insufficient information from single-modal data; by constructing a dynamic weight allocation mechanism and introducing a real-time feedback loop, it can adaptively adjust the contribution level of each modality feature according to changes in user behavior, making the judgment of the timing of outreach more accurate and timely, effectively improving the conversion rate and user satisfaction of marketing outreach; by setting multi-level outreach condition judgment and outreach intensity grading strategies, it can match differentiated outreach methods and intensities according to the user's real-time state, avoiding excessive marketing that may interfere with the user, and protecting the user experience while improving marketing effectiveness; furthermore, through a closed-loop feedback mechanism, it continuously optimizes the decision-making model, so that the quality of outreach timing decisions continuously improves with data accumulation, exhibiting good adaptability and scalability. Attached Figure Description

[0030] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).

[0031] Figure 1 A flowchart illustrating a multimodal user behavior-driven marketing outreach timing decision-making method provided in this application embodiment. Detailed Implementation

[0032] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these embodiments are merely for further explanation of the present invention and should not be construed as limiting the scope of protection of the present invention. Technical engineers in the field can make some non-essential improvements and adjustments to the present invention based on the above-described content. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Please see Figure 1 A multimodal user behavior-driven marketing outreach timing decision-making approach, comprising the following steps:

[0034] The multimodal behavior data acquisition step involves collecting multimodal behavior data of the target user within a preset time window. The multimodal behavior data includes text interaction data, image browsing data, and voice interaction data. Each modal behavior data is preprocessed to obtain standardized text modal feature sequences, image modal feature sequences, and voice modal feature sequences.

[0035] The behavioral feature extraction step involves semantically encoding the text modal feature sequence to obtain a text behavioral feature vector, visually encoding the image modal feature sequence to obtain an image behavioral feature vector, acoustically encoding the speech modal feature sequence to obtain a speech behavioral feature vector, and concatenating the text behavioral feature vector, image behavioral feature vector, and speech behavioral feature vector according to the acquisition time into a multimodal joint feature vector.

[0036] The modal weight allocation step involves calculating the initial weight coefficient for each modality based on the proportion of data volume and data update frequency of each modality's behavioral data within the preset time window, and then calculating the weight adjustment factor for each modality according to the following formula:

[0037] W_i=α_i×(1+β×log(1+f_i / F))

[0038] Where W_i is the weight adjustment factor of the i-th mode, α_i is the initial weight coefficient of the i-th mode, β is the preset adjustment coefficient, f_i is the data update frequency of the i-th mode within the preset time window, and F is the average data update frequency of all modes.

[0039] The real-time state assessment step involves weighting and fusing the multimodal joint feature vector with the weight adjustment factors of each modality to generate a user real-time state representation vector. The user real-time state representation vector is then input into a pre-trained intent classification model, which outputs the user's current intent category and intent confidence.

[0040] The reach condition determination step involves obtaining a preset reach condition set, which includes multiple reach conditions and a reach timing threshold corresponding to each reach condition. It then determines whether the intent confidence level reaches the reach timing threshold. If it does, the reach condition is determined to be met at the current moment; otherwise, monitoring continues.

[0041] The reach intensity calculation step involves, when it is determined that the reach conditions are met at the current moment, calculating the reach intensity value based on the user's real-time state representation vector and the intent confidence, and performing marketing outreach according to the reach intensity value and the corresponding reach method.

[0042] The feedback update step involves collecting user feedback behavior data after the marketing outreach is executed, generating a feedback evaluation score based on the user feedback behavior data, and using the feedback evaluation score to update the parameters of the intent classification model and the outreach timing threshold in the outreach condition set.

[0043] The core concept of this invention lies in collecting user behavior data across different modalities, including text interaction, image browsing, and voice interaction. This multimodal data is then jointly analyzed and its features are fused to construct a multidimensional feature vector representing the user's real-time state. Based on the user's real-time state and intent recognition results, the timing and intensity of marketing outreach are dynamically determined, and the decision quality is continuously optimized through a feedback mechanism. The technical solution of this invention is described in detail below with specific implementation steps.

[0044] I. Multimodal Behavioral Data Acquisition

[0045] In the specific implementation process, it is first required to deploy a data acquisition module on the terminal device of the target user (such as a smartphone, tablet computer, personal computer, etc.) or on the server side, and the module is configured to collect multi-modal behavior data of the user within a preset time window. The preset time window can be flexibly set according to actual application scenarios, for example, it can be set to the last 5 minutes, the last 10 minutes or the last 30 minutes. The longer the window length, the richer the user behavior information that can be captured, but the real-time performance will be correspondingly reduced. In a preferred embodiment, the preset time window is set to the last 10 minutes.

[0046] The collection of text interaction data includes: text messages input by the user in an application program or a web page (such as search keywords, comment content, chat messages), text content browsed by the user (such as text description on a product detail page, main content of an information article), and text input behavior records of the user on the interaction interface (such as input speed, modification times, etc.). The collected text interaction data undergoes preprocessing, and the preprocessing steps include using an open-source word segmentation tool (such as Jieba Word Segmentation) to perform word segmentation processing on Chinese text, and removing stop words (such as "de", "le", "shi" and other words that contribute no actual semantic meaning), so as to obtain a standardized text modal feature sequence.

[0047] The collection of image browsing data includes: image content browsed by the user (such as product images, advertisement images, information accompanying images), the duration the user stays on the image (reflecting the user's attention degree to the image content), and the user's zoom and drag operation records on the image (reflecting the user's intention to explore image details). The collected image browsing data undergoes preprocessing, and the preprocessing steps include uniformly scaling images of different sizes to a fixed size (such as 224×224 pixels), and performing standardization processing on each pixel value (such as mapping the pixel value from the 0-255 range to the 0-1 interval), so as to obtain a standardized image modal feature sequence.

[0048] The collection of voice interaction data includes: voice instructions input by the user (such as voice search, voice control commands), the duration the user listens to voice content (such as the duration of listening to voice messages, voice navigation), and the speech rate and pitch features during the voice interaction process (which can be extracted through voice signal processing). The collected voice interaction data undergoes preprocessing, and the preprocessing steps include performing pre-emphasis processing on the voice signal to enhance high-frequency components, performing framing processing (such as 25 milliseconds per frame, 10 milliseconds frame shift) and applying a Hamming window to reduce spectral leakage, so as to obtain a standardized voice modal feature sequence.

[0049] 2. Behavior Feature Extraction

[0050] After obtaining the standardized feature sequences of the three modals, it is required to perform feature encoding on each modal respectively, so as to convert the original feature sequences into feature vectors with fixed dimensions for subsequent fusion processing.

[0051] For text modal feature sequences, semantic encoding is performed using pre-trained language models (such as BERT, RoBERTa, etc.) based on a Transformer architecture. Specifically, the segmented text sequence is input into the pre-trained language model, and the vector corresponding to the [CLS] marker in the output of the last hidden layer of the model is taken as the semantic representation of the entire text sequence. This vector has a fixed dimension (such as 768 dimensions), which is the text behavior feature vector.

[0052] For image modal feature sequences, a convolutional neural network model (such as ResNet, EfficientNet, etc.) is used for visual encoding. Specifically, the standardized image is input into the convolutional neural network, and the vector obtained by pooling the feature map before the global average pooling layer or fully connected layer of the network is taken as the visual representation of the image. This vector has a fixed dimension (such as 512-dimensional or 1024-dimensional), which is the image behavior feature vector.

[0053] For speech modal feature sequences, recurrent neural network models (such as LSTM, GRU, etc.) are used for acoustic encoding. Specifically, the preprocessed speech frame sequence is input into the recurrent neural network, and the hidden state of the last time step of the network or the mean of the hidden states of all time steps is taken as the acoustic representation of the speech. This vector has a fixed dimension (such as 256-dimensional or 512-dimensional), which is the speech behavior feature vector.

[0054] After obtaining the feature vectors of the three modalities, they need to be aligned according to the acquisition time. Specifically, using the smallest acquisition time granularity (e.g., 1 second) as the unit, the feature vectors of the three modalities are aligned to the same time scale in the time dimension. For any missing modal feature vector at a certain time scale, it is filled with a zero vector to ensure that each of the three modalities has a corresponding feature vector at every time scale. After alignment, the text behavior feature vectors, image behavior feature vectors, and speech behavior feature vectors at the same time scale are concatenated to form a multimodal joint feature vector.

[0055] III. Modal Weight Allocation

[0056] Behavioral data from different modalities have varying abilities to represent user states in different scenarios. For example, in a scenario where a user actively searches for products, text interaction data (search terms) has a stronger representational ability; while in a scenario where a user browses product images, image browsing data has a stronger representational ability. Therefore, this invention designs a dynamic weight allocation mechanism that adaptively adjusts the weights of each modality based on the data activity level of each modality within a preset time window.

[0057] First, calculate the proportion of each modality's behavioral data within a preset time window. The proportion refers to the percentage of behavioral data from a particular modality relative to the total data from all modalities. For example, if there are 100 text interaction data points, 50 image browsing data points, and 30 voice interaction data points within the preset time window, then the proportion of text modality's data is approximately 100 / 180 ≈ 0.556. The initial weight coefficient is positively correlated with the proportion of data; that is, modalities with a larger proportion of data are assigned higher initial weights.

[0058] Secondly, calculate the data update frequency of each modality's behavioral data within a preset time window. The data update frequency refers to the number of new data entries for a particular modality's behavioral data per unit of time (e.g., per minute). A higher update frequency indicates more active user behavior in that modality and a more timely response to the current user status.

[0059] Then, calculate the weight adjustment factor for each mode according to the following formula:

[0060] W_i=α_i×(1+β×log(1+f_i / F))

[0061] Where W_i is the weight adjustment factor for the i-th mode, α_i is the initial weight coefficient for the i-th mode, β is a preset adjustment coefficient (ranging from 0.1 to 0.5, preferably 0.3), f_i is the data update frequency of the i-th mode within a preset time window, and F is the average data update frequency of all modes. This formula means that, based on the initial weights, the weights are adjusted upwards according to the update frequency of the mode relative to the average level; modes with higher update frequencies receive larger weight adjustment factors.

[0062] After calculating the weight adjustment factor for each mode, the weight adjustment factors for each mode are normalized so that the sum of the weight adjustment factors for all modes equals 1. The normalized weight adjustment factor is denoted as W_i'.

[0063] IV. Real-time Status Assessment

[0064] After obtaining the normalized weight adjustment factors for each modality and the joint feature vector of the multimodalities, weighted fusion is performed to generate the user's real-time state representation vector. The weighted fusion is calculated according to the following formula:

[0065] V=Σ(W_i'×H_i)

[0066] Where V is the user's real-time state representation vector, W_i' is the normalized weight adjustment factor for the i-th modality, and H_i is the behavioral feature vector of the i-th modality. Essentially, this formula is a weighted sum of the feature vectors from the three modalities, with the weights dynamically determined by the data activity level of each modality.

[0067] The user's real-time state representation vector V is input into a pre-trained intent classification model. The training data for the intent classification model is obtained through manual annotation. Annotators label the corresponding intent categories based on the user's behavior sequence, with each category being mutually exclusive. Annotation consistency is verified by calculating the Kappa coefficient among the annotators; the Kappa coefficient must be no less than 0.7.

[0068] The intent classification model is a multi-class classification model based on a softmax classifier. Its input is a real-time state representation vector, and its output is the probability distribution of each intent category. Intent categories include, but are not limited to, purchase intent (the user has a clear intention to purchase), browsing intent (the user is browsing products or content but has not yet formed a purchase intention), consultation intent (the user has a consultation need, such as needing more product information), and exit intent (the user is about to leave the current page or application). The training process of the intent classification model is as follows: A large number of user behavior samples labeled with intent categories are collected; multimodal behavioral features of each sample are extracted; and real-time state representation vectors are generated according to the above steps. The labeled intent categories are used as supervision signals, and the classification model is trained using the cross-entropy loss function and stochastic gradient descent optimization algorithm.

[0069] After the intent classification model outputs the probabilities of each category, the maximum value among all category probabilities is taken as the intent confidence score P. The higher the intent confidence score, the more certain the model is in judging the current user's intent.

[0070] V. Determination of Reach Conditions

[0071] The set of outreach conditions is pre-configured and stored in the system. This set contains multiple outreach conditions and the corresponding outreach timing threshold for each condition. Each outreach condition corresponds to an intent category. That is, when the user's intent is identified as belonging to a certain category and the confidence level reaches the corresponding threshold, the corresponding marketing outreach is triggered.

[0072] Specifically, the timing thresholds include a first threshold T_first and a second threshold T_second, where the first threshold is lower than the second threshold. For example, in one implementation, the first threshold can be set to 0.6 and the second threshold can be set to 0.85.

[0073] When the intent confidence P reaches the first threshold T_first and is lower than the second threshold T_second, the first-level reach condition is determined to be met at the current moment. When the intent confidence P reaches the second threshold T_second, the second-level reach condition is determined to be met at the current moment. The first-level reach condition corresponds to the situation where the user's intent is relatively clear but not very strong, and the second-level reach condition corresponds to the situation where the user's intent is very clear and strong.

[0074] In addition, the set of outreach conditions also includes a time window constraint for each outreach condition. The time window constraint limits the number of times an outreach condition of the same level to be triggered only once within a preset time period (e.g., 30 minutes). If an outreach condition of the same level has already been triggered within the preset time period, it will not be considered as fulfilling the outreach condition again at the current moment. This constraint mechanism avoids repeated marketing outreach to the same user within a short period, thus preventing harassment.

[0075] If the intent confidence P does not reach the first threshold T_first, it is determined that the current time does not meet the reach conditions, and user behavior data in subsequent time periods will continue to be monitored.

[0076] VI. Calculation of Contact Intensity

[0077] When determining whether the current moment meets the reach conditions, it is necessary to calculate the reach strength value based on the user's real-time status and intent confidence level in order to determine the appropriate strength for marketing outreach.

[0078] The impact intensity value is calculated according to the following formula:

[0079] S=C×(1+γ×max(0,P-T_first) / (T_second-T_first))

[0080] Where S is the reach intensity value, C is the normalized value of the magnitude of the user's real-time state representation vector V (that is, the magnitude of V is normalized to the [0,1] interval), γ is the preset intensity scaling factor (the value ranges from 0.5 to 2.0, preferably 1.0), P is the intent confidence level, T_first is the first threshold, and T_second is the second threshold.

[0081] The physical meaning of this formula is as follows: the basic reach intensity is determined by the magnitude of the user's real-time state representation vector (the larger the magnitude, the more significant the user's behavioral characteristics), and then adjusted upwards based on the position of the intent confidence P relative to the first and second thresholds. When P is less than or equal to T_first (i.e., only the first-level reach condition is met), max(0,P-T_first) = 0, and the reach intensity value S = C. When P is greater than or equal to T_second (i.e., the second-level reach condition is met), the reach intensity value S = C × (1 + γ), reaching its maximum value. When P is between T_first and T_second, the reach intensity value gradually increases from C to C × (1 + γ) using linear interpolation.

[0082] After calculating the reach intensity value S, the corresponding reach method is matched according to S. The reach methods include the first-level reach method and the second-level reach method. The reach intensity of the first-level reach method is lower than that of the second-level reach method. The reach duration D_duration and the reach intensity value S satisfy the following linear relationship: D_duration = D_min + S × (D_max - D_min), where D_min is the shortest reach duration, D_max is the longest reach duration, and the value of S ranges from [0,1].

[0083] Specifically, when S is less than or equal to a preset first intensity threshold, the first-level outreach method is matched; when S is greater than the first intensity threshold, the second-level outreach method is matched. The first-level outreach method includes displaying a non-intrusive marketing information card or a lightweight pop-up notification window on the user's current browsing interface; these methods have less interference with the user. The second-level outreach method includes displaying a full-screen marketing page on the user's current browsing interface or pushing a marketing notification with sound prompts; these methods have a stronger visual impact and are more effective at capturing user attention.

[0084] The reach duration of the first-level reach method is shorter than that of the second-level reach method. In addition, the reach intensity calculation step also calculates the reach duration based on the reach intensity value S. The reach duration is positively correlated with S, that is, the larger S is, the longer the reach duration.

[0085] VII. Feedback Updates

[0086] After executing a marketing outreach, collect user feedback data. This feedback data includes: whether the user clicked on the marketing content (click behavior), whether the user closed the marketing content (closing behavior), the duration the user spent browsing the marketing content, and whether the user made a purchase or registered after the outreach (subsequent conversion behavior).

[0087] Feedback evaluation scores are generated based on the aforementioned feedback behavior data. Specifically: if a user clicks or converts within a preset response time (e.g., 30 seconds), a positive feedback score (e.g., +1) is awarded; if a user closes the app within the preset response time, a negative feedback score (e.g., -1) is awarded; if a user does not interact with the app within the preset response time, a zero feedback score (e.g., 0) is awarded.

[0088] The parameters of the intent classification model are updated using feedback evaluation scores. Specifically, the feedback evaluation scores are used as reward signals, and reinforcement learning algorithms (such as policy gradient algorithms or Q-learning algorithms) are employed to update the model parameters of the intent classification model using gradients. The update direction is to make the intent judgments corresponding to behavior sequences that receive positive feedback more accurate, and to suppress the intent judgments corresponding to behavior sequences that receive negative feedback.

[0089] Simultaneously, the contact timing thresholds in the contact condition set are updated using feedback evaluation scores. Specifically, the values ​​of the first threshold T_first and the second threshold T_second are dynamically adjusted based on the statistical distribution of accumulated feedback evaluation scores (e.g., calculating the average feedback score of the past N contacts). If recent feedback scores are generally high, it indicates that the current threshold setting is reasonable or too conservative, and the threshold can be appropriately increased to filter out higher-value contact opportunities; if recent feedback scores are generally low, it indicates that the current threshold setting is too aggressive, and the threshold can be appropriately decreased to expand the contact coverage.

[0090] Through the aforementioned closed-loop feedback mechanism, the decision-making model of this invention can continuously optimize itself during continuous operation, and the decision quality of the timing of the event gradually improves with the accumulation of data.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal user behavior-driven marketing outreach timing decision-making method, characterized in that, Includes the following steps: The multimodal behavior data acquisition step involves collecting multimodal behavior data of the target user within a preset time window. The multimodal behavior data includes text interaction data, image browsing data, and voice interaction data. Each modal behavior data is preprocessed to obtain standardized text modal feature sequences, image modal feature sequences, and voice modal feature sequences. The behavioral feature extraction step involves semantically encoding the text modal feature sequence to obtain a text behavioral feature vector, visually encoding the image modal feature sequence to obtain an image behavioral feature vector, acoustically encoding the speech modal feature sequence to obtain a speech behavioral feature vector, and concatenating the text behavioral feature vector, image behavioral feature vector, and speech behavioral feature vector according to the acquisition time into a multimodal joint feature vector. The modal weight allocation step involves calculating the initial weight coefficient for each modality based on the proportion of data volume and data update frequency of each modality's behavioral data within the preset time window, and then calculating the weight adjustment factor for each modality according to the following formula: W_i=α_i×(1+β×log(1+f_i / F)) Where W_i is the weight adjustment factor of the i-th mode, α_i is the initial weight coefficient of the i-th mode, β is the preset adjustment coefficient, f_i is the data update frequency of the i-th mode within the preset time window, and F is the average data update frequency of all modes. The real-time state assessment step involves weighting and fusing the multimodal joint feature vector with the weight adjustment factors of each modality to generate a user real-time state representation vector. The user real-time state representation vector is then input into a pre-trained intent classification model, which outputs the user's current intent category and intent confidence. The reach condition determination step involves obtaining a preset reach condition set, which includes multiple reach conditions and a reach timing threshold corresponding to each reach condition. It then determines whether the intent confidence level reaches the reach timing threshold. If it does, the reach condition is determined to be met at the current moment; otherwise, monitoring continues. The reach intensity calculation step involves, when it is determined that the reach conditions are met at the current moment, calculating the reach intensity value based on the user's real-time state representation vector and the intent confidence, and performing marketing outreach according to the reach intensity value and the corresponding reach method. The feedback update step involves collecting user feedback behavior data after the marketing outreach is executed, generating a feedback evaluation score based on the user feedback behavior data, and using the feedback evaluation score to update the parameters of the intent classification model and the outreach timing threshold in the outreach condition set.

2. The marketing outreach timing decision method driven by multimodal user behavior according to claim 1, characterized in that, In the multimodal behavior data acquisition step, the text interaction data includes text messages input by the user, text content browsed by the user, and records of the user's text input behavior on the interactive interface; the image browsing data includes image content browsed by the user, the duration of the user's stay on the image, and records of the user's zooming and dragging operations on the image; the voice interaction data includes voice commands input by the user, the duration of the user's listening to voice content, and the speech rate and pitch characteristics during the voice interaction process; the preprocessing includes word segmentation and stop word removal of the text interaction data, size normalization and pixel value standardization of the image browsing data, and pre-emphasis and frame-segmentation windowing of the voice interaction data.

3. The marketing outreach timing decision method driven by multimodal user behavior according to claim 2, characterized in that, In the behavioral feature extraction step, semantic encoding of the text modality feature sequence is performed using a pre-trained language model based on a transformer, mapping the text modality feature sequence to a fixed-dimensional text semantic vector; visual encoding of the image modality feature sequence is performed using a convolutional neural network model, mapping the image modality feature sequence to a fixed-dimensional image visual vector; acoustic encoding of the speech modality feature sequence is performed using a recurrent neural network model, mapping the speech modality feature sequence to a fixed-dimensional speech acoustic vector; the alignment according to acquisition time refers to aligning the feature vectors of the three modalities to the same time scale in the time dimension using the smallest acquisition time granularity as the unit, and filling the feature vectors of missing modalities with zero vectors.

4. The marketing outreach timing decision method driven by multimodal user behavior according to claim 3, characterized in that, In the modal weight allocation step, the data volume ratio of each modality refers to the proportion of the data volume of each modality's behavioral data within the preset time window to the total data volume; the data update frequency of each modality refers to the number of new data entries of each modality's behavioral data per unit time; the initial weight coefficient is positively correlated with the data volume ratio; the preset adjustment coefficient β ranges from 0.1 to 0.5; after calculating the weight adjustment factor of each modality, the weight adjustment factor of each modality is normalized so that the sum of the weight adjustment factors of all modalities equals 1; the preset time window is 5 minutes to 30 minutes.

5. The marketing outreach timing decision method driven by multimodal user behavior according to claim 4, characterized in that, In the real-time state assessment step, the weighted fusion of the multimodal joint feature vector and the weight adjustment factor of each modality is calculated according to the following formula: V=Σ(W_i'×H_i) Wherein, V is the user's real-time state representation vector, W_i' is the weight adjustment factor after normalization of the i-th modality, and H_i is the behavioral feature vector of the i-th modality; the intent classification model is a multi-classification model based on a softmax classifier, and the intent categories include purchase intent, browsing intent, consultation intent, and exit intent; the intent confidence is the maximum value among the probabilities of each category output by the intent classification model.

6. The marketing outreach timing decision method driven by multimodal user behavior according to claim 5, characterized in that, In the reach condition determination step, each reach condition in the reach condition set corresponds to an intent category. The reach timing threshold includes a first threshold and a second threshold, where the first threshold is lower than the second threshold. When the intent confidence reaches the first threshold and is lower than the second threshold, it is determined that the current moment meets the first-level reach condition. When the intent confidence reaches the second threshold, it is determined that the current moment meets the second-level reach condition. The reach condition set also includes a time window constraint corresponding to each reach condition. The time window constraint is used to limit the same user to triggering the same-level reach condition only once within a preset time length. If the same-level reach condition has already been triggered within the preset time length, it is not determined to meet the reach condition again at the current moment.

7. The marketing outreach timing decision method driven by multimodal user behavior according to claim 6, characterized in that, In the reach intensity calculation step, the reach intensity value is calculated based on the user's real-time state representation vector and the intent confidence level according to the following formula: S=C×(1+γ×max(0,P-T_first) / (T_second-T_first)) Wherein, S is the reach intensity value, C is the normalized value of the modulus of the user's real-time state representation vector, γ is a preset intensity scaling factor, P is the intent confidence, T_first is the first threshold, and T_second is the second threshold; when the intent confidence P is lower than or equal to the first threshold T_first, the reach intensity value is C; when the intent confidence P is higher than or equal to the second threshold T_second, the reach intensity value is C multiplied by (1+γ); when the intent confidence P is between the first threshold and the second threshold, the reach intensity value is calculated by linear interpolation; the reach method includes a first-level reach method and a second-level reach method, the reach intensity of the first-level reach method is lower than the reach intensity of the second-level reach method, the first-level reach method is matched when the reach intensity value is less than or equal to the preset first intensity threshold, and the second-level reach method is matched when the reach intensity value is greater than the first intensity threshold.

8. The marketing outreach timing decision method driven by multimodal user behavior according to claim 7, characterized in that, The first-level reach method includes displaying a non-intrusive marketing information card or popping up a lightweight prompt window on the user's current browsing interface; the second-level reach method includes displaying a full-screen marketing page or pushing a marketing notification with sound prompts on the user's current browsing interface; the reach duration of the first-level reach method is less than the reach duration of the second-level reach method; the reach intensity calculation step also calculates the reach duration based on the reach intensity value, and the reach duration is positively correlated with the reach intensity value.

9. The marketing outreach timing decision method driven by multimodal user behavior according to claim 8, characterized in that, In the feedback update step, the user feedback behavior data includes the user's click behavior, close behavior, browsing duration, and subsequent conversion behavior in response to the marketing outreach; the feedback evaluation score is calculated as follows: if the user clicks or converts within a preset response time, a positive feedback score is assigned; if the user closes the app within a preset response time, a negative feedback score is assigned; if the user does not interact within a preset response time, a zero feedback score is assigned; updating the parameters of the intent classification model using the feedback evaluation score refers to using the feedback evaluation score as a reward signal and employing a reinforcement learning algorithm to perform gradient updates on the model parameters of the intent classification model; updating the outreach timing threshold in the outreach condition set using the feedback evaluation score refers to dynamically adjusting the values ​​of the first threshold and the second threshold based on the statistical distribution of the accumulated feedback evaluation scores.

Images