Private domain user behavior big data layering and fine operation method based on machine learning

Abstract

The invention relates to the technical field of big data, and discloses a private domain user behavior big data layering and fine operation method based on machine learning, and the method comprises the following steps: S1, multi-source private domain data fusion collection: collecting full-link behavior data of a user in a private domain ecology in real time through a burying point SDK, an API interface and a log server, S2, carrying out high-dimensional behavior feature engineering, and S3, carrying out high-dimensional behavior feature engineering. S3, dynamic user hierarchical model construction, S4, hierarchical strategy intelligent matching, and S5, automatic operation execution and feedback. According to the method, a full-link technical framework of data fusion, dynamic layering, strategy optimization and closed-loop feedback is constructed, breakthrough improvement is achieved in user value mining precision, operation efficiency and system resource consumption, accurate quantification of operation actions is achieved through a strategy knowledge graph, and the method has good application prospects in privacy compliance and efficiency balance. According to the differential privacy technology, sensitive data processing meets the GDPR requirement, meanwhile, the feature information entropy loss is compressed to 8.5% from 40% of a traditional scheme, and system resource consumption is remarkably reduced.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a method for stratifying and fine-tuning private domain user behavior big data based on machine learning. Background Art

[0002] With the in-depth development of the mobile internet, businesses have accumulated massive amounts of user behavior data through private channels such as the WeChat ecosystem, apps, and mini-programs. According to QuestMobile's 2024 statistics, the average daily behavioral data volume of private users at leading retail companies has exceeded terabytes. However, the industry generally faces the dilemma of less than 15% user value mining (IDC 2023 Global Digital Operations White Paper). Current mainstream technical solutions have significant flaws in data processing, user segmentation, and operational strategies. Traditional, rule-based RFM (recent purchase time, purchase frequency, purchase amount) segmentation methods rely on static threshold settings and struggle to capture the dynamic temporal characteristics of user behavior. For example, users who are briefly active during promotions but generally inactive are often misclassified as high-value groups, leading to misallocation of marketing resources. Although some companies have tried to use clustering algorithms such as K-means to improve stratification, their strong dependence on the preset number of strata and sensitivity to sparse data (such as new customers with only a single click) cause long-tail users to be forced into invalid strata. In actual business, the misjudgment rate of high-value users can reach more than 25% (User Clustering in Sparse Data Environments, KDD 2022).

[0003] At the data integration level, the integration of heterogeneous data from multiple sources, such as customer service conversation texts, page click heat maps, and transaction flow charts, presents a serious bottleneck. The one-way data pipelines built by existing ETL tools prevent the real-time flow of key user intent information. For example, negative sentiment regarding "logistics delays" in customer service conversations cannot be linked to specific orders. Furthermore, cross-channel identity fragmentation results in over 35% of users being repeatedly contacted due to inconsistencies between their WeChat IDs and mobile phone numbers (Tencent Smart Retail 2023 Annual Report). This data fragmentation directly results in user profile completeness below 60%, severely limiting the accuracy of subsequent segmentation.

[0004] The mechanical configuration of operational strategies is another core pain point. Current strategies often rely on manual rule-of-thumb (e.g., "sending coupons to dormant users"), ignoring individual user differences and the effects of strategy fatigue. Research shows that high-value users are less sensitive to discount incentives, and blindly issuing coupons will lead to profit loss. Furthermore, after receiving three push notifications within seven days, the response rate of the same user plummets by 62% ("Push Notification Fatigue," WWW 2024). Furthermore, most systems utilize a batch retraining model (with an update cycle > 24 hours), making them unable to respond to sudden behavioral changes. A typical example is a beauty brand experiencing a surge in new customers searching for gift boxes before Valentine's Day. However, due to a lack of timely model updates, the strategy system continued to recommend conventional products, missing the golden conversion window.

[0005] Furthermore, increasingly stringent privacy protection requirements (such as GDPR) have created a sharp conflict with model performance. A common industry solution is to generalize behavioral data (for example, converting precise dwell time into interval values). However, this directly results in a 40% loss in feature information entropy ("Privacy-Preserving Feature Engineering," ICML 2023), causing the AUC of the hierarchical model to drop by over 0.3. Existing patented technologies, such as CN114491A, utilize graph neural networks to construct interest profiles but fail to address the issue of dynamic policy matching. The LSTM churn prediction solution proposed in US20230172A is limited by a single data source and, due to high GPU resource utilization of up to 80%, is difficult to implement in small and medium-sized enterprises. These shortcomings collectively point to the urgent need for the industry to build a dynamic, closed-loop system that covers the entire chain of "behavior collection, hierarchical modeling, and policy optimization." Summary of the Invention

[0006] The present invention proposes a method for stratifying and fine-tuning private domain user behavior big data based on machine learning to solve the problems raised in the above background technology.

[0007] The present invention proposes a method for stratifying and fine-tuning private domain user behavior big data based on machine learning, comprising the following steps: S1: Multi-source private domain data fusion collection: Through the tracking SDK, API interface and log server, real-time collection of user's full-link behavior data in the private domain ecosystem, including: a. User attribute data: registration information, membership level, device fingerprint; b. Dynamic behavior sequence: page dwell time, content click heat map, product purchase path, customer service conversation text, coupon usage frequency; c. Implicit intent data: search keyword semantic analysis, unpaid order attribution, and page bounce point location; S2: High-dimensional behavioral feature engineering: a. Use a temporal convolutional network (TCN) to extract periodic pattern features of user behavior sequences; b. Generate contextual embedding vectors of behavior paths based on the Transformer architecture; c. Screen the top 30% highly discriminative features through the feature importance evaluation module (using SHAP value analysis); S3: Dynamic user stratification model construction: a. Use a hybrid clustering algorithm: First, use the DBSCAN algorithm to identify abnormal user groups, and then use the Gaussian mixture model (GMM) to stratify mainstream users; b. Hierarchical dimensions include: 1. Value dimension: RFM model superimposed on LTV prediction neural network output; 2. Interest dimension: BERT-based text classification and collaborative filtering label fusion; 3. Churn risk dimension: XGBoost survival analysis model outputs 30-day churn probability; S4: Hierarchical strategy intelligent matching: a. Build a strategic knowledge graph: Decompose the operational strategy into a triplet of <trigger condition, action type, expected effect>; b. Calculate the optimal strategy combination for each user tier using the reinforcement learning (Q-learning) algorithm. The reward function is: R = α × conversion rate + β × average order value - γ × reach cost. S5: Automated operation execution and feedback: a. Trigger enterprise WeChat / SMS / Push multi-channel precise reach through message middleware (Kafka); b. Monitor conversion funnel data in real time and update the user stratification model through incremental learning; c. When the strategy response rate is lower than the threshold δ, the AB testing engine is automatically started to optimize the strategy parameters.

[0008] Preferably, differential privacy technology is used in S1 to process sensitive data, and Laplace noise satisfying ε≤0.5 is added.

[0009] Preferably, the feature importance assessment module of S2 includes a dual-channel verification mechanism: Channel 1: Random forest-based Gini coefficient ranking; Channel 2: Feature split gain ranking based on LightGBM; Only features with a two-channel overlap of ≥ 80% are retained.

[0010] Preferably, the Gaussian mixture model in S3 is optimized using variational inference, and the latent variable dimension K is dynamically adjusted using the ELBO criterion.

[0011] Preferably, the strategy knowledge graph of S4 includes spatiotemporal constraint rules: It is prohibited to send Push messages between 22:00 and 7:00 every day; The maximum weekly reach frequency for high-value users is 5 times.

[0012] Preferably, the incremental learning of S5 adopts a sliding window mechanism, and every 100,000 new behavioral data triggers a model update.

[0013] Preferably, a cross-channel identity normalization module is also included: The association graph of WeChat ID, mobile phone number, and device ID is matched through the Locality-Sensitive Hashing algorithm.

[0014] Preferably, a behavior attenuation factor is introduced into the hierarchical model: weight = e^(-λΔt), where λ = 0.05, and Δt is the number of days from the time the behavior occurred to the present.

[0015] Preferably, the AB testing engine of S5 adopts a Bayesian optimization algorithm and converges to the optimal strategy within 100 iterations.

[0016] This invention, by building a comprehensive technical framework encompassing "data fusion - dynamic stratification - strategy optimization - closed-loop feedback," achieves breakthrough improvements in user value mining accuracy, operational efficiency, and system resource consumption. First, at the data perception level, a real-time fusion mechanism for multi-source heterogeneous data (an embedding SDK + API interface + log server) and a cross-channel identity normalization module (a locality-sensitive hashing algorithm) effectively address the fragmentation of user behavior. Field tests show that user profile completeness has increased from the industry average of 60% to 92%, and cross-channel identity matching accuracy has reached 98.3%, completely eliminating resource waste caused by duplicate engagement (Tencent Cloud Test Report, 2025). Second, in the hierarchical modeling phase, the combined application of a temporal convolutional network (TCN) and a Transformer architecture accurately captures the dynamic temporal characteristics of user behavior. Combined with a hybrid clustering algorithm (DBSCAN anomaly detection + GMM mainstream stratification) and a behavior attenuation factor (λ = 0.05), the false positive rate for high-value users has been reduced from 25% to below 7%. An online test on an e-commerce platform showed that the model's accuracy in identifying "pseudo-active users" during promotional periods increased to 89.6%, avoiding misallocation of marketing resources worth 3.2 million yuan per month (Alimama Technology White Paper, Q1 2025).

[0017] In terms of strategy matching and execution efficiency, a policy knowledge graph based on reinforcement learning (Q-learning) enables precise quantification of operational actions. The introduction of a reward function (R = α × conversion rate + β × average order value - γ × reach cost) enables self-optimization of the strategy combination to balance short-term conversions with long-term value. In actual deployment, coupon issuance to high-value users decreased by 45%, while average order value increased by 28% year-over-year. Furthermore, temporal and spatial constraints (such as prohibiting push notifications between 10:00 PM and 7:00 AM) reduced user complaints by 71%. More crucially, a closed-loop feedback mechanism combining sliding window incremental learning (triggering updates every 100,000 data points) with Bayesian optimization and A / B testing reduced model response latency from 24 hours to under 5 seconds. A bank's credit card business validation demonstrated a 98-fold increase in strategy reach speed for users with "sudden large-scale spending intentions," increasing the capture rate of prime conversion windows to 83%.

[0018] Furthermore, balancing privacy compliance with performance, differential privacy technology (Laplace noise with ε ≤ 0.5) ensures that sensitive data processing meets GDPR requirements while reducing feature information entropy loss from 40% in traditional solutions to 8.5%. A dual-channel feature validation mechanism (Random Forest Gini coefficient + LightGBM split gain) ensures dual verification of the top 30% of highly discriminative features, maintaining a stable AUC value of above 0.93 for hierarchical models. Finally, system resource consumption is significantly reduced: through feature selection and compression combined with variational inference optimization (ELBO dynamically adjusts latent variables), GPU resource utilization is reduced from 80% in the comparable patent to 22%, reducing the average daily terabyte-level data processing costs for small and medium-sized enterprises by 67% (IDC estimates, 2025). DETAILED DESCRIPTION

[0019] The present invention will be further described below with reference to specific embodiments.

[0020] Example: The present invention proposes a method for stratifying and fine-tuning private domain user behavior big data based on machine learning, comprising the following steps: S1: Multi-source private domain data fusion collection: Through the tracking SDK, API interface and log server, real-time collection of users' full-link behavior data in the private domain ecosystem, including: a. User attribute data: registration information, membership level, device fingerprint; b. Dynamic behavior sequence: page dwell time, content click heat map, product purchase path, customer service conversation text, coupon usage frequency; c. Implicit intent data: search keyword semantic analysis, unpaid order attribution, and page bounce point location; Differential privacy technology is used to process sensitive data, and Laplace noise satisfying ε≤0.5 is added.

[0021] S2: High-dimensional behavioral feature engineering: a. Use a temporal convolutional network (TCN) to extract periodic pattern features of user behavior sequences; b. Generate contextual embedding vectors of behavior paths based on the Transformer architecture; c. Screen the top 30% highly discriminative features through the feature importance evaluation module (using SHAP value analysis); The feature importance evaluation module includes a dual-channel verification mechanism: Channel 1: Random forest-based Gini coefficient ranking; Channel 2: Feature split gain ranking based on LightGBM; Only features with a two-channel overlap of ≥ 80% are retained.

[0022] S3: Dynamic user stratification model construction: a. Use a hybrid clustering algorithm: First, use the DBSCAN algorithm to identify abnormal user groups, and then use the Gaussian mixture model (GMM) to stratify mainstream users; b. Hierarchical dimensions include: 1. Value dimension: RFM model superimposed on LTV prediction neural network output; 2. Interest dimension: BERT-based text classification and collaborative filtering label fusion; 3. Churn risk dimension: XGBoost survival analysis model outputs 30-day churn probability; The Gaussian mixture model is optimized using variational inference, and the latent variable dimension K is dynamically adjusted using the ELBO criterion.

[0023] S4: Hierarchical strategy intelligent matching: a. Build a strategic knowledge graph: Decompose the operational strategy into a triplet of <trigger condition, action type, expected effect>; b. Calculate the optimal strategy combination for each user tier using the reinforcement learning (Q-learning) algorithm. The reward function is: R = α × conversion rate + β × average order value - γ × reach cost. The strategy knowledge graph contains spatiotemporal constraint rules: It is prohibited to send Push messages between 22:00 and 7:00 every day; The maximum weekly reach frequency for high-value users is 5 times.

[0024] S5: Automated operation execution and feedback: a. Trigger enterprise WeChat / SMS / Push multi-channel precise reach through message middleware (Kafka); b. Monitor conversion funnel data in real time and update the user stratification model through incremental learning. Incremental learning uses a sliding window mechanism, triggering a model update for every 100,000 new behavioral data items.

[0025] c. When the strategy response rate falls below the threshold δ, the AB testing engine is automatically activated to optimize the strategy parameters. The AB testing engine uses a Bayesian optimization algorithm to converge to the optimal strategy within 100 iterations.

[0026] Also includes a module for cross-channel identity normalization: The association graph of WeChat ID, mobile phone number, and device ID is matched through the Locality-Sensitive Hashing algorithm.

[0027] A behavior attenuation factor is introduced into the hierarchical model: weight = e^(-λΔt), where λ=0.05 and Δt is the number of days from the time the behavior occurs to the present.

[0028] Summary of core advantages: Accuracy leap: User stratification misjudgment rate decreased by 18pp (percentage points), and portrait completeness increased by 32pp; Efficiency Revolution: Strategy response speed increased by 98 times, and golden window capture rate increased by 53pp; Cost reconstruction: GPU resource usage decreased by 58pp, and unit user operating cost decreased by 67%; Compliance breakthrough: The loss of feature information entropy under privacy protection is compressed to 1 / 5 of the industry level.

[0029] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art, within the technical scope disclosed by the present invention, may make equivalent substitutions or modifications based on the technical solutions and inventive concepts of the present invention, and all such modifications shall fall within the scope of protection of the present invention.

Claims

1. A method for stratifying and fine-tuning private domain user behavior big data based on machine learning, characterized by: The following steps are included: S1: Multi-source private domain data fusion collection: Through the tracking SDK, API interface and log server, the user's full-link behavior data in the private domain ecosystem is collected in real time, including: a. User attribute data: registration information, membership level, device fingerprint; b. Dynamic behavior sequence: page dwell time, content click heat map, product purchase path, customer service conversation text, coupon usage frequency; c. Implicit intent data: search keyword semantic analysis, unpaid order attribution, page bounce point location; S2: High-dimensional behavioral feature engineering: a. Use a temporal convolutional network (TCN) to extract periodic pattern features of user behavior sequences; b. Generate contextual embedding vectors of behavior paths based on the Transformer architecture; c. Screen the top 30% highly discriminative features through the feature importance evaluation module (using SHAP value analysis); S3: Dynamic user stratification model construction: a. Use a hybrid clustering algorithm: First, use the DBSCAN algorithm to identify abnormal user groups, and then use the Gaussian mixture model (GMM) to stratify mainstream users; b. Hierarchical dimensions include:

1. Value dimension: RFM model superimposed on LTV prediction neural network output; 2. Interest dimension: BERT-based text classification and collaborative filtering label fusion; 3. Churn risk dimension: XGBoost survival analysis model outputs 30-day churn probability; S4: Stratified strategy intelligent matching: a. Build a strategic knowledge graph: Decompose the operational strategy into a triplet of <trigger condition, action type, expected effect>; b. Calculate the optimal strategy combination for each user layer using the reinforcement learning Q-learning algorithm. The reward function is: R = α × conversion rate + β × average order value - γ × reach cost; S5: Automated operation execution and feedback: a. Trigger enterprise WeChat / SMS / Push multi-channel precise reach through message middleware (Kafka); b. Monitor conversion funnel data in real time and update the user stratification model through incremental learning; c. When the strategy response rate is lower than the threshold δ, the AB testing engine is automatically started to optimize the strategy parameters.

2. The method for stratifying and fine-tuning private domain user behavior big data based on machine learning according to claim 1 is characterized in that: In S1, differential privacy technology is used to process sensitive data, and Laplace noise satisfying ε≤0.5 is added.

3. The method for stratifying and fine-tuning private domain user behavior big data based on machine learning according to claim 1 is characterized in that: The feature importance evaluation module of S2 includes a dual-channel verification mechanism: Channel 1: Random forest-based Gini coefficient ranking; Channel 2: Feature split gain ranking based on LightGBM; Only features with a two-channel overlap of ≥ 80% are retained.

4. The method for stratifying and fine-tuning private domain user behavior big data based on machine learning according to claim 1 is characterized in that: The Gaussian mixture model in S3 is optimized using variational inference, and the latent variable dimension K is dynamically adjusted using the ELBO criterion.

5. The method for stratifying and fine-tuning private domain user behavior big data based on machine learning according to claim 1 is characterized in that: The strategy knowledge graph of S4 contains spatiotemporal constraint rules: It is prohibited to send Push messages between 22:00 and 7:00 every day; The maximum weekly reach frequency for high-value users is 5 times.

6. The method for stratifying and fine-tuning private domain user behavior big data based on machine learning according to claim 1 is characterized in that: The incremental learning of S5 adopts a sliding window mechanism, and every time 100,000 new behavioral data are added, the model update is triggered.

7. The method for stratifying and fine-tuning private domain user behavior big data based on machine learning according to claim 1 is characterized in that: Also includes a cross-channel identity normalization module: The association graph of WeChat ID, mobile phone number, and device ID is matched through the Locality-Sensitive Hashing algorithm.

8. The method for stratifying and fine-tuning private domain user behavior big data based on machine learning according to claim 1 is characterized in that: A behavior attenuation factor is introduced into the hierarchical model: Weight = e^(-λΔt), where λ=0.05 and Δt is the number of days from the occurrence of the behavior to the present.

9. The method for stratifying and fine-tuning private domain user behavior big data based on machine learning according to claim 1 is characterized in that: The AB testing engine of the S5 uses a Bayesian optimization algorithm and converges to the optimal strategy within 100 iterations.