A data processing and control method, device and system based on user state event triggering

Abstract

This application discloses a data processing and control method, apparatus, and system based on user state event triggering, belonging to the field of computer technology. The method includes: acquiring the original dataset of the target user; monitoring the real-time data stream and detecting user state change events based on predefined event rules; in response to the detection of a state change event, triggering dynamic updates to the user feature vector, and incrementally training the data processing model based on the new data corresponding to the event to update its model parameters; and generating processing result data based on the updated feature vector and model. This application solves the technical problems of lagging model updates and inability to adapt to data stream changes in the prior art through a collaborative mechanism of event-driven and incremental learning, achieving dynamic adaptation, low-latency updates, and accurate output in the data processing system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing and control method, apparatus and system based on user state event triggering. Background Technology

[0002] In the field of data-driven personalized services, existing technologies typically rely on static or periodically updated models to process user data and generate service content, such as in insurance product recommendation scenarios. Current research and practice in personalized insurance recommendation mainly focus on the following technical approaches: Collaborative filtering-based recommendation systems utilize user-product interaction matrices and mine potential relationships through matrix factorization or neighborhood-based methods. Rule-engine-based expert systems match insurance products using predefined rules (such as age and income thresholds) and employ the C4.5 decision tree algorithm for risk classification of users. Shallow machine learning models, such as logistic regression or random forests, perform binary classification of user features, but feature engineering relies on human experience. Text analysis-based models, such as TF-IDF or LDA topic models, extract keywords from user consultation records, but their semantic understanding capabilities are limited.

[0003] Because existing technologies rely solely on simple user information and behavioral data for recommendations, current solutions suffer from the following technical bottlenecks. First, at the data representation level, the data dimensions are relatively limited, relying only on structured data (such as age and income) and ignoring the multimodal features of unstructured behavioral data (such as browsing paths and conversation text). This results in insufficient sparsity and representational power of user feature vectors, leading to coarse-grained user profiles. Specifically, this manifests as poor technical indicators such as high feature sparsity, low feature information entropy, and weak cross-modal feature correlation.

[0004] Secondly, regarding model updates and adaptability, dynamic adaptability is insufficient. Traditional models lack online learning mechanisms, resulting in high update latency and a lack of incremental learning mechanisms, making it impossible to respond to changes in user behavior in real time. This leads to a lag in the system's output based on outdated data patterns even after changes in user status (such as the occurrence of significant life events).

[0005] Furthermore, at the semantic understanding and matching level, the model has limitations in semantic understanding. Rule engines and shallow NLP techniques struggle to parse the implicit intentions in user consultation records, resulting in low accuracy in semantic matching based on shallow text features. This makes the matching results sensitive to input noise and unable to understand deep semantic relationships, leading to poor stability and low semantic discriminativeness in the matching results.

[0006] These technical bottlenecks collectively lead to the inefficiencies, unstable results, and poor adaptability in the matching process between user feature vectors and product feature vectors in existing insurance recommendation systems due to insufficient feature engineering, rigid model update mechanisms, and a single semantic matching method. Therefore, there is an urgent need for a data processing and control solution that can integrate multi-dimensional data, achieve dynamic model adaptation, and possess deep semantic understanding capabilities. Summary of the Invention

[0007] This application aims to provide a data processing and control method, apparatus and system based on user state event triggering, in order to solve the technical problems of low data processing efficiency caused by insufficient data representation, lagging model updates and coarse semantic matching in the prior art.

[0008] In a first aspect, embodiments of this application provide a data processing and control method based on user state event triggering. The method involves: acquiring the original dataset of a target user; monitoring real-time data streams related to the target user; detecting state change events of the target user based on predefined event rules; in response to the detection of the state change event, triggering a dynamic update of the target user's user feature vector; and incrementally training a preset data processing model to update its model parameters based on new data corresponding to the state change event; and generating processing result data for the target user based on the updated user feature vector and the updated data processing model.

[0009] In some embodiments, to construct a user representation with low sparsity and high information density to solve the feature sparsity problem, the processed result data is a personalized recommendation list; the original dataset includes at least structured attribute data, unstructured text data, and time-series behavior log data; triggering the dynamic update of the user feature vector includes: performing multimodal feature fusion processing on the original dataset to generate a unified user feature vector with low sparsity; wherein, the multimodal feature fusion processing includes: performing word embedding encoding on the unstructured text data to generate a first dense vector, performing sequence pattern encoding on the time-series behavior log data to generate a second dense vector, and concatenating and reducing the dimensionality of the first dense vector, the second dense vector, and the standardized structured attribute data.

[0010] In some embodiments, to optimize feature selection and further improve representation quality, the multimodal feature fusion processing of the original dataset further includes: using a clustering algorithm to segment the time-series behavior log data into user groups and generating behavior pattern label features; and using an information gain-based feature selection algorithm to select a subset of features from the concatenated feature set that are more correlated with a preset target than a preset threshold, so as to update the unified user feature vector.

[0011] In some embodiments, to achieve stable and accurate matching and solve the problem of coarse matching, the generation of processing result data includes: using a deep semantic matching network to calculate the matching degree between the user feature vector of the target user and the feature vectors of multiple objects to be matched, and generating a sorted list as the processing result data based on the matching degree; wherein, the deep semantic matching network maps user-side features and object-side features to a high-dimensional semantic space through a dual-tower neural network structure before performing similarity calculation.

[0012] In some embodiments, to enhance the discriminative power of the deep semantic matching network, the training process of the deep semantic matching network employs a contrastive loss function. The contrastive loss function is used to increase the semantic similarity score between the positive sample object and the target user, and to reduce the gap in semantic similarity score between the difficult negative sample object and the target user.

[0013] In some embodiments, to specifically implement a low-latency model update mechanism, the data processing model is a gradient boosting decision tree model that supports incremental learning; the incremental training of the data processing model specifically includes: retrieving context data associated with the state change event from historical data to form an incremental training dataset; performing a partial fitting operation on the gradient boosting decision tree model with the incremental training dataset, and introducing a regularization term to prevent overfitting.

[0014] In some embodiments, to enhance the credibility and transparency of the results, after generating the processing result data, the following steps are further included: based on causal inference technology, analyzing the decision contribution of each feature dimension in the user feature vector to generating any output item in the processing result data; and generating interpretable explanatory text corresponding to the output item based on the decision contribution.

[0015] In some embodiments, to address the cold start problem caused by the scarcity of new user data, a cold start processing step for newly registered users is included before obtaining the original dataset of the target users: obtaining the initial interaction behavior sequence generated by the newly registered users within a preset time window; inputting the initial interaction behavior sequence into a fast adaptation model trained by meta-learning, wherein the fast adaptation model adjusts its internal parameters based on the initial interaction behavior sequence and outputs the initial feature vector of the newly registered users for initial processing.

[0016] In some embodiments, to protect data privacy and achieve distributed collaborative training, the training process of the data processing model or deep semantic matching network adopts a federated learning framework, including: calculating the update gradient of model parameters locally on multiple clients using private data; uploading the encrypted update gradient to a central server; having the central server aggregate the update gradient to update the global model, and distributing the updated global model parameters to each client.

[0017] Preferably, the method further includes: calculating the feature quality index of the user feature vector, the feature quality index including feature sparsity or feature information entropy; when the feature quality index is lower than a preset first threshold, triggering the re-adjustment of the parameters of the multimodal feature fusion processing.

[0018] Preferably, after incrementally training the preset data processing model, the method further includes: calculating the model update delay of this incremental training and evaluating the performance retention rate of the updated model on the validation set; if the model update delay exceeds a preset second threshold, or the performance retention rate is lower than a preset third threshold, then performing model rollback or triggering consolidation training containing more historical data.

[0019] Preferably, after generating the processing result data, the method further includes: calculating the stability index of the matching result for the same user under continuous requests; if the stability index of the matching result is lower than a preset fourth threshold, then enabling result caching for the current request or smoothing the output of the deep semantic matching network.

[0020] Secondly, embodiments of this application also provide a data processing and control device based on user state event triggering, used to implement the method described in the first aspect above. It includes: a data acquisition module for acquiring the original dataset of the target user; an event monitoring module for listening to real-time data streams related to the target user and detecting state change events of the target user based on predefined event rules; a quantity update module for triggering dynamic updates of the user feature vector of the target user in response to the detection of the state change event, and incrementally training a preset data processing model to update its model parameters based on the new data corresponding to the state change event; and a recommendation generation module for generating processing result data for the target user based on the updated user feature vector and the updated data processing model.

[0021] Thirdly, embodiments of this application also provide a data processing system, characterized in that it includes a terminal device and a server; the terminal device is used to collect user raw data and send it to the server, and to receive and display processing result data from the server; the server is used to run the data processing and control device based on user state event triggering as described in the second aspect above, and / or to execute the method described in the first aspect above, generate the processing result data and send it to the terminal device or associated business system.

[0022] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the method described in the first aspect above.

[0023] Fifthly, embodiments of this application also provide an electronic device, characterized in that it includes: a memory for storing a computer program; and a processor for executing the computer program stored in the memory to implement the method described in the first aspect above.

[0024] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0025] By establishing a low-latency model update mechanism based on event-driven and incremental learning, the model lag problem is solved. Optimizations are directly made to reduce model update latency and computational overhead, enabling rapid model adaptation triggered by user state change events with low computational cost.

[0026] By constructing a multimodal feature fusion system for low sparsity and high information entropy, the problem of feature sparsity is solved. The aim is to significantly reduce feature sparsity and increase feature information entropy, thereby laying a data foundation for accurate matching.

[0027] By introducing a robust semantic matching architecture that integrates deep representation learning and contrastive training, the problem of coarse matching is solved. This aims to improve the stability and semantic discriminative power of matching results, thereby overcoming the fragility and shallowness of traditional keyword matching methods.

[0028] By integrating the above technologies, a paradigm shift from static matching to dynamic adaptation and from shallow features to deep understanding has been achieved, constructing a precise, intelligent, and adaptive data processing and control system.

[0029] Furthermore, the method utilizes a built-in configurable metric threshold system to automate monitoring and feedback control of the data processing flow. Specifically, the system continuously calculates or evaluates multiple core metrics, including but not limited to feature sparsity (characterizing feature quality), model update latency (reflecting model timeliness), and matching result stability (measuring matching effectiveness). The system presets performance or trigger thresholds for each metric. When the calculated value of a metric deviates from the preset threshold range (e.g., feature sparsity exceeds the optimization threshold, update latency exceeds the timeliness threshold), the system automatically triggers corresponding compensation or optimization processes, such as initiating feature recalculation, scheduling incremental model training, or adjusting the matching strategy, thereby forming a closed-loop self-optimizing system based on quantitative metric awareness. Attached Figure Description

[0030] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0031] Figure 1 This is a flowchart illustrating a data processing and control method based on user state events.

[0032] Figure 2 This is a detailed flowchart illustrating the multimodal feature fusion processing provided in the embodiments of this application;

[0033] Figure 3 This is a schematic diagram illustrating the structure and recommendation generation process of the deep semantic matching network provided in the embodiments of this application;

[0034] Figure 4 This is a schematic diagram of the model update process based on incremental learning provided in the embodiments of this application;

[0035] Figure 5 This is a schematic diagram of the module composition of a data processing and control device triggered by user status events;

[0036] Figure 6 This is a schematic diagram of the architecture of the data processing system provided in the embodiments of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0039] Example 1: Data Processing and Control Method Based on User Status Event Triggering

[0040] This embodiment provides a data processing and control method based on user state event triggering. Its core concept lies in achieving a paradigm shift from static matching to dynamic adaptation and from shallow features to deep understanding through integrated data pipelines, a fused algorithm engine, and a dynamic adjustment mechanism. This method systematically solves the technical problem of lagging data processing model updates through a "event-awareness-dynamic update" control closed loop. (Reference) Figure 1 The method includes the following steps 110-140:

[0041] Step 110: Obtain the original dataset of the target user.

[0042] This step provides the system with raw, multi-source user data. Here, the target user refers to an individual entity requiring personalized data processing services, such as insurance customers or e-commerce users. The raw dataset refers to a collection of data directly obtained from various data sources without deep processing. The system can acquire multi-dimensional user data through various channels (such as apps, web pages, health monitoring devices, etc.), including but not limited to: basic information: age, gender, occupation, income level; behavioral data: browsing history, consultation history, purchase history; health data: physical examination reports, medical history, lifestyle habits; financial status: income, expenditure, asset allocation. This data constitutes the raw material for subsequent feature engineering and model analysis. In general data processing scenarios, the raw dataset can cover any observational data related to the entity's state.

[0043] Data cleaning and feature engineering. Using big data processing frameworks (such as Apache Spark or Hadoop), the raw data is cleaned and standardized to ensure data quality. In feature engineering, the following key features are extracted:

[0044] User spending power: calculated using income and expenditure data.

[0045] User risk preference: based on historical behavioral data (such as whether they tend to buy high-risk investment products).

[0046] Health risk factors: such as BMI index, history of chronic diseases, etc.

[0047] Step 120: Listen to the real-time data stream related to the target user, and detect the target user's state change events based on predefined event rules.

[0048] This step establishes the foundation for the system's "dynamic adaptability" and is crucial for achieving real-time response. The system has a built-in real-time monitoring module that dynamically adjusts recommendations based on changes in the user's life. Specifically, the system continuously monitors real-time data streams related to the target user, which can originate from application logs, database change logs, sensor streams, etc. The state change events refer to predefined data patterns or signals that characterize significant changes in the user's or entity's state. For example, in an insurance scenario, these could be predefined as major life events such as "marriage," "childbirth," or "career change." The detection mechanism is based on these predefined event rules, which can be simple logical judgments (such as a field value changing from "unmarried" to "married") or more complex pattern recognition or lightweight classifiers. The technical effect of this step is to transform continuous data streams into discrete, meaningful event signals, providing a basis for triggering subsequent update processes.

[0049] Step 130: In response to the detection of the state change event, trigger a dynamic update of the user feature vector of the target user, and based on the new data corresponding to the state change event, perform incremental training on the preset data processing model to update its model parameters.

[0050] Once a state change event is detected, the system immediately (or according to a predetermined strategy) triggers two key technical actions:

[0051] Step 130A: Trigger dynamic updates to the user feature vector.

[0052] This means the system will recalculate or adjust the feature vector used to represent the user to reflect their latest state. This ensures that the data input to the downstream model is "fresh." For example, when a user gets married, their "family structure" related features need to be updated.

[0053] Step 130B: Based on the new data corresponding to the state change event (i.e., the data carried by the event itself and the associated data that it triggers), perform incremental training on the preset data processing model to update its model parameters.

[0054] The data processing model here can be any machine learning model used for risk assessment, preference prediction, classification, or regression. Through incremental training (rather than full retraining), the model can quickly absorb new knowledge and adapt to new data distributions with lower computational overhead. For example, when a large number of users give birth, the model should be able to associate these users with "education savings insurance" products more quickly.

[0055] This step transforms model and feature updates from passive, periodic batch processing to proactive, event-driven real-time processing through an event-triggered mechanism, thereby directly optimizing the two technical metrics of "model update latency" and "incremental update computational overhead".

[0056] Step 140: Based on the updated user feature vector and the updated data processing model, generate processing result data for the target user.

[0057] This step is the output stage of the process. Using dynamically updated data and models, it generates a processing result that best matches the user's current state. The processing result data varies depending on the specific application scenario and can be predicted tags (such as risk levels), ranking lists (such as product recommendations), decision suggestions, or any other form of structured output. For example, in an insurance recommendation scenario, this step outputs a personalized list of life insurance product recommendations, ensuring that the user is recommended the life insurance product that best suits their individual needs.

[0058] Through the closed loop formed by the above four steps, this application realizes the function of dynamically adjusting the recommendation strategy to cope with changes in users' lives and ensuring that the recommendations always match the actual needs of users.

[0059] Example 2: Processing method including multimodal feature fusion (solving feature sparsity)

[0060] This embodiment is a detailed expansion of step 130A in Embodiment 1, specifically describing a sub-process for solving the problem of user feature sparsity through multimodal feature fusion. The concept is to construct a more comprehensive and accurate user profile through multi-source data fusion and in-depth analysis. Figure 2 The detailed flowchart of the multimodal feature fusion processing provided in the embodiments of this application illustrates how to gradually generate a unified low-sparse user feature vector from the original heterogeneous data, including the processing processes such as text encoding layer 210, sequence encoding layer 220, structured data processing layer 230, feature splicing layer 240, and optimization and dimensionality reduction layer 250.

[0061] In one embodiment of this application, the processing result data is a personalized recommendation list. The original dataset includes at least structured attribute data, unstructured text data, and time-series behavior log data.

[0062] The triggering of dynamic updates to the user feature vector includes: performing multimodal feature fusion processing on the original dataset to generate a unified user feature vector with low sparsity.

[0063] The multimodal feature fusion processing includes: performing word embedding encoding on the unstructured text data to generate a first dense vector, performing sequence pattern encoding on the time-series behavior log data to generate a second dense vector, and concatenating and reducing the dimensionality of the first dense vector, the second dense vector, and the standardized structured attribute data.

[0064] The following section will explain this step in detail using specific scenarios for insurance recommendations.

[0065] The text data encoding layer performs word embedding encoding on unstructured text data to generate a first dense vector. This unstructured text data includes user consultation records, product reviews, social media activity, etc. Traditional methods (such as TF-IDF) produce high-dimensional sparse vectors. This application employs word embedding techniques, such as Word2Vec, GloVe, or Transformer-based pre-trained models (such as BERT), to map text into low-dimensional, dense semantic space vectors. In this way, semantically similar words or sentences are close in distance within the vector space. For example, user consultations on "retirement planning" and "retirement life security" will result in semantically similar vector representations.

[0066] The sequence data encoding layer performs sequence pattern encoding on the time-series behavior log data to generate a second dense vector. This time-series behavior log data includes timestamped records of user behavior, such as browsing paths, click sequences, and purchase history. This application employs sequence pattern encoding techniques, such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or Transformer encoders, to capture temporal dependencies and long-term preference patterns in user behavior. The final hidden state or pooled output of the encoder serves as the second dense vector representing the user behavior sequence. For example, analyzing a user's sequence from "browsing health insurance" -> "checking BMI knowledge" -> "consulting about critical illness insurance" can encode the user's attention to health risks.

[0067] The structured data processing layer implements the standardization of structured attribute data: the structured attribute data, such as age, income, occupation category, etc., needs to be standardized (such as Z-score standardization) or normalized to eliminate the influence of dimensions and make it suitable for mathematical operations with dense vectors.

[0068] The feature concatenation layer concatenates the first dense vector, the second dense vector, and the standardized structured attribute vector generated above to form a higher-dimensional intermediate feature vector.

[0069] Subsequently, the optimization and dimensionality reduction layer employs dimensionality reduction techniques (such as Principal Component Analysis (PCA) and Autoencoder) to remove redundant information and reduce noise, ultimately generating a fixed-dimensional, low-sparse "unified user feature vector." This vector is the technical representation of the user profile in the computer and forms the basis for all subsequent calculations.

[0070] In a further optimized embodiment of this application, the system is configured with a feature quality monitoring submodule. This module periodically calculates the average feature sparsity (FS) and average feature information entropy (FE) of newly generated batches of user feature vectors. The system sets quality warning thresholds and optimization trigger thresholds for FS and FE, respectively. When the calculated FS is consistently higher than the warning threshold, or the FE is consistently lower than the warning threshold, the monitoring submodule will issue an alarm. If the indicators further deteriorate and reach the optimization trigger threshold, the system will automatically trigger a readjustment of the parameters of the feature fusion pipeline, or start a new round of clustering and feature selection processes based on the latest data, until the feature quality indicators return to the normal range. This ensures that the features produced by the user profile construction process always meet the requirements of downstream models for the quality of input data.

[0071] Compared to existing technologies, this data processing method delves deeper into multi-dimensional user data, encompassing browsing history, consultation records, purchase history, health status, family structure, and other rich information to construct a more comprehensive and accurate user profile. This expansion of data depth and breadth enables the recommendation system to more accurately grasp users' personalized needs.

[0072] Furthermore, in order to optimize the feature set and improve its relevance and information content, the multimodal feature fusion processing of the original dataset also includes: using a clustering algorithm to segment the time-series behavioral log data into user groups and generating behavioral pattern label features.

[0073] User profiling is one of the core components of the entire system, primarily relying on clustering and classification algorithms. For example, K-means clustering is used to group users. The goal is to minimize the sum of squared errors within each cluster, and its loss function is defined as:

[0074]

[0075] Where: K is the number of clusters. It is the i-th cluster. The i-th cluster is the center of the i-th cluster. By optimizing the above loss function, the optimal clustering result is found, forming different user groups. Based on the clustering results, user profiles are further refined. For example, some groups may be more concerned about health insurance, while others may focus more on retirement planning. The "group ID" or "behavioral pattern label" generated after clustering can be added to the user feature vector as a new classification feature, enhancing the semantic information of the features.

[0076] An information gain-based feature selection algorithm is employed to filter a subset of features from the concatenated feature set that are more relevant to a preset target than a preset threshold, thereby updating the unified user feature vector. During feature selection, information gain or the chi-square test is used to select the most important features. Here, the "preset target" in a recommendation scenario could be "whether to purchase a certain type of insurance product." By calculating the information gain between each feature and the target, features that contribute significantly to the prediction target are retained, while redundant or irrelevant features are removed. This further reduces the feature dimensionality, improves model training efficiency and generalization ability, and constitutes the final, optimized unified user feature vector.

[0077] Example 3: Recommendation generation method including deep semantic matching (solving matching coarseness)

[0078] This embodiment is a detailed elaboration of the process of generating processing result data described in step 140 of Embodiment 1, specifically describing how to solve the matching coarseness problem through a deep semantic matching network. Its concept lies in introducing deep learning algorithms and natural language processing techniques to improve the intelligence level of recommendations. Figure 3 The diagram illustrates the structure and recommendation generation process of the deep semantic matching network provided in this embodiment of the application, showing the dual-tower structure of the user tower 310 and the object tower 320, the semantic space similarity calculation layer 330, and the sorting list generation layer 340.

[0079] The generation of processing result data includes: using a deep semantic matching network to calculate the matching degree between the user feature vector of the target user and the feature vectors of multiple objects to be matched, and generating a sorted list as the processing result data based on the matching degree.

[0080] The deep semantic matching network uses a dual-tower neural network structure to map user-side features and object-side features to a high-dimensional semantic space before calculating similarity.

[0081] The following is a step-by-step explanation:

[0082] In a broad sense of data processing, the object to be matched can be any entity that needs to be evaluated for its relevance to the user, such as products, articles, services, and advertisements. The deep semantic matching network is a type of deep learning model used to learn complex matching relationships between two inputs. Its core is to learn a mapping function from the original feature space to the semantic space, making related entities closer in that space.

[0083] Network architecture (dual-tower neural network): This is a common and effective architecture for implementing deep semantic matching. For example... Figure 3 As shown, the network comprises two structurally identical or similar subnetworks (referred to as "towers"). One tower (the user tower) takes the unified user feature vector generated in Example 2 as input; the other tower (the object tower) takes the feature vector of the object to be matched as input. The object feature vector can be generated through a similar feature engineering process, for example, by embedding and encoding words into the terms and conditions of an insurance product, combined with its structured attributes (such as premium, sum assured, and coverage period). Each tower typically consists of multiple fully connected layers 312 / 322, and may also contain embedding layers 311 / 321 for processing discrete features. The outputs of the two towers are two semantic vectors (or "embedding vectors") with the same dimensions.

[0084] Semantic Space and Similarity Calculation 330: The role of the two towers is to map users and objects from their respective original high-dimensional, heterogeneous feature spaces to a shared, more semantically meaningful high-dimensional semantic space. In this shared space, the geometric distance between vectors (such as cosine similarity and Euclidean distance) directly reflects their semantic relevance. For example, a user vector focusing on family health and a critical illness insurance product vector will have a high cosine similarity in this space. The network learns this mapping relationship through training.

[0085] Matching and Ranking 340: For a given user, calculate the similarity between their semantic vector and the semantic vectors of all objects in the candidate pool to obtain a set of matching scores. Sort these scores in descending order, and the resulting ranked list is the processing result data (such as a Top-N recommendation list).

[0086] In recommendation algorithms, deep learning algorithms are used to learn user behavior patterns and product characteristics in a more complex way, and natural language processing technology is used to deeply understand and analyze the text terms of life insurance products, thereby achieving more intelligent and accurate recommendations.

[0087] As a further optimization, to ensure the stability of the recommendation service, the system includes a matching stability monitor. For consecutive requests from the same user within a certain time window (where no significant changes in their underlying features have occurred), the monitor calculates the stability (MS) of the matching results between the successively obtained recommendation lists, such as Jaccard similarity. The system presets a minimum stability threshold. If the calculated MS value consistently falls below this threshold, it indicates abnormal fluctuations in the output of the deep semantic matching network, which may be due to noise interference or model instability. In this case, the system will automatically trigger at least one of the following operations: 1) Enable result caching for the current user's request; 2) Perform output smoothing processing on the matching network for that user dimension; 3) Record the anomaly and trigger a health diagnosis and fine-tuning of the matching network. This ensures a consistent user experience.

[0088] As a further optimized embodiment, in order to train a deep semantic matching network with strong discriminative ability, this embodiment further specifies that: the training process of the deep semantic matching network adopts a contrastive loss function, which is used to increase the semantic similarity score between the positive sample object and the target user, and reduce the gap in semantic similarity score between the difficult negative sample object and the target user.

[0089] It's important to note that during model training, positive samples refer to objects that the user has actually interacted with (such as clicks or purchases), while negative samples refer to objects that the user has not interacted with or has interacted with negatively. Simple negative sampling may lead the model to learn simple patterns. Hard negative samples are those that are similar to positive samples in terms of original features or shallow semantics, but which the user did not select. Training objectives such as contrastive loss or triplet loss not only require bringing the user closer to positive samples but also pushing the user away from hard negative samples. This forces the model to learn more refined and discriminative semantic features, thereby improving semantic similarity discrimination and making the final matching results more accurate and stable.

[0090] It should be noted that the deep semantic matching network described in this embodiment is a preferred implementation of the present invention, but the present invention does not exclude compatibility and integration with other recommendation algorithms. In practical applications, especially in the cold start phase with sparse data or in edge scenarios requiring lightweight computation, the system can also use traditional collaborative filtering algorithms or logistic regression models as supplements.

[0091] For example, user-based collaborative filtering algorithms can be used to calculate the similarity between users, and the similarity metric can be cosine similarity:

[0092]

[0093] in, and Representing two different users, It is a collection of products that the two parties have interacted with. Indicates user For the product The ratings or implicit feedback. Collaborative filtering predicts a user's preference for untouched products by finding neighboring users whose behavioral patterns are most similar to those of the target user.

[0094] Another similarity metric is the Pearson correlation coefficient.

[0095]

[0096] in, and Representing users respectively and users The average rating. The Pearson correlation coefficient, by subtracting users' own average rating habits, can better eliminate bias caused by inconsistent user rating scales.

[0097] For example, logistic regression models can serve as a lightweight implementation of content-based recommendation, used to predict the probability of a user's preference for a specific product. Its prediction function is defined as:

[0098]

[0099] in, For the feature vectors of users and products, For weight parameters, This is the bias term. The model is trained using the cross-entropy loss function:

[0100]

[0101] In actual deployment, the system can dynamically switch or integrate the above algorithms and deep semantic matching networks according to the amount of data, computing resources and real-time requirements, so as to achieve the best balance between recommendation accuracy and computational efficiency.

[0102] Example 4: Model update method including a specific incremental learning mechanism

[0103] This embodiment is a detailed expansion of step 130B in embodiment 1, specifically describing the technical implementation of model updating in solving the model lag problem. Figure 4 The schematic diagram of the incremental learning-based model update process provided in the embodiments of this application shows the complete process from the state change event detection 410 trigger, through the associated data retrieval 420, loading the existing model 430, performing incremental training 440 to the verification update 450 and the formation of the feedback loop 460.

[0104] The data processing model is a gradient boosting decision tree model that supports incremental learning;

[0105] The incremental training of the data processing model specifically includes:

[0106] Step 410: Time detection of state variables;

[0107] Step 420: Retrieve contextual data associated with the state change event from historical data to form an incremental training dataset;

[0108] To construct the incremental training dataset, when a state change event is detected (such as "a user purchased a basic life insurance policy"), the system does not simply train using only this new record. Instead, it retrieves contextual data associated with the state change event from historical data. For example, it retrieves all recent browsing and consultation records related to life insurance for that user, as well as purchase behavior data of other users with similar policies. This data collectively constitutes the "incremental training dataset." The purpose of this is to allow the model to learn not only the new event itself during updates but also its patterns within a richer context, avoiding overfitting or bias caused by a single sample.

[0109] Step 430: Load the existing model;

[0110] In some embodiments, Gradient Boosting Decision Tree (GBDT) is employed as the risk assessment model (or, more generally, the data processing model). GBDT is an ensemble learning algorithm based on decision trees, exhibiting good nonlinear fitting capabilities and robustness. More importantly, many efficient implementations of GBDT (such as XGBoost and LightGBM) support incremental learning or online learning, meaning that a trained model can be updated with new data via partial_fit or a similar interface without retraining from scratch. This provides the algorithmic foundation for achieving low-latency updates.

[0111] Step 440: Perform incremental training;

[0112] The gradient boosting decision tree model is partially fitted using the incremental training dataset, and a regularization term is introduced to prevent overfitting.

[0113] For example, using a pre-constructed incremental training dataset, a partial fitting method is applied to an existing GBDT model. This method adjusts the parameters of the existing model (mainly the weights of the leaf nodes of the trees or adding weak learners) with a small learning rate based on the new data. Simultaneously, to prevent overfitting to the new mini-batch data, regularization terms (such as L1 / L2 regularization, maximum tree depth, minimum number of samples per leaf node, etc.) are maintained or fine-tuned during incremental training. Model training and optimization optimize the model hyperparameters (such as the number and depth of trees) through cross-validation and grid search. Regularization terms are also introduced to prevent overfitting. This process significantly reduces computational cost (R) compared to full retraining.

[0114] The goal of GBDT is to optimize model parameters by minimizing the loss function. For binary classification problems (such as whether a user has a high health risk), the log loss function (LogLoss) is used.

[0115]

[0116] in: It is a real label (0 or 1). This represents the predicted probability. N is the sample size. For regression problems (such as predicting a user's financial risk index), the mean squared error (MSE) is used as the loss function:

[0117]

[0118] In incremental learning, the loss function is calculated based on the incremental dataset.

[0119] In a further optimized embodiment of this application, the system includes an incremental update management module responsible for monitoring the performance of each incremental training iteration. It calculates two key metrics: the actual model update latency (L) and the performance retention rate (or AUC change) of the updated model on the validation set. The system presets a maximum allowable latency threshold and a minimum performance retention threshold. If the actual update latency L exceeds the maximum allowable threshold, the system may choose a lighter update strategy or adjust computing resources upon the next event trigger. If the updated model performance retention rate is lower than the minimum performance retention threshold, the incremental update is deemed to have failed or introduced noise. The system automatically rolls back to the previous model version and may trigger a 'consolidation training' containing more historical data. This judgment mechanism based on both latency and performance thresholds ensures that model updates are both fast and reliable.

[0120] Step 450: Verification and Update;

[0121] After performing partial fitting, the system typically performs a rapid evaluation of the updated model's performance on an independent validation set or through cross-validation to verify whether its key metrics (such as AUC and accuracy) have significantly decreased. If the validation passes, the version identifier in the model repository is updated, and the new model parameters are hot-deployed to the online inference service. If the performance degradation exceeds the tolerance threshold, a rollback mechanism may be triggered to revert to the previous stable version, or a retraining process with more historical data may be initiated.

[0122] Step 460: Feedback loop closed;

[0123] The updated model will be immediately used to process new user requests, and its generated predictions (such as recommendation lists) will be displayed to the user. Subsequent user interactions (such as clicks, purchases, and ignores) will be collected by the system in real time, becoming a source of new 'state change events' or 'incremental data', thereby triggering the incremental learning process described in this embodiment again. In this way, 'event detection -> data retrieval -> incremental training -> validation and update -> new data generation' constitutes a continuously self-optimizing and dynamically adaptable technical closed loop, ensuring that the system can respond to changes in user needs over a long period of time.

[0124] Example 5: A method including an interpretability optimization process

[0125] This embodiment adds a post-processing optimization process after the process described in Embodiment 1 is completed, aiming to improve the transparency and credibility of the system output. Existing technologies lack the ability to generate recommendation reasons. The optimization point of this application is: after generating the processing result data, it further includes the following steps 501-502:

[0126] Step 510: Based on causal inference technology, analyze the decision contribution of each feature dimension in the user feature vector to the generation of any output item in the processing result data.

[0127] It should be noted that traditional feature importance analysis (such as SHAP value and LIME) is more of a correlation analysis. This application proposes to use causal inference techniques to isolate confounding factors and estimate the causal effect of a specific user characteristic (such as "browsing education savings insurance 3 times in the past week") on the final recommendation of a certain product (such as "a certain education annuity insurance"). This can more reliably identify the key factors that truly drive the model's decision.

[0128] Step 520: Generate interpretable explanatory text corresponding to the output item based on the decision contribution.

[0129] It should be noted that the contribution score of each feature is calculated using a causal inference model. Then, based on a pre-set template or natural language generation model, these key features and their contributions are transformed into human-readable and interpretable explanatory text. For example: "We recommend 'Future Star Education Annuity Insurance' to you mainly because: 1. Your recent browsing history shows a high level of concern for your children's education planning (contribution 35%); 2. Your family structure includes school-age children (contribution 28%); 3. Your risk assessment shows good long-term financial stability (contribution 20%)."

[0130] This explanation not only enhances user experience and trust, but also makes it easier for business personnel to understand and review the model's decision-making logic.

[0131] Example 6: A method including a cold start optimization process

[0132] This embodiment addresses the data sparsity problem of new users (or more broadly, new entities) by adding a preprocessing optimization process before the main process of Embodiment 1. Before obtaining the original dataset of the target users, a cold start processing step for newly registered users is also included:

[0133] Step 610: Obtain the initial interaction sequence generated by the newly registered user within a preset time window;

[0134] "Cold start" refers to the situation where new users lack historical behavioral data, making it impossible to generate effective feature vectors through conventional feature engineering processes, resulting in poor initial recommendation quality.

[0135] To achieve data acquisition, the system defines a "preset time window" (such as 24 hours after registration or the first 10 sessions after initial login), focusing on collecting the user's initial interaction sequence within this window. Although the amount of data is small, it represents the user's initial interests and intentions.

[0136] Step 620: Input the initial interaction behavior sequence into the fast adaptation model trained by meta-learning. The fast adaptation model adjusts its internal parameters based on the initial interaction behavior sequence and outputs the initial feature vector of the newly registered user for initial processing.

[0137] In one embodiment, the system performs meta-learning training offline. The goal is to train a "fast-adaptive model" that learns "how to quickly learn the features of a new user from a very small number of samples." When a new user arrives, their initial interaction sequence is input into the model. Internally, the model performs several steps of rapid gradient descent or parameter tuning, then outputs a reasonable initial feature vector. This vector can serve as a temporary representation of the user before accumulating sufficient data, used to execute the subsequent steps of Embodiment 1 to generate the initial recommendation.

[0138] As new user interaction data increases, the system will gradually replace this initial vector with the feature vector generated by the standard feature fusion process in Example 2, achieving a smooth transition.

[0139] Example 7: A method including a privacy-preserving training framework

[0140] This embodiment provides an optimized scheme to replace or enhance the model training process in embodiment 4, focusing on collaborative training while protecting data privacy.

[0141] The training process of the data processing model or deep semantic matching network can adopt a federated learning framework, including steps 131-133:

[0142] Step 131: Calculate the update gradient of the model parameters locally on multiple clients using private data;

[0143] For example, federated learning is a distributed machine learning paradigm suitable for data privacy-sensitive scenarios (such as where user health data held by different insurance branches or partner hospitals cannot be centralized). Each client (such as a branch office server) holds its local private data. During training rounds, the client downloads the latest global model and uses its own private data to calculate the gradient updates for the model parameters. The original data is always kept locally and never leaves the client.

[0144] Step 132: Upload the encrypted update gradient to the central server;

[0145] For example, the client encrypts the calculated gradient (such as with homomorphic encryption or differential privacy noise) before uploading it to a central server. The central server then aggregates (e.g., by weighted averaging) all the encrypted gradients uploaded by the clients.

[0146] Step 133: The central server aggregates the update gradients to update the global model and distributes the updated global model parameters to each client.

[0147] For example, the server uses the aggregated gradients to update the global model, and then distributes the updated model parameters to each client to complete one iteration.

[0148] This framework can be used to train the GBDT model in Example 4 (which requires federated learning support) or the deep semantic matching network in Example 3. This allows the solution to utilize a wider range of data for model training while strictly adhering to data privacy regulations (such as GDPR), thereby improving model performance.

[0149] Example 8: Data processing and control device based on user status event triggering

[0150] This embodiment provides a hardware or software device corresponding to the methods described in Embodiments 1-7. (See reference...) Figure 5 The device 800 includes the following modules, each with a function corresponding to the steps described above:

[0151] The data acquisition module 81 is used to acquire the original dataset of the target user; this module is configured to perform the operation as described in step 110 of embodiment 1.

[0152] The event monitoring module 82 is used to listen to the real-time data stream related to the target user and detect the state change event of the target user based on predefined event rules. This module is configured to implement the operation described in step 120 of embodiment 1 and is a key component for the system to achieve dynamic adaptability.

[0153] The incremental update module 83 is used to respond to the detection of the state change event, trigger a dynamic update of the user feature vector of the target user, and perform incremental training on a preset data processing model based on the new data corresponding to the state change event to update its model parameters. This module is the core control unit, configured to implement operations such as step 130 in Embodiment 1 and the related detailed steps in Embodiments 2, 4, and 7. It may internally integrate a feature fusion submodule, an incremental training submodule, etc.

[0154] Preferably, the incremental update module 83 further includes the following functions: monitoring metrics such as model update latency (L) and performance retention rate during or after incremental training; determining whether the update is effective based on preset latency and performance thresholds; and triggering control actions such as model rollback or training consolidation. This module integrates the monitoring and triggering logic of the update management module.

[0155] The recommendation generation module 84 is used to generate processing result data for the target user based on the updated user feature vector and the updated data processing model. This module is configured to implement the operations described in step 140 of Embodiment 1 and the related detailed steps in Embodiments 3 and 5. It can integrate a deep semantic matching network and an interpretation generator.

[0156] Preferably, the recommendation generation module 84 further includes the following functions: while generating the recommendation list, calculating indicators such as matching result stability (MS); determining whether the output is abnormal based on a preset stability threshold, and triggering control actions such as result caching, output smoothing, or model diagnosis. This module integrates the logic of the stability monitor.

[0157] More preferably, the feature quality monitoring (such as FS, FE) logic can be integrated into the feature fusion pipeline responsible for feature generation (which can be called by the incremental update module or an independent feature service module), or implemented by an independent feature quality monitoring submodule, which works in conjunction with the incremental update module 83.

[0158] Example 9: Data Processing System

[0159] This embodiment provides a specific system architecture for deploying the aforementioned method and apparatus. (Refer to...) Figure 6 The data processing system 900 includes a terminal device 91 and a server 92.

[0160] The terminal device 91 is used to collect raw user data and send it to the server, as well as to receive and display the processing result data from the server. For example, in an insurance scenario, the terminal device can be a mobile app or web browser used by the policyholder, responsible for collecting the user's clicks, inputs, authorization information, etc., and displaying the final insurance product recommendation list and explanation.

[0161] The server 92 is used to run the data processing and control device 800 based on user status events as described in Embodiment 8, and / or execute the method according to any one of Embodiments 1 to 7 to generate the processing result data and send it to the terminal device 91 or the associated business system. The server is the core computing unit, carrying all data processing, model training, and matching calculation tasks. The overall architecture of the system can be divided into the following modules: data collection module, data analysis and processing module, user profile construction module, risk assessment and product matching module, and real-time monitoring and adjustment module. Each module contains specific algorithms and technical details. Furthermore, it may also include the quality monitoring sub-module, update management module, and stability monitor mentioned above, including the above-mentioned automated monitoring, judgment, and triggering control process based on metric thresholds. The server can be a distributed cluster, including a data layer, service layer, business logic layer, etc., to support high concurrency and real-time processing.

[0162] It should be noted that the execution entities of each step in the method provided in each embodiment can be the same device, or the method can be executed by different devices. For example, the event monitoring module may be deployed on edge computing nodes to reduce latency, while complex incremental model training and depth matching may be completed by a cloud central server.

[0163] Other embodiments relating to computer-readable storage media and electronic devices:

[0164] This application also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of embodiments 1 to 7 of this application.

[0165] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] Therefore, this application also proposes a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the methods described in any embodiment of this application.

[0167] Furthermore, this application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method as described in any one of embodiments 1 to 7 of this application. This electronic device can be the aforementioned server 602, or it can be a dedicated computing device integrating some of the functions.

[0168] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, a network interface, and memory. Memory may include non-persistent storage in computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media. Computer-readable media includes both permanent and non-persistent, removable and non-removable media; information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of a program, or other data.

[0169] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0170] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of this application means the presence of the stated feature, integer, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The term “and / or” as used herein includes all or any unit and all combinations of one or more associated listed items.

[0171] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical, terminological, and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0172] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A data processing and control method based on user state event triggering, characterized in that, include: Obtain the raw dataset of the target users; Monitor real-time data streams related to the target user and detect state change events of the target user based on predefined event rules; In response to the detection of the state change event, a dynamic update of the user feature vector of the target user is triggered, and based on the new data corresponding to the state change event, the preset data processing model is incrementally trained to update its model parameters. Based on the updated user feature vector and the updated data processing model, processing result data for the target user is generated.

2. The method according to claim 1, characterized in that, The processing result data is a personalized recommendation list; The original dataset includes at least structured attribute data, unstructured text data, and time-series behavior log data; The triggering of dynamic updates to the user feature vector includes: The original dataset is subjected to multimodal feature fusion processing to generate a low-sparseness unified user feature vector; wherein, the multimodal feature fusion processing includes: performing word embedding encoding on the unstructured text data to generate a first dense vector, performing sequence pattern encoding on the time-series behavior log data to generate a second dense vector, and concatenating and reducing the dimensionality of the first dense vector, the second dense vector, and the standardized structured attribute data.

3. The method according to claim 2, characterized in that, The multimodal feature fusion processing of the original dataset further includes: Clustering algorithms are used to segment users from the time-series behavioral log data and generate behavioral pattern label features. An information gain-based feature selection algorithm is used to select a subset of features from the concatenated feature set that are more relevant to a preset target than a preset threshold, so as to update the unified user feature vector.

4. The method according to claim 1, characterized in that, The generated processing result data includes: A deep semantic matching network is used to calculate the matching degree between the user feature vector of the target user and the feature vectors of multiple objects to be matched, and a sorted list is generated based on the matching degree as the processing result data; wherein, the deep semantic matching network uses a dual-tower neural network structure to map user-side features and object-side features to a high-dimensional semantic space before performing similarity calculation.

5. The method according to claim 4, characterized in that, The training process of the deep semantic matching network adopts a contrastive loss function, which is used to increase the semantic similarity score between positive sample objects and the target user, and reduce the difference in semantic similarity score between difficult negative sample objects and the target user.

6. The method according to claim 1, characterized in that, The data processing model is a gradient boosting decision tree model that supports incremental learning; the incremental training of the data processing model specifically includes: Contextual data associated with the state change event is retrieved from historical data to form an incremental training dataset; The gradient boosting decision tree model is partially fitted using the incremental training dataset, and a regularization term is introduced to prevent overfitting.

7. The method according to claim 1, characterized in that, Finally, the following steps are also included: Based on causal inference technology, the contribution of each feature dimension in the user feature vector to the decision-making of generating any output item in the processing result data is analyzed. Based on the decision contribution, generate interpretable explanatory text corresponding to the output item; And / or, Prior to the steps described above, a cold start processing step for newly registered users is also included: Obtain the sequence of initial interactive behaviors generated by the newly registered user within a preset time window; The initial interaction behavior sequence is input into a fast adaptation model trained by meta-learning. The fast adaptation model adjusts its internal parameters based on the initial interaction behavior sequence and outputs the initial feature vector of the newly registered user for initial processing.

8. The method according to claim 1, characterized in that, The training process of the data processing model or deep semantic matching network adopts a federated learning framework, including: Calculate the update gradient of model parameters locally on multiple clients using private data; The encrypted update gradient is uploaded to the central server; The central server aggregates the update gradients to update the global model and distributes the updated global model parameters to each client.

9. A data processing and control device based on user state event triggering, used to implement the method according to any one of claims 1 to 8, characterized in that, include: The data acquisition module is used to acquire the raw dataset of the target user; The event monitoring module is used to listen to real-time data streams related to the target user and detect state change events of the target user based on predefined event rules. The incremental update module is used to respond to the detection of the state change event, trigger the dynamic update of the user feature vector of the target user, and perform incremental training on the preset data processing model based on the new data corresponding to the state change event to update its model parameters. The recommendation generation module is used to generate processing result data for the target user based on the updated user feature vector and the updated data processing model.

10. A data processing system, characterized in that, Including terminal devices and servers; The terminal device is used to collect user raw data and send it to the server, as well as to receive and display processing result data from the server; The server is configured to run the data processing and control device based on user state event triggering as described in claim 9, and / or execute the method according to any one of claims 1 to 8 to generate the processing result data table and send it to the terminal device or associated business system.

Images