A data operation analysis method and system based on machine learning
By distinguishing between parsing obstruction and regular missing states in the server cluster of the data governance platform, generating metadata identifiers and blocking the numerical imputation of invalid features and model training, the problem of downstream analysis link anomalies caused by upstream data structure changes is solved, improving the accuracy of machine learning models and the system's adaptability.
Patent Information
- Application Number
- CN202611610263.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-19
- Publication Date
- 2026-11-17
AI Technical Summary
Silent changes to the underlying data structure of upstream business systems cause hidden cascading anomalies in the downstream data access and analysis links, which in turn severely interfere with the training and inference of machine learning models.
By accurately distinguishing between parsing obstruction and regular missing states in the server cluster of the data governance platform, metadata identifiers are generated to block the numerical imputation process of parsing obstructed data items, feature variable masks and derived masks are generated to limit the contribution of invalid features to model loss calculation and parameter updates, and parsing rules are adaptively updated online by identifying parsing obstruction distributions.
It significantly improves the accuracy, robustness, and adaptability of the data operation and analysis system, reduces the risk of model failure, and improves the efficiency and reliability of data processing.
Smart Images

Figure CN123115170A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data operation and analysis technology, and more specifically, to a data operation and analysis method and system based on machine learning. Background Technology
[0002] An upstream routine upgrade changed the field from a single-level flat structure to a multi-level nested structure. The downstream parser did not update synchronously, and the new format collapsed into a continuous string of null values. The cleaning module, following the default rule of "using the global average as a fallback if the numeric field is missing," filled these null values with the same specific value. This effectively rewrote the semantics that originally represented "no behavior" into "average behavior."
[0003] During the feature engineering phase, the derived metrics exhibited significant artificial clustering around this mean. The screening module, using statistical correlation assessment, misclassified this spurious feature as a high-discrimination core feature, directly selecting it into the model and assigning it high weights. During training, the loss function was dominated by this distorted distribution, and backpropagation drastically pulled the weight matrix in the direction of fitting this cluster, ultimately causing the model to converge to an erroneous local extremum.
[0004] After deployment to production, the false cluster could no longer be found in the real data, the model output changed drastically, the operational strategy also failed, and the entire ML (Machine Learning) operational analysis chain basically became ineffective. Summary of the Invention
[0005] This application discloses a data operation analysis method and system based on machine learning, which aims to solve the problem that silent changes in the underlying data structure of upstream business systems cause hidden cascading anomalies in the downstream data access and analysis links, thereby seriously interfering with the training and inference of machine learning models.
[0006] The technical solution of this application is as follows: In a first aspect, this application discloses a data operation analysis method based on machine learning, applied to a server cluster of a data governance platform, the method comprising: The system acquires the original input data. When the scalar value of the target data item cannot be extracted from the original input data according to the preset data item parsing rules, it performs structural feature analysis on the text fragments in the original input data that correspond to the target data item in order to distinguish between the parsing blocked state and the normal missing state of the target data item, and generates metadata identifiers for the target data item that indicate the parsing blocked state or the normal missing state. Identify target data items in a parsing blocked state based on metadata identifiers, prevent target data items from entering the numerical imputation process, and generate feature variable mask information to characterize the validity of each target data item; generate derivative mask information for derived statistical indicators containing target data items based on feature variable mask information, and combine feature variable mask information and derivative mask information to screen the validity of candidate feature variables; remove candidate feature variables whose effective sample ratio is lower than a preset threshold from the candidate feature variable set after setting the correlation evaluation between the candidate feature variable and the business objective to zero; When training a machine learning model, based on the feature variable mask information and the derived mask information, the feature variable channels marked as invalid by the mask are restricted from contributing to the model loss calculation and the driving effect of the feature variable channels on the model parameter update is blocked. When receiving real-time business data, the system performs data item parsing and feature variable management on the real-time business data, and performs pattern recognition based on a machine learning model, outputting the pattern recognition results. Based on the pattern recognition results and the distribution of blocked data items in real-time business data, the data item parsing rules are adaptively updated, and the updated data item parsing rules are applied to subsequent data item parsing, forming an end-to-end closed loop of offline data governance and online pattern recognition.
[0007] Secondly, this application also discloses a data operation and analysis system based on machine learning, including a processor and a memory, wherein the memory stores computer program instructions, and the system includes: The data item parsing module, coupled with the processor, is used to acquire raw input data. When the scalar value of the target data item cannot be extracted from the raw input data according to the preset data item parsing rules, the structural feature analysis is performed on the text fragments in the raw input data corresponding to the target data item to distinguish between the parsing blocked state and the normal missing state of the target data item, and to generate metadata identifiers for the target data item indicating the parsing blocked state or the normal missing state. The feature variable governance module, coupled with the data item parsing module, is used to identify target data items in a parsing blocked state based on metadata identifiers, prevent target data items from entering the numerical imputation process, and generate feature variable mask information representing the validity of each target data item. Based on the feature variable mask information, it generates derivative mask information for derived statistical indicators containing target data items, and combines the feature variable mask information and the derivative mask information to screen the validity of candidate feature variables. After setting the correlation evaluation between candidate feature variables with a valid sample ratio lower than a preset threshold and the business objective to zero, they are removed from the candidate feature variable set. The model training module, coupled with the feature variable governance module, is used to limit the contribution of feature variable channels marked as invalid by the mask to the model loss calculation when training the machine learning model, based on the feature variable mask information and the derived mask information, and to block the driving effect of feature variable channels on model parameter updates. The pattern recognition module, coupled with the data item parsing module, feature variable management module, and model training module, is used to parse data items and manage feature variables in real-time business data when receiving real-time business data, and to perform pattern recognition based on the machine learning model output by the model training module, and output the pattern recognition results. The parsing rule update module, coupled with the pattern recognition module and the data item parsing module, is used to adaptively update the data item parsing rules based on the pattern recognition results and the distribution of parsing blocked data items in real-time business data. The updated data item parsing rules are then written back to the data item parsing module, forming an end-to-end closed loop between offline data governance and online pattern recognition.
[0008] This application addresses the issue of silent changes to upstream data structures causing downstream ML models to fail by designing a closed-loop governance system covering the entire chain of "access—features—training—inference".
[0009] In the access phase: Structural feature analysis is performed on the raw data to accurately distinguish between the two states of "parsing obstruction" and "normal missing values", and metadata identifiers are generated to avoid mistaking parsing failures as missing values and filling them with the mean.
[0010] Feature identification process: Based on the identifier, blocked items are identified and parsed, preventing them from entering the numerical imputation process, and feature variable masks and derived masks are generated; candidate features with a low proportion of effective samples are directly set to zero correlation and removed to prevent false features from entering the model.
[0011] Training phase: Use masks to limit the contribution of invalid features to loss calculation and parameter updates, ensuring that the model is not dominated by noise and converges to the real business distribution.
[0012] Inference phase: Online identification of parsing obstruction distributions in real-time data, and adaptive update of parsing rules accordingly, forming an end-to-end closed loop of offline governance and online identification.
[0013] Results: End-to-end data governance and model protection from access to inference significantly improves the system's accuracy, robustness, and adaptability. Attached Figure Description
[0014] Figure 1 This is a flowchart illustrating a data operation and analysis method based on machine learning provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a data operation and analysis system based on machine learning provided in an embodiment of the present invention. Detailed Implementation
[0015] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0016] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0017] Traditional data operation and analysis systems often encounter problems when handling silent changes to the data structure of upstream business systems. This is because the parsing rules of the data access module fail to update synchronously, resulting in unparseable composite fields being uniformly converted into continuous null records. These null values are frequently filled with global averages during data preprocessing, altering the true meaning of the original data, distorting the distribution of derived statistical indicators, and potentially introducing spurious correlations in feature selection and model training. Ultimately, this leads to a decline in the model's recognition capabilities in practical applications.
[0018] In this regard, refer to Figure 1 , Figure 1 This is a flowchart illustrating a data operation and analysis method based on machine learning provided in an embodiment of the present invention. The method includes: S11, Obtain the original input data. When the scalar value of the target data item cannot be extracted from the original input data according to the preset data item parsing rules, perform structural feature analysis on the text fragments in the original input data corresponding to the target data item to distinguish between the parsing blocked state and the normal missing state of the target data item, and generate metadata identifiers for the target data item indicating the parsing blocked state or the normal missing state. S12, based on the metadata identifier, identify the target data item in the parsing blocked state, block the target data item from entering the numerical imputation process, and generate feature variable mask information characterizing the validity of each target data item; based on the feature variable mask information, generate derivative mask information for the derived statistical indicators containing the target data item, and combine the feature variable mask information and the derivative mask information to screen the candidate feature variables for validity, and remove the candidate feature variables whose effective sample ratio is lower than a preset threshold from the candidate feature variable set after setting the correlation evaluation between the candidate feature variables and the business target to zero; S13, When training a machine learning model, based on the feature variable mask information and the derived mask information, the feature variable channels marked as invalid by the mask are restricted from contributing to the model loss calculation and the driving effect of the feature variable channels on the model parameter update is blocked. S14, when receiving real-time business data, perform data item parsing and feature variable management on the real-time business data, and perform pattern recognition based on the machine learning model, and output the pattern recognition result; S15, based on the pattern recognition results and the distribution of blocked data items in the real-time business data, the data item parsing rules are adaptively updated, and the updated data item parsing rules are applied to subsequent data item parsing, forming an end-to-end closed loop of offline data governance and online pattern recognition.
[0019] To better understand the method proposed in this application, it is necessary to explain some of the key terms involved.
[0020] "Raw input data" refers to various types of unprocessed data accessed from upstream business systems, which may include structured, semi-structured, or unstructured information.
[0021] "Target data items" refer to specific data fields that need to be extracted and used in data operation analysis, such as user ID, transaction amount, behavior type, etc.
[0022] "Scalar numerical value" refers to a single numerical value that can be directly used for numerical calculations or statistical analysis, as opposed to complex text or compound structures.
[0023] "Parsing blocked state" refers to a situation where a data item cannot be correctly extracted as a scalar value due to changes in the underlying data structure or mismatch in parsing rules.
[0024] "Normal missing state" means that the data item itself is empty or does not exist in the original data, and is not caused by a parsing problem.
[0025] "Metadata identifier" is a tag generated for each target data item to indicate whether it is in a parsing blocked state or a normal missing state.
[0026] "Feature variable mask information" is a binary or multi-valued tag that characterizes the validity of each target data item and is used to indicate whether the data item is valid.
[0027] "Derivative mask information" is a mask generated for derived statistical indicators based on feature variable mask information, reflecting the effectiveness of the derived indicators.
[0028] "Feature variable path" refers to the computation path in a machine learning model that corresponds to a specific feature variable input.
[0029] "Model loss calculation" refers to the calculation process used to measure the prediction error of a machine learning model during training.
[0030] "Model parameter update" refers to the process of adjusting the internal weights and biases of the model based on the loss calculation results during model training.
[0031] "Data item parsing rules" refers to a set of preset rules used to extract target data items from raw input data.
[0032] "Adaptive update" refers to the process of dynamically adjusting the data item parsing rules based on real-time business data and pattern recognition results.
[0033] This method is applied to the server cluster of the data governance platform, which means that the entire processing flow takes place in a distributed computing environment to handle massive amounts of data and support high-concurrency real-time business needs.
[0034] Specifically, the implementation chain of this solution is as follows: Access parsing: Attempt to extract target fields from raw data according to preset rules; when parsing fails, perform structural feature analysis (regular expression matching or lexical analysis) on the corresponding text fragment, identify abnormal patterns such as nested structures, distinguish between "parsing blocked" and "normal missing", and record them with metadata.
[0035] Blocking and mask generation: Based on the identifier, the blocked item is identified and parsed, and its entry into the mean, median and other numerical filling processes is blocked; at the same time, a binary feature mask corresponding to each data item is generated (1=valid, 0=invalid).
[0036] Derivative Mask: If the original data item on which the derived statistical indicator depends is marked as invalid by the mask, then the derived indicator is also marked as invalid.
[0037] Feature selection: Calculate the effective sample ratio of each candidate feature. Features with a correlation ratio below a preset threshold (e.g., 5%) are directly set to zero and removed to avoid low effective data interfering with training.
[0038] During training: Feature channels marked as invalid by the mask are multiplied by a dynamic loss weight close to 0 for their prediction error; during backpropagation, the gradients of these channels are set to zero so that they cannot drive parameter updates.
[0039] Online inference and adaptation: Real-time data is fed into the model after going through the same governance process; when the recognition accuracy decreases and the proportion of parsing obstruction increases abnormally, the parsing rule is automatically updated (scanning new structural features and dynamically generating recognition rules), forming an end-to-end closed loop between offline governance and online inference.
[0040] This embodiment applies to a server cluster of a data governance platform. The cluster is configured with a master control node, a group of computing nodes, model service nodes, and object storage services. The nodes are connected through an internal message bus and run on a Linux operating system. The processor has a clock speed of no less than 2.4 GHz, memory of no less than 64 gigabytes, and storage capacity of no less than 4 terabytes. It deploys a distributed gradient boosting decision tree (GBDT) training framework and a parameter server collaboration component. The training samples are sourced from user-authorized and anonymized behavior log aggregation tables, covering four types of fields: browsing, clicking, dwell time, and conversion. The sample size per batch is 100,000 records, with 312 feature dimensions, including three types: basic statistical features, derived cross features, and time window aggregation features. Missing or abnormal annotations are pre-completed by the anonymization pipeline.
[0041] During the data input phase, the master node retrieves the raw input data for the current batch from the object storage service at a preset period and broadcasts it to the compute node group. The compute nodes then extract fields from each sample according to preset data item parsing rules. When field extraction fails, for example, if the text fragment corresponding to the target data item "Number of active days in the last 7 days" is "Number of active days in the last 7 days: N / A" or "7d_active=null", the preset rules cannot extract the scalar value. The computing node performs structural feature analysis on the corresponding text fragment. First, it performs multi-pattern matching with preset composite structural feature character combinations. For example, it matches key-value pair patterns in the form of "field name: value" or string patterns in the form of "field name=value" where the value is surrounded by double quotes or single quotes. If the multi-pattern matching fails, it enters the character sequence scanning stage to identify whether there is a character combination consisting of a start-leading character (letter, underscore, or Chinese start character), a key-value separator character (colon or equal sign), and a string enclosing character (double quotes, single quotes, or backticks). Based on the identification results, it dynamically generates or adjusts the composite structural feature identification rules and reuses them in the remaining samples of this batch and subsequent batches. After the above processing, the text fragment "Number of active days in the last 7 days: N / A" was identified as a key-value pair structure but with non-scalar values, corresponding to a "parsing blocked state," and its metadata identifier was recorded as 1. The text fragment contained no corresponding content for this field, corresponding to a "general missing state," and its metadata identifier was recorded as 0. Of the 100,000 samples in this batch, 8.3% were in a parsing blocked state, and 11.5% were in a general missing state.
[0042] In the feature variable governance phase, the computing node blocks target data items in a parsing-obstructed state from entering the numerical imputation process based on metadata identifiers, avoiding the replacement of non-scalar content with zero values or mean errors. Subsequently, feature variable mask information M is generated. The mask information is a binary vector with a length equal to the feature dimension. Positions corresponding to parsing-obstructed states are marked as 0, while positions corresponding to regular missing states retain the original missing markers. Based on M, derived mask information M' is generated for derived statistical indicators containing the target data item. These derived statistical indicators include cross-features such as "number of active days in the last 7 days × number of conversions in the last 30 days" and time window features such as "moving average of active days in the last 7 days." Masking any component triggers a derived mask. Then, M and M' are combined to screen the effectiveness of candidate feature variables, and the proportion of effective samples for each candidate feature variable in the mask information is calculated to obtain the effective mask ratio. The ratio of positive to negative samples for the current batch's business objective, "Whether the user converts the next day," is calculated to be 1:6.7, which is lower than the preset sparsity threshold of 1:3. Therefore, the validity filtering threshold is dynamically adjusted to 0.35 with a preset step size of 0.05. Based on the dynamically adjusted threshold, the statistical importance scores between the 47 candidate feature variables (out of 312 features) with an effective mask ratio lower than 0.35 and the business objective are set to zero and removed. For the remaining 265 candidate feature variables, a subset of valid samples is extracted and their correlation with the business objective is calculated, and the results are sorted in descending order.
[0043] In the model training phase, a distributed GBDT framework was adopted, with a learning rate of 0.05, a maximum number of iterations of 100, a maximum tree depth of 6, a minimum number of samples per leaf node of 20, a subsampling ratio of 0.8, a column sampling ratio of 0.8, and a logarithmic loss function. During training, M and M' were simultaneously distributed to each computing node. In the forward propagation phase, the prediction error of the feature variable channels marked as invalid by the mask was multiplied by a dynamic loss weight factor generated from the continuity validity score, based on M and M', to limit their contribution to the model loss calculation. In the backpropagation phase, each computing node set the values of the feature variable channels in the instantaneous input gradient to zero according to the effective mask ratio of the feature variable channels processed by its own node, and sent it to the parameter server for consistency verification and unified zeroing processing. Before the gradient optimizer updates its internal cache state, the parameter server aggregates the effective mask ratios reported by each node, calculates the global aggregated effective mask ratio of each feature variable channel, and broadcasts it to each computing node. When the global aggregation effective mask ratio of a certain feature variable channel is less than the state gating threshold of 0.1, the gradient optimizer freezes the first-order momentum and second-order variance updates for that channel, using the historical retained values from the previous batch. During the second-order variance denominator protection lower bound locking and update increment calculation phase, the protected second-order variance term is calculated by reading the base of the historical effective variance mean, and the connection weight update increment for that channel is truncated to zero. When the global aggregation effective mask ratio is not less than 0.1, the gradient optimizer performs state updates and weight updates normally.
[0044] After 100 iterations, the area under the receiver operating characteristic (AUC) of the model on 100,000 held-out test samples improved from 0.812 in the baseline to 0.847, a relative improvement of 4.31%; the recall improved from 0.384 to 0.432 with a fixed false positive rate of 0.05, a relative improvement of 12.50%; and the training time decreased from 47 minutes in the baseline to 39 minutes and 51 seconds, a relative reduction of 15.21%. Even with 30% of feature variable channels masked, the model maintained a stable feature importance ranking, with the consistency ratio of the top 20 key features reaching 0.90 compared to the unmasked scenario.
[0045] In the online pattern recognition phase, the model service node receives real-time business data streams, performs the same parsing and feature variable management processes as the offline phase on each sample, and loads the trained GBDT model for pattern recognition, outputting the user's conversion probability for the next day. The data item parsing module synchronously calculates the parsing failure rate for each data source and user group. When the parsing failure rate of a certain data source rises from the baseline of 2.1% to 6.8% within a 5-minute sliding window, an adaptive update process for the structural feature analysis rules is triggered: structural features are sampled from the latest 2000 real-time samples of the corresponding data source, identifying new composite structural feature character combinations "field name [space] equal sign [space] string," and new recognition rules are dynamically generated and sent to the subsequent processing links of the corresponding data source. The confidence assessment module evaluates the confidence level of parsing failure based on the proportion of blocked data items in real-time business data, the stability of structural distortion features, and historical parsing success rate. When the confidence level of a batch drops from 0.92 to 0.61, a smoothing mechanism for pattern recognition results is triggered. A weighted average of the pattern recognition results within a 30-minute time window is calculated to output a stable pattern recognition result. The average latency of online pattern recognition is reduced from 220 milliseconds in the baseline scheme to 195 milliseconds, a relative reduction of 11.36%.
[0046] In the adaptive update phase, the parsing rule update module writes the updated data item parsing rules back to the data item parsing module, ensuring that the new rules are prioritized in subsequent batches. During a 14-day control run, the overall parsing failure rate of the server cluster integrated with the adaptive update process decreased from 4.2% to 1.7%, a relative decrease of 59.52%; the new rule hit rate reached 88.3%; and the model's AUC in rolling evaluation remained between 0.840 and 0.852, with a standard deviation of 0.0043, representing a 52.75% improvement in stability compared to the baseline scheme's 0.0091. The data involved in the above embodiments all originated from user-authorized samples that underwent anonymization processing, complying with relevant data protection regulations.
[0047] In summary, this application effectively solves the problems of poor robustness and decreased model accuracy in traditional data operation and analysis systems when facing changes in data structure by introducing refined data item status differentiation, intelligent data governance strategies, and adaptive parsing rule update mechanisms, thus significantly improving the efficiency and reliability of data operation and analysis.
[0048] Specifically, the parsing of the above data items includes the following steps: Extract the text fragments from the original input data that correspond to the target data item; Perform multi-pattern matching on the text fragment to identify whether there is a preset combination of composite structural feature characters; When multi-pattern matching fails, the text segment is scanned to identify whether there is a character combination consisting of a non-scalar value start-leading character, a key-value separator character, and a string enclosing character. Based on the identified character combinations, a recognition rule for composite structural features is dynamically generated; The text fragment is re-examined according to the dynamically generated recognition rules to distinguish between the parsing obstruction state and the normal missing state of the target data item, and the metadata identifier is generated.
[0049] Extracting text fragments refers to locating and extracting the text portion associated with a specific target data item from the original input data. For example, if the original data is a log string or unstructured text, it is necessary to use regular expressions, delimiter recognition, or machine learning methods to accurately identify and extract the text regions that may contain the value of the target data item.
[0050] Furthermore, multi-pattern matching aims to quickly identify whether a text fragment contains known, pre-defined combinations of structural features. These combinations may include specific prefixes, suffixes, delimiter patterns, or nested structures, such as specific tags or key-value pair structures in JSON or XML fragments. Through efficient multi-pattern matching algorithms, it is possible to quickly determine whether a text fragment conforms to a certain known structured pattern.
[0051] When multi-pattern matching fails to identify a preset combination of composite structural features, the system will perform a character sequence scan on the text fragment. This scan aims to identify whether there is a character combination consisting of a non-scalar numerical start character, a key-value separator, and a string enclosing character. For example, the start character could be "{", "[", "<", etc., the key-value separator could be "=", "=", etc., and the string enclosing character could be "" or "'". These character combinations typically indicate that although the data item is not a simple scalar numerical value, it may contain some parsable structure, such as a nested JSON object or a list.
[0052] Based on the identified character combinations, the system can dynamically generate or adjust recognition rules for complex structural features. These rules can be regular expressions, finite state machines, or context-free grammars, used to accurately describe and parse the internal structure of the current text segment. The purpose of dynamically generating rules is to adapt to the diversity and variability of data formats, improving the flexibility and robustness of parsing.
[0053] Finally, the text fragment is re-examined according to dynamically generated recognition rules. This re-examination process attempts to parse the text fragment using the newly generated rules. If parsing is successful and the structure is identified, but scalar values cannot be extracted, it can be determined as a parsing blocked state; if parsing fails, or the text fragment does not conform to any recognizable structure at all, it may be determined as a regular missing state. Therefore, metadata identifiers indicating the parsing blocked state or the regular missing state are generated for the target data item, so that subsequent processes can differentiate between them.
[0054] The solution in this application, through the detailed data item parsing steps described above, is able to perform in-depth structural feature analysis on raw input data from which scalar values cannot be directly extracted.
[0055] This application further proposes the following steps for effectively screening candidate feature variables by combining feature variable mask information and derived mask information: The distribution of effective samples of each candidate feature variable in the feature variable mask information and derived mask information is statistically analyzed, and the effective mask ratio of the candidate feature variable is calculated. Calculate the positive and negative sample ratios for the current batch of business objectives; When the proportion of positive samples is lower than the preset sparsity determination threshold, the validity filtering threshold is dynamically adjusted according to the preset step size. The effective mask ratio of each candidate feature variable is determined based on the dynamically adjusted validity filtering threshold. For candidate feature variables that are below the dynamically adjusted validity filtering threshold, the statistical importance score between the candidate feature variable and the business objective is set to zero, and they are removed from the candidate feature variable set. For candidate feature variables that are not lower than the dynamically adjusted validity filtering threshold, extract a valid sample subset of the candidate feature variables, perform correlation calculation with the business objective, and sort them according to the calculation results.
[0056] Specifically, this step refines the dynamic threshold mechanism for feature selection.
[0057] Effectiveness quantification: Statistically analyze the distribution of effective samples for each candidate feature in the mask information, calculate its effective mask ratio, and use it as a quantitative indicator of data quality and usability.
[0058] Positive and negative sample evaluation: Simultaneously calculate the ratio of positive and negative samples for the current batch of business objectives to determine the sparsity of positive samples.
[0059] Dynamic threshold adjustment: When the proportion of positive samples is lower than the sparsity judgment threshold, the validity filtering threshold is dynamically lowered by a preset step size to avoid overly strict filtering that may eliminate potential valid features.
[0060] Screening and relevance calculation: Features with a correlation below the adjusted threshold are set to zero and removed; features with a correlation above the threshold are extracted into a subset of valid samples, and their correlation with the business objective is calculated (Pearson, Spearman, etc.). After being sorted by correlation strength, they are then used for training.
[0061] Example: In a financial fraud detection scenario, the proportion of positive samples in a certain batch is only 0.001 (below the threshold of 0.01). The validity threshold is dynamically adjusted from 0.80 to 0.75 in steps of 0.05. Features A and C are retained and sorted by relevance for training, while feature B is removed. This mechanism effectively solves the limitation of fixed thresholds in scenarios with sparse positive samples where key features are easily missed.
[0062] The driving effect of the aforementioned blocking ineffective feature variables on model parameter updates includes: After the backpropagation error is differentiated, the instantaneous input gradient of each feature variable channel is calculated; when the feature variable mask information or derived mask information indicates that the effective mask ratio of the feature variable channel is zero, the instantaneous input gradient is set to zero. Before the gradient optimizer updates the internal cache state, state gating is performed based on feature variable mask information or derived mask information; When the effective mask ratio is less than the preset state gating threshold, the gradient optimizer freezes the state update and forces the first-order momentum update value and second-order variance update value of the current batch to use the historical retention value of the previous batch. When the effective mask ratio is not lower than the state gating threshold, the gradient optimizer performs state update calculation normally. During the second-order variance denominator protection lower bound locking and update increment calculation stage, when the effective mask ratio is less than the state gating threshold, the gradient optimizer reads the historical effective variance mean base to calculate the protected second-order variance term and truncates the connection weight update increment of the feature variable channel to zero. When the effective mask ratio is not lower than the state gating threshold, the gradient optimizer calculates the weight update increment based on the protected second-order variance term. Update the incremental weight parameters in situ according to the calculated weights.
[0063] Specifically, after the backpropagation error derivative is calculated, the system calculates the instantaneous input gradient for each feature variable channel. The instantaneous input gradient refers to the gradient of the loss function with respect to the input feature variables on the current batch of data; it reflects the immediate contribution of each feature variable to the model's prediction error. When the feature variable mask information or derived mask information explicitly indicates that the effective mask ratio for a certain feature variable channel is zero, it means that this channel is entirely composed of invalid samples in the current batch, and its data is unreliable. In this case, to prevent invalid data from negatively impacting model training, the instantaneous input gradient of this channel will be set to zero. This aims to cut off the driving force of invalid feature variables on model parameter updates at the source.
[0064] Furthermore, before the gradient optimizer updates its internal cached state (e.g., the first-order momentum and second-order variance in the Adam optimizer), the system performs state gating on each feature variable channel based on feature variable mask information or derived mask information. State gating is a conditional control mechanism used to determine whether the optimizer should update its internal state based on the data of the current batch. When the effective mask ratio of a feature variable channel is less than a preset state gating threshold, it indicates that the channel is not effective enough, and the gradient information of its current batch may be unreliable. In this case, the gradient optimizer will freeze the state update of that channel, forcing the use of the historical retained values from the previous batch as the first-order momentum update value and second-order variance update value for the current batch. This can effectively avoid optimizer state fluctuations caused by low-quality data, thereby enhancing training stability. Conversely, when the effective mask ratio is not lower than the state gating threshold, the gradient optimizer will perform state update calculations normally, allowing effective data to drive the evolution of the optimizer state.
[0065] Furthermore, during the second-order variance denominator protection lower bound locking and update increment calculation phase, the system again makes a judgment based on the effective mask ratio. The second-order variance term is used in adaptive learning rate optimizers (such as Adam and RMSprop) to adjust the learning rate and is crucial for model convergence speed and stability. When the effective mask ratio is less than the state gating threshold, the gradient optimizer reads the historical effective variance mean base to calculate the protected second-order variance term, ensuring a relatively stable variance estimate can be used even if the current batch of data is of poor quality. More importantly, at this time, the connection weight update increment of the feature variable channel will be truncated to zero, meaning that even if the gradient and variance are calculated, the weight of that channel will not change. When the effective mask ratio is not lower than the state gating threshold, the gradient optimizer calculates the weight update increment normally based on the protected second-order variance term. Finally, the weight parameters are updated in situ according to the calculated weight update increment (which may be zero).
[0066] The proposed solution employs refined intervention in multiple key stages, including backpropagation during machine learning model training, gradient optimizer state updates, and final weight updates, to precisely block the driving effect of ineffective feature variables.
[0067] This simplified embodiment illustrates the mechanism using a single computing node as an example. The operating environment is a training server equipped with a graphics processing unit (GPU) accelerator card, with a processor clock speed of 2.8 GHz, 128 GiB of memory, and 24 GiB of GPU memory. It employs an Adaptive Moment Estimation (Adam) optimizer with an initial learning rate of 0.001, a first-order moment decay rate of 0.9, a second-order moment decay rate of 0.999, a numerical stability term of 1e-8, and a weight decay of 1e-5. The training model is a five-layer deep neural network (DNN). The first four layers are fully connected layers with 512, 256, 128, and 64 neurons respectively, using a rectified linear unit (ReLU) activation function. The fifth layer is a binary classification output layer with a sigmoid activation function. The training samples are business behavior samples that have been authorized by the user and have undergone anonymization. There are 1024 samples per batch, with 312 input feature dimensions. The business objective is a positive and negative binary label for whether the user will convert the next day, and the ratio of positive to negative samples is 1:3.2.
[0068] Before training begins, the feature variable management module distributes the feature variable mask information M and derived mask information M' for this batch. Both are binary vectors of length 312, where 0 indicates that the corresponding feature variable channel is masked, and 1 indicates that it is available. In this batch, 48 dimensions of the 312-dimensional features are masked, and the proportion of samples with parsing difficulties in the original input data is 7.6%.
[0069] During the forward propagation phase, the model sequentially performs linear transformations and activation operations, outputting the probability value of a sample belonging to a positive example, and calculating the log loss. During the backpropagation phase, the gradients of the weights of each layer and the instantaneous input gradients of each feature variable channel are calculated sequentially using the time-based backpropagation algorithm. In this batch, after the instantaneous input gradients of feature variable channels 17, 38, and 64 are calculated, they are zeroed based on M and M': for channels 17 and 38, both M and M' values are 0, the effective mask ratio is 0, and the instantaneous input gradient is zeroed; for channel 64, M is 0 but M' is 1, the effective mask ratio is 0.5, and the gradient is retained for subsequent calculations after being determined according to the preset state gating threshold of 0.1.
[0070] Before updating the internal cache state, the gradient optimizer performs state gating on each feature variable channel. The global aggregation effective mask ratio for channels 17 and 38 is 0, less than the state gating threshold of 0.1, so the corresponding first-order momentum *m* and second-order variance *v* retain the historical values from the previous batch. The global aggregation effective mask ratio for channel 64 is 0.5, not less than the state gating threshold, so the update calculation of *m* and *v* is performed normally. During the second-order variance denominator protection lower bound locking phase, the historical effective variance mean base of 0.0183 is read as the protected second-order variance term. For channels 17 and 38, the connection weight update increment is truncated to zero, and the weight parameters are not updated in this batch. For channel 64, the weight update increment is calculated according to the Adam update formula and updated in situ.
[0071] After 200 rounds of training iterations, the model's AUC on 50,000 reserved test samples improved from 0.798 without this mechanism to 0.831, a relative improvement of 4.14%. Under the stress scenario where the proportion of feature variable channels being masked increased to 35%, the AUC improved from 0.756 to 0.809, a relative improvement of 7.01%. No loss oscillations caused by gradient contamination of invalid feature variable channels occurred during training, and the standard deviation of the single-batch loss decreased from 0.043 to 0.019, a relative reduction of 55.81%. The data involved in this simplified embodiment are all from samples collected with user authorization and after de-identification processing, which comply with relevant data protection regulations.
[0072] This application further proposes steps to limit the contribution of invalid feature variables to the calculation of model loss, including: Obtain the continuity validity score for each feature variable channel, whereby the continuity validity score reflects the validity of the feature variable channel; The dynamic loss weight factor of the feature variable channel is calculated based on the continuity validity score, and the dynamic loss weight factor is positively correlated with the continuity validity score; The prediction error of the feature variable channel is multiplied by the dynamic loss weight factor to limit the contribution of the feature variable channel to the model loss calculation.
[0073] Specifically, the continuity validity score is a metric that quantitatively evaluates the data validity of each feature variable channel. This score can be calculated based on a combination of factors, such as the proportion of valid samples in the current batch of data, historical parsing success rate, data distribution stability, and confidence level of structurally differentiated features, through weighted averaging or prediction using machine learning models. Its purpose is to provide a more refined validity measure than binary masks to reflect the continuous changes in data quality.
[0074] The dynamic loss weight factor is a multiplicative factor calculated based on the continuous validity score, used to adjust the contribution of the corresponding feature variable channel to the total model loss. This weight factor is positively correlated with the continuous validity score, meaning that channels with higher validity scores have larger dynamic loss weight factors and contribute more to the model loss; conversely, channels with lower validity scores have smaller dynamic loss weight factors and contribute less to the model loss. For example, linear functions, sigmoid functions, or other non-linear mapping functions can be used to map the continuous validity score to a weight factor between 0 and 1.
[0075] In practical applications, multiplying the prediction error of each feature variable channel by the dynamic loss weighting factor means that when calculating the total model loss, the prediction error generated by each feature variable channel is no longer simply included or completely ignored, but rather weighted according to its dynamic loss weighting factor. For example, if the prediction error of a channel is E and its dynamic loss weighting factor is W, then the contribution of that channel to the total loss is E * W. The purpose is to differentiate the processing of feature variable channels with different levels of effectiveness, ensuring that the model can utilize data more rationally during training and avoiding model training bias due to data quality issues.
[0076] The proposed solution addresses the problem of overly coarse handling of channel loss contributions from invalid feature variables in traditional methods by introducing a continuous validity score and a dynamic loss weight factor.
[0077] In some preferred embodiments, a specific example is given below. Suppose that when training a machine learning model, there are three feature variable channels A, B, and C. The continuity validity score of channel A is calculated to be 0.9, indicating that its data quality is high; the continuity validity score of channel B is calculated to be 0.5, indicating that its data quality is moderate, with some missing data or parsing obstacles; the continuity validity score of channel C is calculated to be 0.1, indicating that its data quality is poor.
[0078] When calculating the dynamic loss weight factor, a simple linear mapping function can be used, for example: Dynamic loss weight factor = Continuity effectiveness score.
[0079] Therefore, the dynamic loss weighting factor for channel A is 0.9; the dynamic loss weighting factor for channel B is 0.5; and the dynamic loss weighting factor for channel C is 0.1.
[0080] When the model performs forward propagation and calculates the prediction error for each channel—for example, the prediction error for channel A is E_A, the prediction error for channel B is E_B, and the prediction error for channel C is E_C—the contribution of each channel to the total loss is weighted separately when calculating the total loss. Channel A's contribution to the total loss = E_A * 0.9 Channel B's contribution to the total loss = E_B * 0.5 Channel C's contribution to the total loss = E_C * 0.1 In this way, even if the data quality of channels B and C is lower than that of channel A, they can still participate in the calculation of the model loss with weights commensurate with their effectiveness, rather than being completely ignored. This allows the model to learn from all available, even partially effective, data, thereby improving the model's training efficiency and final prediction performance. Simultaneously, for channels with extremely poor data quality (e.g., continuity validity scores close to zero), their contribution to the loss will also approach zero, effectively limiting the negative impact of invalid data on model training.
[0081] This application further proposes that the above-mentioned pattern recognition steps include: Extract the data source identifier and user group identifier from the real-time business data; The system detects whether the parsing failure rate of the data source or the user group exhibits abnormal fluctuations within a preset time window; abnormal fluctuations trigger the adaptive update process of the structural feature analysis rules. In the adaptive update process, the latest real-time data samples from the corresponding data source or user group are sampled for structural features, and the sampled data is scanned to see if there is a character combination consisting of a start-introducing character, a key-value separator character, and a string enclosing character. Based on the identified character combinations, the composite structural feature recognition rules of the corresponding data source or user group are dynamically generated or adjusted, and the recognition rules are loaded and applied to the subsequent real-time data processing of the corresponding data source or user group. The confidence assessment module for maintaining parsing obstruction status evaluates the confidence of parsing obstruction status for each data batch based on the proportion of parsing obstructed data items in the real-time business data, the stability of structural alienation features, and the historical parsing success rate. When the confidence level is lower than a preset threshold, a smoothing mechanism for the pattern recognition results is activated. Based on the pattern recognition results within the time window, a weighted average or trend prediction is performed to output a stable pattern recognition result.
[0082] Specifically, a data source identifier can be understood as a unique code or name that identifies the source of data, such as the ID of a business system, database, or external interface. A user group identifier refers to a grouping of users with similar behavioral patterns or attributes, such as tags based on region, age group, or consumption habits. The purpose of extracting these identifiers is to enable refined management and targeted processing of data from different sources or different user groups.
[0083] Detecting abnormal fluctuations in the parsing failure rate involves monitoring the parsing success rate of data items from a specific data source or user group in real time and comparing it to the historical average, standard deviation, or a preset dynamic threshold. When the failure rate significantly deviates from the normal range within a preset time window (e.g., the past 5 minutes, 1 hour, or 1 day), it is considered an abnormal fluctuation. Such abnormal fluctuations usually indicate potential changes in the underlying data structure, requiring timely intervention.
[0084] Furthermore, structural feature sampling refers to selecting a representative small sample of data from the latest real-time data of a data source or user group experiencing abnormal fluctuations. The sampled data is scanned for character combinations consisting of a start-leading character, a key-value separator, and a string enclosing character. For example, the start-leading character could be "{" or "[", the key-value separator could be "" or "=", and the string enclosing character could be "" or "'". These character combinations are key to identifying complex data structures (such as JSON, XML, or custom formats).
[0085] Therefore, dynamically generating or adjusting composite structure feature recognition rules based on the identified character combinations means that the system automatically creates new parsing rules or modifies existing rules based on new structural patterns discovered in the sampled data. For example, if a new key-value pair format is discovered, the system will generate corresponding regular expressions or parsing logic. These rules are then loaded and applied to subsequent real-time data processing for that data source or user group to ensure that data items can be correctly parsed.
[0086] Furthermore, the confidence assessment module for parsing blocked states aims to quantify the reliability of parsing blocked data items in the current data batch. This module comprehensively considers multiple factors, including the proportion of blocked data items in real-time business data (i.e., how many data items failed to be parsed successfully), the stability of structural alienation features (i.e., whether newly discovered structural features persist or are only occasional), and the historical parsing success rate (i.e., the past parsing performance of this data source or user group). A confidence score is derived through a weighted calculation of these indicators.
[0087] When the confidence level falls below a preset threshold, a smoothing mechanism for the pattern recognition results is activated. This means that when the system's confidence in the current data batch's parsing failure is low, a smoothing strategy is employed to avoid drastic fluctuations or misjudgments in the pattern recognition results due to data quality issues. This strategy can perform a weighted average of the pattern recognition results within a time window, for example, giving higher weight to recent results; or perform trend prediction, correcting the current results based on historical trends. The aim is to output more stable and reliable pattern recognition results.
[0088] This application's solution achieves refined management of real-time business data by introducing the extraction of data source identifiers and user group identifiers. When abnormal fluctuations in the parsing failure rate of a specific data source or user group are detected, an adaptive update process for structural feature analysis rules can be triggered in a timely manner, thereby avoiding the accumulation of parsing errors caused by changes in data structure.
[0089] In some preferred embodiments, it is assumed that the data governance system of an e-commerce platform needs to perform pattern recognition on real-time transaction data from different regions (such as East China and South China) and different user groups (such as new users and old users).
[0090] First, the system extracts the data source identifier (e.g., "East China Trading System") and the user group identifier (e.g., "New User") from the received real-time transaction data.
[0091] Subsequently, the system continuously monitored the transaction data parsing failure rate for the "East China Trading System" or the "New Users" group. For example, within a certain preset time window, the system found that the parsing failure rate of the order details field for the "East China Trading System" suddenly increased from the normal 0.1% to 5%.
[0092] Such abnormal fluctuations immediately trigger an adaptive update process for the structural feature analysis rules. In this process, the system extracts a small sample, such as 100 order records, from the latest real-time order data of the "East China Transaction System." The system performs structural feature sampling on these samples, scanning for new start characters, key-value separators, or string enclosing character combinations. For example, the system might discover that the order details field has changed from the original JSON format (using "{", "=", and "") to a new XML fragment format (using "<", "=", and "'").
[0093] Based on the identified XML structural features, the system will dynamically generate or adjust the composite structural feature recognition rules for the "East China Trading System" and immediately load these new rules for subsequent real-time order data processing in the system.
[0094] Simultaneously, the system maintains a confidence assessment module for parsing blocked states. This module comprehensively evaluates the confidence level of the parsing blocked state of the current batch of data based on the proportion of parsing blocked data items in the current batch of the "East China Trading System" (e.g., although the rules have been updated, a small amount of data still fails to be parsed for other reasons), the stability of newly identified XML structure features (e.g., whether this XML format continues to appear), and the historical parsing success rate of the "East China Trading System".
[0095] If the assessed confidence level is lower than a preset threshold (e.g., 0.7), it indicates that there may still be uncertainty in the current data analysis, and the system will activate a smoothing mechanism for the pattern recognition results. For example, for the fraud transaction pattern recognition results of the "East China Trading System," the system will not directly output the recognition results for the current batch, but will instead perform a weighted average based on the recognition results from the past hour, or predict a more stable fraud risk value based on historical trends, ultimately outputting a smoothed and more reliable pattern recognition result.
[0096] Through the above process, even if the structure of the data source changes, the system can quickly adapt and continuously provide stable and reliable pattern recognition services.
[0097] The steps described above for setting the instantaneous input gradient to zero include: On each computing node, obtain the effective mask ratio of each feature variable channel in the current batch, indicated by feature variable mask information or derived mask information. The effective mask ratio is compared with a preset zero value determination threshold to determine the instantaneous validity status of each feature variable channel; if it is instantaneously invalid, the instantaneous input gradient of the feature variable channel at this node is set to zero. The instantaneous input gradient, after being zeroed out, is sent to the gradient aggregation module. In the gradient aggregation module, consistency checks are performed on the instantaneous input gradients from different nodes to identify cases where the instantaneous validity states of the same feature variable channel are inconsistent on different nodes. When an inconsistency is detected, the instantaneous input gradient of the feature variable channel is uniformly set to zero according to a preset gradient aggregation strategy; The instantaneous input gradient, which has been uniformly zeroed out, is output for subsequent model parameter updates.
[0098] Specifically, in the distributed training architecture, each computing node is responsible for processing a portion of the data and calculating its corresponding gradient. During backpropagation, each computing node obtains the effective mask ratio of the feature variable channel it processes. This effective mask ratio reflects the proportion of valid samples for that channel in the current batch of data. Subsequently, the obtained effective mask ratio is compared with a preset zero-value judgment threshold. This zero-value judgment threshold is used to determine whether a feature variable channel should be considered transiently invalid. For example, when the effective mask ratio is below a certain minimum value (such as 0.01%), it can be judged as transiently invalid. Once determined to be transiently invalid, the computing node immediately sets the transient input gradient of the feature variable channel on that node to zero. This is intended to locally prevent invalid feature variables from contributing to the model parameter updates.
[0099] Furthermore, to ensure global consistency, each computing node sends its locally zeroed instantaneous input gradients to a centralized gradient aggregation module. This module collects instantaneous input gradients from all computing nodes. After receiving gradients from all nodes, the module performs a consistency check. The purpose of this check is to identify inconsistencies in whether the same feature variable channel is considered instantaneously valid or invalid on different computing nodes. For example, a channel might be zeroed on one node but not on another. When such inconsistency is detected, the gradient aggregation module uniformly zeros the instantaneous input gradients of the feature variable channel according to a pre-defined gradient aggregation strategy. This strategy may include a majority rule or a conservative strategy such as "if one node determines it to be invalid, it is globally invalid." Finally, the uniformly zeroed instantaneous input gradients are output and used for subsequent model parameter updates, ensuring the accuracy and stability of the model updates.
[0100] The solution proposed in this application solves the problem that different nodes may have inconsistent judgments on the validity status of the same feature variable channel in a distributed training environment by introducing a collaborative mechanism between distributed computing nodes and a global consistency check of the gradient aggregation module.
[0101] As a specific implementation, assume a machine learning model is trained in a distributed manner across three computing nodes (node A, node B, and node C). During the backpropagation phase of a training batch, the instantaneous input gradient of the feature variable channel X needs to be processed. First, nodes A, B, and C each obtain the effective mask ratio of the feature variable channel X in their processed sub-batch data. Assume node A calculates the effective mask ratio of channel X to be 0.005%, which is lower than a preset zero-value threshold (e.g., 0.01%), therefore node A sets the instantaneous input gradient of channel X to zero. Node B calculates the effective mask ratio of channel X to be 0.02%, which is higher than the zero-value threshold, therefore node B does not set the instantaneous input gradient of channel X to zero. Node C calculates the effective mask ratio of channel X to be 0.008%, which is lower than the zero-value threshold, therefore node C sets the instantaneous input gradient of channel X to zero. Subsequently, nodes A, B, and C send their processed instantaneous input gradients to the gradient aggregation module. After receiving gradients from three nodes, the gradient aggregation module finds that channel X is set to zero at nodes A and C, but not at node B, indicating an inconsistency in its instantaneous validity. In this case, the gradient aggregation module processes the data according to a pre-defined gradient aggregation strategy. For example, if the strategy is "global invalidation if any node determines it is invalid," the module will uniformly set the instantaneous input gradient of channel X to zero. If the strategy is "majority rules," since two nodes determine it is invalid and one node determines it is valid, the module will still uniformly set the instantaneous input gradient of channel X to zero. Finally, the instantaneous input gradient of channel X, after being uniformly zeroed, is output and used for subsequent model parameter updates. In this way, even with local discrepancies, the consistency and accuracy of global gradient updates are ensured.
[0102] Specifically, the execution state gating determination includes: Each computing node obtains the effective mask ratio of the feature variable channel processed by its own node and sends it to the central coordination module; The central coordination module aggregates the effective mask ratios reported by each node and calculates the global aggregated effective mask ratios for each feature variable channel. The central coordination module broadcasts the global aggregation effective mask ratio to each computing node; Each computing node performs the state gating decision based on the global aggregate effective mask ratio.
[0103] Specifically, this step addresses the issue of inconsistent independent judgments among nodes in distributed training.
[0104] Local computation: Each computation node first calculates the local effective mask ratio of the channel it is responsible for, based on the local data subset and feature variable mask.
[0105] Reporting and Aggregation: Each node reports its local proportions to the central coordination module, which then aggregates them into a global aggregated effective mask proportion by means of averaging, weighted averaging, or extreme values, thus eliminating local differences.
[0106] Broadcasting and unified judgment: The central module broadcasts the global ratio back to all nodes; each node no longer uses the local value, but performs state gating judgment based on the global ratio to ensure that the blocking strategy is consistent globally.
[0107] Example: In a three-node distributed training, nodes A / B / C calculate 0.7, 0.6, and 0.8 for channel X respectively. The central module aggregates and averages the values to obtain 0.7 and broadcasts it. When the preset threshold is 0.75, all nodes determine that the channel is invalid and freeze the status update, thus achieving consistency in distributed training.
[0108] This application further proposes execution state gating determination, including: Each client obtains the effective mask ratio of the feature variable channel it processes locally and sends it to the central coordination module; The central coordination module aggregates the proportion of received valid masks within a preset waiting time window and calculates the global aggregated valid mask proportion for each feature variable channel. When the waiting time window ends, the central coordination module broadcasts the global aggregate effective mask ratio to each client; Each client performs state gating determination based on the global aggregate effective mask ratio; If the effective mask ratios of all clients are not collected within the waiting time window, the central coordination module calculates the globally aggregated effective mask ratio based on the received effective mask ratios, combined with historical data or preset default values, and broadcasts it to each of the clients.
[0109] Specifically, this section refines the global state gating mechanism under distributed training. Clients calculate the effective mask ratio for their assigned channels based on local data and report it to the central coordination module. Within a preset waiting time window (e.g., 500ms), the central module aggregates the local ratios from each client (by averaging or weighted averaging) to obtain the global aggregated effective mask ratio for each channel, serving as a unified consensus on effectiveness within the distributed system. At the end of the waiting window, the central module broadcasts the global ratio to all clients, and each client uniformly executes state gating decisions based on this ratio, ensuring global consistency of the blocking strategy. To improve robustness, if reports are not fully received within the waiting window (due to fault or network latency), the central module no longer waits indefinitely. Instead, it calculates and broadcasts an alternative global aggregated ratio based on the received ratio combined with historical batch values or default values, ensuring uninterrupted training.
[0110] For example, the three clients are grouped according to channels 1-10 / 11-20 / 21-30, and each calculates its local proportion and reports it. The central module aggregates the average and broadcasts it. If client C times out due to a fault, the central module uses the proportions of A and B combined with historical values as a backup to maintain training continuity.
[0111] refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of a data operation and analysis system based on machine learning provided in an embodiment of the present invention, including a processor and a memory, wherein the memory stores computer program instructions, characterized in that the system includes: The data item parsing module, coupled to the processor, is used to acquire raw input data. When the scalar value of the target data item cannot be extracted from the raw input data according to the preset data item parsing rules, the module performs structural feature analysis on the text fragments in the raw input data corresponding to the target data item to distinguish between the parsing blocked state and the normal missing state of the target data item, and generates metadata identifiers for the target data item indicating the parsing blocked state or the normal missing state. The feature variable governance module, coupled with the data item parsing module, is used to identify target data items in a parsing blocked state based on the metadata identifier, block the target data items from entering the numerical imputation process, and generate feature variable mask information characterizing the validity of each target data item; based on the feature variable mask information, it generates derivative mask information for derived statistical indicators containing the target data items, and combines the feature variable mask information and the derivative mask information to screen the validity of candidate feature variables. After setting the correlation evaluation between candidate feature variables with a valid sample ratio lower than a preset threshold and the business objective to zero, they are removed from the candidate feature variable set. The model training module, coupled with the feature variable governance module, is used to limit the contribution of feature variable channels marked as invalid by the mask to the model loss calculation when training the machine learning model, based on the feature variable mask information and the derived mask information, and to block the driving effect of the feature variable channels on the model parameter update. The pattern recognition module, coupled with the data item parsing module, the feature variable management module and the model training module, is used to perform data item parsing and feature variable management on the real-time business data when receiving real-time business data, and to perform pattern recognition based on the machine learning model output by the model training module, and output the pattern recognition result. The parsing rule update module, coupled with the pattern recognition module and the data item parsing module, is used to adaptively update the data item parsing rules based on the pattern recognition results and the distribution of parsing blocked data items in the real-time business data, and write the updated data item parsing rules back to the data item parsing module, forming an end-to-end closed loop of offline data governance and online pattern recognition.
[0112] In some embodiments, at the hardware level, the server cluster consists of 1 master node, 8 compute nodes, 2 parameter servers, 3 model service nodes, and 1 object storage service node. The nodes are connected via an internal message bus with a rate of 25 gigabits per second, running on a Linux operating system. The processor clock speed is no less than 2.4 gigahertz, the single-node memory is no less than 64 gigabytes, and the storage capacity is no less than 4 terabytes. At the software level, a distributed gradient boosting decision tree (GBDT) training framework, parameter server collaboration components, a real-time feature calculation engine, and a rule configuration center are deployed. The rule configuration center is responsible for uniformly distributing data item parsing rules and composite structure feature recognition rules. At the algorithm level, the main model is a hierarchical GBDT plus logistic regression (LR) fusion structure. The GBDT part is responsible for high-order nonlinear feature extraction, and the LR part is responsible for linear scoring fusion. The fusion weights are selected from the candidate set by offline grid search, using the largest area under the receiver operating characteristic curve (AUC) as the final weight.
[0113] At the data level, both training and online samples are derived from user-authorized collection and anonymized business behavior samples, covering seven categories of fields including browsing, clicking, dwell time, conversion, favorites, sharing, and payment. The feature dimensions are 312, including 128 basic statistical features, 96 derived cross features, and 88 time window aggregation features. Missing and anomaly annotations are uniformly injected by the anonymization pipeline. The sample size is 10 million records, divided into training set, validation set, and holdout test set in an 8:1:1 ratio. The business objective is a binary label for whether the user converts the next day, with a positive to negative sample ratio of 1:4.7.
[0114] In terms of algorithm implementation details, the composite structure feature recognition rules are implemented using a deterministic finite state automaton. The state transition table is stored in the rule configuration center, and rule updates are triggered by the parsing rule update module. Feature variable mask information is stored in sparse vector format with a length equal to the feature dimension. Masked positions are marked with 0, and available positions are marked with 1. The storage cost is 0.4% of the original feature tensor. Derived mask information is stored in bitmap format at the entry point of the derived statistical index construction pipeline. During the construction process, if any component is masked, the corresponding derived index mask is set. The gradient optimizer uses a variant of Adaptive Moment Estimation (Adam). The state gating threshold is determined on the validation set using a grid search method, with a candidate range of 0.05 to 0.20 and a step size of 0.025. The search result is the value corresponding to the maximum AUC. The continuity validity score is statistically analyzed on the most recent 10 batches using an exponentially moving weighted method, with a weight decay coefficient of 0.9. The dynamic loss weight factor is linearly positively correlated with the continuity validity score, with a value range of 0. Up to 1; In the state gating determination stage, the global aggregate effective mask ratio is calculated by the parameter server based on the weighted average of the values reported by each computing node according to the sample size of each node, and the broadcast period is the time window between the end of a single batch of training and the start of the next batch of training.
[0115] In terms of performance evaluation methods, classification performance was comprehensively measured by three indicators: AUC, recall at a fixed false positive rate of 0.05, and Kolmogorov-Smirnov Statistic (KS). Regression performance was measured by mean absolute error (MAE) and root mean square error (RMSE). Training efficiency was measured by the training time per batch and the total number of epochs required for model convergence. Online service performance was measured by average pattern recognition latency and request processing capacity per unit time. Closed-loop stability was measured by the mean and standard deviation of AUC over 14 days of rolling evaluation. All evaluations were conducted under the same hardware environment and the same hyperparameter configuration, and control variables were kept consistent.
[0116] In terms of experimental design, three control groups were set up. The first group was the baseline scheme, which did not introduce a feature variable masking mechanism. When the parsing of target data items was blocked, they were directly filled with zero or mean values, and the gradient optimizer did not perform state gating and variance protection. The second group was the simplified scheme, which only introduced feature variable masking information for the validity screening of candidate feature variables. Gradient zeroing and state gating were not performed during the model training phase. The third group was the proposed scheme, which fully introduced feature variable masking information, derived masking information, dynamic loss weight factors, instantaneous gradient zeroing, state gating, and variance protection. Experimental results showed that on 10 million training samples and 1 million held-out test samples, the AUC of the proposed scheme was 0.852, the AUC of the baseline scheme was 0.812, and the AUC of the simplified scheme was 0.829. The proposed scheme improved the AUC by 4.93% compared to the baseline scheme and by 2.77% compared to the simplified scheme. Under stress conditions where 35% of the feature variable channels are masked, the AUC of this proposed solution is 0.812, while the AUC of the baseline solution decreases to 0.741, and the AUC of the simplified solution decreases to 0.768. This proposed solution represents a 9.58% improvement in AUC compared to the baseline solution and a 5.73% improvement compared to the simplified solution. In the 14-day rolling evaluation, the mean AUC of this proposed solution is 0.849 with a standard deviation of 0.0041; the mean AUC of the baseline solution is 0.809 with a standard deviation of 0.0097; and the mean AUC of the simplified solution is 0.826 with a standard deviation of 0.0072. The stability of this proposed solution is 57.73% better than the baseline solution and 43.06% better than the simplified solution.
[0117] Regarding model interpretability, the hierarchical GBDT part uses feature importance ranking and single decision tree path output as interpretability carriers. Feature importance is calculated by weighting split gain, and the top 20 key features are output in the form of feature name, numerical range, split threshold, and number of covered samples. The LR part uses feature weights and feature contribution values as interpretability carriers. The single sample scoring result is output in the form of the sum of the contribution values of each feature. The positive or negative sign and absolute value of the feature contribution value reflect the direction and degree of influence of the feature on the single sample scoring. The above interpretable outputs are filtered through a de-identification pipeline and then distributed to business operations personnel for use, without involving the backtracking of the original data.
[0118] Regarding the closed-loop operation performance, during the 30-day actual operation of the server cluster, the overall parsing failure rate decreased from 4.5% to 1.5%, a relative decrease of 66.67%; the new rule hit rate remained in the range of 85% to 90%; the model's AUC in rolling evaluation remained in the range of 0.840 to 0.855, with a standard deviation of 0.0041; the average latency for pattern recognition remained in the range of 180 milliseconds to 200 milliseconds, and the request processing capacity per unit time remained in the range of 1200 to 1500 requests per second; the model trained by this solution achieved a consistency ratio of 0.91 for the top 20 key features in offline spot checks by business operations personnel, and a consistency ratio of 0.88 between the model score and business annotations. All data involved in this supplementary embodiment are from user-authorized samples that have undergone anonymization processing, complying with relevant data protection regulations.
[0119] The machine learning-based data operations analysis system proposed in this application aims to address the performance degradation caused by data parsing anomalies in traditional data operations analysis systems when faced with silent changes in the data structure of upstream business systems. Traditional systems often lack the ability to finely identify data parsing obstructions, simply treating failed data as regular missing data and performing inappropriate numerical imputation. This introduces spurious associations in feature engineering and model training, ultimately leading to poor robustness and decreased accuracy of the model in practical applications.
[0120] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A data operation analysis method based on machine learning, applied to a server cluster of a data governance platform, characterized in that, The method includes: The system acquires raw input data. When the target data item cannot be extracted from the raw input data according to the preset data item parsing rules, it performs structural feature analysis on the text fragments in the raw input data that correspond to the target data item to distinguish between the parsing blocked state and the normal missing state of the target data item, and generates metadata identifiers for the target data item that indicate the parsing blocked state or the normal missing state. The target data item in the parsing blocked state is identified based on the metadata identifier, and the target data item is blocked from entering the numerical imputation process. Feature variable mask information representing the validity of each target data item is generated. Based on the feature variable mask information, derivative mask information is generated for the derived statistical indicators containing the target data item. The candidate feature variables are screened for validity by combining the feature variable mask information and the derivative mask information. The correlation between the candidate feature variables with the effective sample ratio below a preset threshold and the business target is set to zero and then removed from the candidate feature variable set. When training a machine learning model, based on the feature variable mask information and the derived mask information, the feature variable channels marked as invalid by the mask are restricted from contributing to the model loss calculation and the driving effect of the feature variable channels on the model parameter update is blocked. When receiving real-time business data, the system performs data item parsing and feature variable management on the real-time business data, and performs pattern recognition based on the machine learning model, outputting the pattern recognition results. Based on the pattern recognition results and the distribution of blocked data items in the real-time business data, the data item parsing rules are adaptively updated, and the updated data item parsing rules are applied to subsequent data item parsing, forming an end-to-end closed loop of offline data governance and online pattern recognition.
2. The method of claim 1, wherein, The data item parsing includes: Extract the text fragments from the original input data that correspond to the target data item; Perform multi-pattern matching on the text fragment to identify whether there is a preset combination of composite structural feature characters; When multi-pattern matching fails, the text segment is scanned to identify whether there is a character combination consisting of a non-scalar value start-leading character, a key-value separator character, and a string enclosing character. Based on the identified character combinations, a recognition rule for composite structural features is dynamically generated; The text fragment is re-examined according to the dynamically generated recognition rules to distinguish between the parsing obstruction state and the normal missing state of the target data item, and the metadata identifier is generated.
3. The method of claim 1, wherein, The step of combining the feature variable mask information and the derived mask information to perform effectiveness screening of candidate feature variables includes: The distribution of effective samples for each candidate feature variable in the feature variable mask information and the derived mask information is statistically analyzed, and the effective mask ratio of the candidate feature variable is calculated. Calculate the positive and negative sample ratios for the current batch of business objectives; When the proportion of positive samples is lower than the preset sparsity determination threshold, the effectiveness filtering threshold is dynamically adjusted according to the preset step size. The effective mask ratio of each candidate feature variable is determined based on the dynamically adjusted validity filtering threshold. For candidate feature variables that are below the dynamically adjusted validity filtering threshold, the statistical importance score between the candidate feature variable and the business objective is set to zero, and they are removed from the candidate feature variable set. For candidate feature variables that are not lower than the dynamically adjusted validity filtering threshold, extract a valid sample subset of the candidate feature variables, perform correlation calculation with the business objective, and sort them according to the calculation results.
4. The method of claim 1, wherein, The blocking of the driving effect of the feature variable channel on model parameter updates includes: After the backpropagation error is differentiated, the instantaneous input gradient of each feature variable channel is calculated; when the feature variable mask information or the derived mask information indicates that the effective mask ratio of the feature variable channel is zero, the instantaneous input gradient is set to zero. Before the gradient optimizer updates the internal cache state, a state gating decision is performed based on the feature variable mask information or the derived mask information; When the effective mask ratio is less than the preset state gating threshold, the gradient optimizer freezes the state update and forces the first-order momentum update value and second-order variance update value of the current batch to use the historical retention value of the previous batch. When the effective mask ratio is not lower than the state gating threshold, the gradient optimizer performs state update calculation normally; In the second-order variance denominator protection lower bound locking and update increment calculation stage, when the effective mask ratio is less than the state gating threshold, the gradient optimizer reads the historical effective variance mean base to calculate the protected second-order variance term and truncates the connection weight update increment of the feature variable channel to zero. When the effective mask ratio is not lower than the state gating threshold, the gradient optimizer calculates the weight update increment based on the protected second-order variance term; The incremental in-situ update weight parameters are updated according to the calculated weights.
5. The method according to claim 1, characterized in that, The limitation on the contribution of the feature variable channels to the model loss calculation includes: Obtain the continuity validity score for each feature variable channel, whereby the continuity validity score reflects the validity of the feature variable channel; The dynamic loss weight factor of the feature variable channel is calculated based on the continuity validity score, and the dynamic loss weight factor is positively correlated with the continuity validity score; The prediction error of the feature variable channel is multiplied by the dynamic loss weight factor to limit the contribution of the feature variable channel to the model loss calculation.
6. The method of claim 1, wherein, The pattern recognition includes: Extract the data source identifier and user group identifier from the real-time business data; The system detects whether the parsing failure rate of the data source or the user group exhibits abnormal fluctuations within a preset time window; abnormal fluctuations trigger the adaptive update process of the structural feature analysis rules. In the adaptive update process, the latest real-time data samples from the corresponding data source or user group are sampled for structural features, and the sampled data is scanned to see if there is a character combination consisting of a start-introducing character, a key-value separator character, and a string enclosing character. Based on the identified character combinations, the composite structural feature recognition rules of the corresponding data source or user group are dynamically generated or adjusted, and the recognition rules are loaded and applied to the subsequent real-time data processing of the corresponding data source or user group. The confidence assessment module for maintaining parsing obstruction status evaluates the confidence of parsing obstruction status for each data batch based on the proportion of parsing obstructed data items in the real-time business data, the stability of structural alienation features, and the historical parsing success rate. When the confidence level is lower than a preset threshold, a smoothing mechanism for the pattern recognition results is activated. Based on the pattern recognition results within the time window, a weighted average or trend prediction is performed to output a stable pattern recognition result.
7. The method of claim 4, wherein, Setting the instantaneous input gradient to zero includes: On each computing node, obtain the effective mask ratio of each feature variable channel in the current batch as indicated by the feature variable mask information or the derived mask information; The effective mask ratio is compared with a preset zero value determination threshold to determine the instantaneous validity status of each feature variable channel; if it is instantaneously invalid, the instantaneous input gradient of the feature variable channel at this node is set to zero. The instantaneous input gradient, after being zeroed out, is sent to the gradient aggregation module. In the gradient aggregation module, consistency checks are performed on the instantaneous input gradients from different nodes to identify cases where the instantaneous validity states of the same feature variable channel are inconsistent on different nodes. When an inconsistency is detected, the instantaneous input gradient of the feature variable channel is uniformly set to zero according to a preset gradient aggregation strategy; The instantaneous input gradient, which has been uniformly zeroed out, is output for subsequent model parameter updates.
8. The method of claim 4, wherein, The execution state gating determination includes: Each computing node obtains the effective mask ratio of the feature variable channel processed by its own node and sends it to the central coordination module; The central coordination module aggregates the effective mask ratios reported by each node and calculates the global aggregated effective mask ratios for each feature variable channel. The central coordination module broadcasts the global aggregation effective mask ratio to each computing node; Each computing node performs the state gating decision based on the global aggregate effective mask ratio.
9. The method of claim 4, wherein, The execution state gating determination includes: Each client obtains the effective mask ratio of the feature variable channel it processes locally and sends it to the central coordination module; The central coordination module aggregates the proportion of received valid masks within a preset waiting time window and calculates the global aggregated valid mask proportion for each feature variable channel. When the waiting time window ends, the central coordination module broadcasts the global aggregate effective mask ratio to each client; Each client performs state gating determination based on the global aggregate effective mask ratio; If the effective mask ratios of all clients are not collected within the waiting time window, the central coordination module calculates the globally aggregated effective mask ratio based on the received effective mask ratios, combined with historical data or preset default values, and broadcasts it to each of the clients. 10.A machine learning based data operation analysis system, comprising a processor and a memory, the memory storing computer program instructions, characterized in that, The system includes: The data item parsing module, coupled to the processor, is used to acquire raw input data. When the scalar value of the target data item cannot be extracted from the raw input data according to the preset data item parsing rules, the module performs structural feature analysis on the text fragments in the raw input data corresponding to the target data item to distinguish between the parsing blocked state and the normal missing state of the target data item, and generates metadata identifiers for the target data item indicating the parsing blocked state or the normal missing state. The feature variable governance module, coupled with the data item parsing module, is used to identify target data items in a parsing blocked state based on the metadata identifier, block the target data items from entering the numerical imputation process, and generate feature variable mask information characterizing the validity of each target data item; based on the feature variable mask information, it generates derivative mask information for derived statistical indicators containing the target data items, and combines the feature variable mask information and the derivative mask information to screen the validity of candidate feature variables. After setting the correlation evaluation between candidate feature variables with a valid sample ratio lower than a preset threshold and the business objective to zero, they are removed from the candidate feature variable set. The model training module, coupled with the feature variable governance module, is used to limit the contribution of feature variable channels marked as invalid by the mask to the model loss calculation when training the machine learning model, based on the feature variable mask information and the derived mask information, and to block the driving effect of the feature variable channels on the model parameter update. The pattern recognition module, coupled with the data item parsing module, the feature variable management module and the model training module, is used to perform data item parsing and feature variable management on the real-time business data when receiving real-time business data, and to perform pattern recognition based on the machine learning model output by the model training module, and output the pattern recognition result. The parsing rule update module, coupled with the pattern recognition module and the data item parsing module, is used to adaptively update the data item parsing rules based on the pattern recognition results and the distribution of parsing blocked data items in the real-time business data, and write the updated data item parsing rules back to the data item parsing module, forming an end-to-end closed loop of offline data governance and online pattern recognition.
