Log data quality evaluation and quantitative model method based on reinforcement learning

Abstract

The invention discloses a reinforcement learning-based log data quality evaluation and quantification model method, which comprises the following steps of: performing desensitization, deduplication, timestamp alignment and structured analysis on server logs to form a log sequence; outputting a quality score based on an analysis result, an entity extraction result and quality indexes such as missing and abnormal statistical calculation integrity, consistency and timeliness; setting a rapid scoring device and a reference scoring device, identifying distribution drift according to the scoring deviation of the rapid scoring device and the reference scoring device, and updating a scoring model; and constructing a state space and an action space, learning a governance action selection strategy by adopting Q-learning, re-scoring the governed log, and updating strategy parameters according to quality improvement and resource consumption. According to the method, the log data quality can be continuously evaluated, the quality problem can be found and positioned, and the balance between quality improvement and resource consumption is realized.

Description

Technical Field

[0001] This invention belongs to the field of data governance and data quality management technology, specifically involving data quality assessment, quality trend monitoring, drift detection and adaptive updating for server logs, as well as reinforcement learning-based governance strategy learning and governance and optimization methods. Background Technology

[0002] Server logs are widely used for operation monitoring, fault diagnosis, audit analysis, and data asset management. During the collection, transmission, storage, and parsing processes, log data is prone to problems such as missing fields, inconsistent structure, duplicate records, incorrect timestamps, delayed arrival, outliers, and parsing failures, leading to decreased log quality and impacting the reliability of downstream analysis.

[0003] Existing log data quality management solutions mostly use fixed rules, static thresholds, or offline sampling methods for quality verification, which can only output quality scores or alarm information, making it difficult to maintain stability and effectiveness under dynamic business load and log distribution changes.

[0004] Furthermore, log quality assessment faces the challenge of balancing real-time performance with accuracy: high-precision standard verification is typically computationally expensive and difficult to perform frequently; low-cost, rapid assessment may become inaccurate over time and with changes in distribution, thereby reducing the reliability of quality monitoring and governance decisions.

[0005] Therefore, a closed-loop method is needed that can continuously and quantitatively evaluate log quality in a dynamic environment, adaptively update it based on drift, and automatically select governance actions based on the evaluation results, so as to control resource consumption while ensuring quality objectives. Summary of the Invention

[0006] S1: Data governance and log analysis platform modeling: Determine log field specifications and quality constraints, collect server logs and perform de-identification, deduplication, timestamp alignment and structured parsing to form a log sequence organized by time window;

[0007] S2: Dataset Construction and Entity Recognition: Key entities in the logs are manually labeled to form an entity recognition dataset. The structured log sequence is trained or its parameters are estimated using a Conditional Random Field (CRF) sequence labeling model, and entity extraction is performed on the log text.

[0008] S3: Quality Indicator Calculation and Quantitative Scoring: Based on the parsing results, entity extraction results, and anomalies such as missing values, the quality indicators are statistically calculated and a quality score is obtained. The quality indicators include at least completeness, consistency, and timeliness.

[0009] S3-1: Quality Indicator Calculation: Based on the structured parsing results, entity extraction results, and missing or outlier statistics, completeness is calculated separately. ,consistency With timeliness Quality indicators, and output dimensional sub-indicators and problem location information;

[0010] S3-2: Quantitative Score Generation: The quality indicators are normalized and summarized according to preset weights or rules to form a time window-level quality score. This will generate a quality trend sequence for use in subsequent steps.

[0011] S4: Fast Scoring - Benchmark Scoring Collaboration: Sets the output of the fast scorer to predict the quality score, and calculates it according to a preset block threshold. Trigger the benchmark scorer to output a benchmark quality score;

[0012] S5: Drift Detection and Dynamic Update: When the deviation between the predicted quality score and the baseline quality score exceeds the preset tolerance... When a data distribution shift is detected, a re-scoring of the historical window is triggered to update the baseline quality score, and the fast scorer is updated accordingly.

[0013] S6: Provide feedback on the evaluation results: Construct the quality score, its trend and drift signal into a state space, construct the governance actions into an action space, use Q-learning to learn the governance action selection strategy and execute the governance actions; recalculate the quality score on the logs after governance and use the quality improvement and resource consumption feedback to update the Q-learning parameters or trigger thresholds.

[0014] S6-1: Strategy Learning and Decision Making: Constructing a state space from quality scores, quality change trends, and drift signals. To construct governance actions as an action space Construct a reward function Q-learning is used to learn and manage action selection strategies. And output the governance actions of the current window;

[0015] S6-2: Execute Governance and Closed-Loop Feedback: Execute the governance actions, recalculate the quality score of the governed logs, calculate the quality improvement and resource consumption, and generate reward feedback. Use the reward feedback to update the state-action value function parameters of Q-learning and / or to adjust the trigger threshold. , . Attached Figure Description

[0016] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0017] Figure 1This is a schematic diagram of the overall process described in this invention (corresponding to steps S1-S6).

[0018] Figure 2 The following are the subdivisions of step 3 of the present invention.

[0019] Figure 3 The following are the subdivisions of step 6 of the present invention.

[0020] Figure 4 This is a flowchart illustrating the data integrity verification process using log anomaly identification as described in this invention.

[0021] Figure 5 This is a flowchart of the drift detection and dynamic update process described in this invention.

[0022] Figure 6 This is a schematic diagram of the governance action selection framework based on Q-learning described in this invention. Detailed Implementation

[0023] The following specific examples illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. The technical solution of the present invention will be further described below in conjunction with specific embodiments and accompanying drawings.

[0024] This invention details the specific implementation steps of the proposed reinforcement learning-based log data quality assessment and adaptive governance method. The specific method flow is described below:

[0025] Step 1: Modeling the Data Governance and Log Analysis Platform. It should be noted that this embodiment focuses on server logs, first determining the log field specifications and quality constraints.

[0026] (1) Determining Log Field Specifications and Quality Constraints: Defining the Set of Required Fields It also establishes logical constraints such as field type constraints and format constraints; and defines the structured parsing success conditions and exception record judgment rules.

[0027] (2) Log collection, desensitization and deduplication: Collect raw logs from the log source, desensitize sensitive fields; construct deduplication keys to deduplicat records; write unparseable or conflicting records into the exception queue and include them in the statistics.

[0028] (3) Timestamp alignment and structured parsing: unify the timestamp format and time zone of logs from different sources; perform structured parsing on semi-structured logs to output field-based records; record the reasons for parsing failures for quality statistics.

[0029] (4) Organize by time window: based on window length Logs are split into window sequences. Each window aggregates log records and statistical characteristics within the window, providing input for subsequent quality assessment and decision-making.

[0030] Step 2: Dataset Construction and Entity Recognition.

[0031] (1) Labeling standards and dataset construction: Determining the set of key entity types This includes, but is not limited to, IP addresses, ports, interface paths, error codes, and resource identifiers; it also involves developing annotation standards and manually annotating sample logs to create entity recognition training, validation, and test sets.

[0032] (2) CRF sequence labeling model training and entity extraction: The log text is segmented into words or by character to obtain the observation sequence. Corresponding label sequence CRF uses conditional probability Modeling sequence labeling task: ,in As the normalization factor, For characteristic function, The parameters are defined as follows. During the training phase, the conditional log-likelihood is maximized, and an L2 regularization term can be added. Estimation is performed; the Viterbi algorithm is used to solve for the optimal tag sequence during the decoding stage. This allows us to obtain entity boundaries and entity categories, and output entity coverage and confidence statistics for subsequent quality verification.

[0033] Step 3: Calculation and Quantitative Scoring of Quality Indicators

[0034] (1) The quality indicators include the following properties: First, completeness, which includes field completeness, record completeness, etc. Field completeness is the ratio of the number of non-empty required fields to the total number of required fields in a single log record; record completeness is the number of valid records that meet the structured parsing and type constraints within the time window. The total number of records collected within that time window The ratio. The calculation formula is: ,in, Indicates the total number of required fields. This is the indicator function. Next is consistency, which mainly involves establishing a rule base for field types, formats, and cross-field logical constraints. The pass rate of rules within the statistical window is used to form a consistency score, and the results of conflict record location are output. The calculation formula is: Secondly, there is the timeliness aspect, namely: setting a unique time decay coefficient λ, calculated using the following formula: .in, It's about handling delays. It is the attenuation coefficient, which is set according to the business sensitivity.

[0035] (2) Quality score: The scores of dimensions such as completeness, consistency and timeliness are summed according to their weights to form a window-level quality score. It outputs the dimension sub-scores and positioning results.

[0036] (3) such as Figure 4 As shown, the process operates in parallel along two links: "data" and "log verification rules." After anomaly location, the data body ID is traced back to complete integrity verification. First, the data link includes: Data acquisition: Server logs are collected, anonymized, deduplicated, timestamp aligned, and structured to obtain a set of log records organized by time windows. Data-log relationship establishment: A mapping relationship between data bodies and log records is established based on the association keys in the logs. These association keys include, but are not limited to, request_id, trace_id, session_id, host identifier, service name, and timestamp window number. Next, data body ID location: After the log verification rule link outputs the anomaly location result, the corresponding data body ID is located back through the mapping relationship. Then, integrity verification: For the located data body ID, the field integrity and record integrity of its corresponding log record are calculated within the time window, and the integrity verification conclusion is output. Finally, the identification result output: The integrity verification result and related location information such as missing fields / missing entities / anomaly types are output for subsequent governance actions and evaluation statistics.

[0037] Furthermore, the log verification rule chain includes: log data analysis: analyzing structured log fields, entity extraction results, and anomaly statistics to form a feature set that can be used for rule verification and anomaly identification; log training: training or parameter calibration based on normal / abnormal samples or rule-labeled data from historical windows to obtain a model / rule set for anomaly identification; analysis body formation: constructing an analysis body to carry log verification rules, anomaly identification models, and their operating status, and receiving policy or parameter inputs from the intelligent agent; anomaly log graph identification: constructing a log relationship graph based on field associations, entity relationships, call chain relationships, or event sequence relationships within a time window, and identifying and outputting anomaly patterns in the graph; and anomaly event localization: mapping anomaly patterns to specific log record sets or event chain locations, outputting anomaly event localization results, and feeding these results back to the data chain for data body ID localization.

[0038] Step 4: Rapid scoring – benchmark scoring coordination

[0039] (1) Fast scorer: The fast scorer uses window feature vectors Score the prediction quality of input and output The window features include missing rate, parsing failure rate, field distribution statistics, entity coverage, latency statistics, and historical trend features.

[0040] (2) Benchmark scorer: The benchmark scorer outputs a benchmark quality score using a standard scoring method based on rules or statistical calculations. It is used to verify rapid scoring and generate training labels.

[0041] (3) Block threshold Trigger: Based on preset block threshold Trigger the execution of the benchmark scorer, for example, per Execute once per window or on critical windows to balance evaluation cost and verification frequency.

[0042] Step 5: Drift Detection and Dynamic Update (see...) Figure 5 )

[0043] (1) Drift determination: Define deviation .when Exceeding the preset tolerance When this occurs, it is determined that a data distribution shift or a change in the scoring relationship has taken place.

[0044] (2) Rescoring and label update: After drift is triggered, the baseline quality score of the historical window is recalculated according to the current quality rules to update the training label set.

[0045] (3) Fast scorer update: Based on the updated training data, the fast scorer is retrained or incrementally updated to adapt the predicted score to the new data distribution and quality rules.

[0046] Step 6: Provide feedback on the evaluation results

[0047] (1) State space construction: The quality score, quality change trend and drift signal are constructed into a state space. The quality change trend can be obtained by sliding time window trend statistics, change rate discrimination, or drift signal-assisted discrimination; and features such as resource load level, missing rate level, and parsing failure rate level can be added.

[0048] (2) Action space construction: constructing governance actions as a set of actions. This includes at least adjustments to the acquisition strategy, retrying acquisition, cleaning up abnormal records, completing missing fields, and adjusting entity recognition strategy parameters; among which, adjustments to the acquisition strategy may include adjustments to the sampling rate and the number of retries.

[0049] (3) Reward function design: The reward function is used to guide the Q-learning agent in the candidate source dataset set. Choose the one that is more conducive to improving the target dataset Verification of effects .like Figure 6 As shown, agent selection Then, training is completed on the training branch network, and knowledge is transferred to the validation branch network via "Transfer Knowledge," followed by training on the target dataset. Verification and reward calculation are performed on the above. .

[0050] Let the verification branch be in The evaluation indicators are (This can be the negative of accuracy, F1, or quality score error), the cost introduced by training and transfer is... (This can be represented by statistics such as training duration, number of updates, and computational resource usage), then the reward function is defined as: .

[0051] in: This indicates the effectiveness metric. If the goal is "consistency in quality scoring," then... , , To verify the predicted scores of the branch output, The benchmark score / label score is used; the smaller the error, the greater the reward. Indicates the training / update cost. The formula typically used to represent training time is: or (In cases where the number of updates is weighted).

[0052] , This is a weighting coefficient used to balance "improved validation results" and "update cost control".

[0053] (4) Q-learning update and execution: such as Figure 6 As shown, the Q-learning agent verifies the state of the branch output. As input to the environment state, and with the option to "select source dataset" "As an action. The iteration process for each round is as follows: in state..." The following is adopted Greedy strategy selects action The actions A source dataset is selected from the candidate set. : based on probability Random selection As training; with probability Choose to The largest As an application. Then in the selected The training branch network is trained, i.e., trained, and then the trained knowledge is transferred to the validation branch network via "Transfer Knowledge". Then, on the target dataset... The validation process is then performed to obtain validation performance metrics and cost statistics, and rewards are calculated. And form the next state. (The data sent back to the Agent from the verification branch in the diagram) and (Loop). Finally, update the state-action value function according to the Q-learning rule: the formula is: ,in: Represented as state Next action The value (that is, choosing a certain) (value) This is expressed as the learning rate, which controls the degree of influence of new information. This is indicated as an instant reward; Represented as a discount factor, it measures the importance of future rewards; Indicated as the next state The largest Value. The strategy obtained through the above update is used to guide subsequent actions. This selection process forms a closed-loop optimization process of "selection—training—transfer—validation—feedback".

Claims

1. A method for log data quality assessment and quantification based on reinforcement learning, characterized in that, Includes the following steps: S1: Data governance and log analysis platform modeling: Determine log field specifications and quality constraints, collect server logs and perform de-identification, deduplication, timestamp alignment and structured parsing to form a log sequence organized by time window; S2: Dataset Construction and Entity Recognition: Key entities in the logs are manually labeled to form an entity recognition dataset. The structured log sequence is trained or its parameters are estimated using a Conditional Random Field (CRF) sequence labeling model, and entity extraction is performed on the log text. S3: Quality Indicator Calculation and Quantitative Scoring: Based on the parsing results, entity extraction results, and anomalies such as missing values, the quality indicators are statistically calculated and a quality score is obtained. The quality indicators include at least completeness, consistency, and timeliness. S4: Fast Scoring - Benchmark Scoring Collaboration: Sets the output of the fast scorer to predict the quality score, and calculates it according to a preset block threshold. Trigger the benchmark scorer to output a benchmark quality score; S5: Drift Detection and Dynamic Update: When the deviation between the predicted quality score and the baseline quality score exceeds the preset tolerance... When a data distribution shift is detected, a re-scoring of the historical window is triggered to update the baseline quality score, and the fast scorer is updated accordingly. S6: Provide feedback on the evaluation results: Construct the quality score, its trend and drift signal into a state space, construct the governance actions into an action space, use Q-learning to learn the governance action selection strategy and execute the governance actions; recalculate the quality score on the logs after governance and use the quality improvement and resource consumption feedback to update the Q-learning strategy parameter trigger threshold.

2. The method S1 according to claim 1, characterized in that, Taking server logs as the object, first determine the log field specifications and quality constraints, including: (1) Determining Log Field Specifications and Quality Constraints: Defining the Set of Required Fields And establish logical constraints such as field type constraints and format constraints; at the same time define the structured parsing success conditions and the exception record judgment rules; (2) Log collection, desensitization and deduplication: Collect raw logs from the log source and desensitize sensitive fields; construct deduplication keys to deduplicate duplicate records; write unparseable or conflicting records into the exception queue and include them in the statistics; (3) Timestamp alignment and structured parsing: unify the timestamp format and time zone of logs from different sources; perform structured parsing on semi-structured logs to output field-based records; record the reasons for parsing failures for quality statistics; (4) Organize by time window: based on window length Logs are split into window sequences. Each window aggregates log records and statistical characteristics within the window, providing input for subsequent quality assessment and decision-making.

3. S5 in the method according to claim 1, characterized in that, The fast scorer is a machine learning prediction model that uses drift detection and adaptive updates, including: (1) Drift determination: Define deviation ,when Exceeding the preset tolerance At that time, it is determined that a data distribution shift or a change in the scoring relationship has occurred; (2) Rescoring and label update: After drift is triggered, the baseline quality score of the historical window is recalculated according to the current quality rules to update the training label set; (3) Fast scorer update: Based on the updated training data, the fast scorer is retrained or incrementally updated to adapt the predicted score to the new data distribution and quality rules.

4. S6 in the method according to claim 1, characterized in that, The governance actions include at least: adjustment of acquisition strategy, retrying acquisition, cleaning of abnormal records, completion of missing fields, and adjustment of entity recognition strategy parameters; wherein completion of missing fields includes at least rule-based completion, completion based on historical window statistics, or completion based on learning models; the characterization of quality change trends includes at least trend statistics based on sliding time windows, trend discrimination based on the rate of change, or trend discrimination based on drift detection signals.

Images