Log processing method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a log processing method and device, equipment, a storage medium and a program product. The method comprises the steps of obtaining a to-be-processed log feature vector by extracting features of a to-be-processed log; obtaining a historical log processing operation, a historical downstream model performance index and a historical data quality index; constructing a current state space based on the to-be-processed log feature vector, the historical log processing operation, the historical downstream model performance index and the historical data quality index; inputting the current state space into a log processing model, and enabling the log processing model to select a target action from an action space based on the current state space; wherein the action space is composed of a plurality of discrete log processing operations and a plurality of continuous parameters; the to-be-processed log is processed by using the target action, different log processing requirements can be dynamically adapted, and the accuracy of log processing is improved.

Description

Technical Field

[0001] This application belongs to the field of text processing technology, and in particular relates to a log processing method, apparatus, device, storage medium and program product. Background Technology

[0002] As the core carrier of computer system operating status, user behavior, and abnormal events, logs directly determine the accuracy and reliability of subsequent analysis and mining tasks. Preprocessing logs can effectively repair data defects in the original logs and improve log quality.

[0003] In existing technologies, log preprocessing mainly relies on manual rules, that is, technicians predefine a series of data cleaning and transformation rules based on their experience. Technicians need to select different log processing operations according to the log format and data characteristics in a specific scenario.

[0004] However, in complex log scenarios, log processing operations based on manually selected rules cannot adapt to different log processing needs, resulting in reduced accuracy and usability of log processing. Summary of the Invention

[0005] This application provides a log processing method, apparatus, device, storage medium, and program product that can dynamically adapt to different log processing needs and improve the accuracy of log processing.

[0006] In a first aspect, embodiments of this application provide a log processing method, the method comprising: Extract the features of the logs to be processed to obtain the feature vector of the logs to be processed; Obtain historical log processing operations, historical downstream model performance metrics, and historical data quality metrics. Among them, the historical downstream model performance metrics are used to characterize the impact of historical log processing operations on the performance of downstream task models, and the historical data quality metrics are used to characterize the impact of historical log processing operations on log quality. The current state space is constructed based on the feature vector of the log to be processed, historical log processing operations, historical downstream model performance indicators, and historical data quality indicators. The current state space is input into the log processing model, so that the log processing model selects the target action from the action space based on the current state space. The action space consists of multiple discrete log processing operations and multiple continuous parameters. The continuous parameters are used to continuously numerically control the parameters required in the discrete log processing operations. Each action in the action space consists of a discrete log processing operation and at least one continuous parameter corresponding to the discrete log processing operation. Use the target action to process the logs to be processed.

[0007] In one feasible implementation, the method further includes: Obtain log processing training set data. The log processing training set data includes multiple log processing training samples. Each log processing training sample includes: the current state space, which consists of the current log feature vector training sample, the historical log processing operation training sample, the historical downstream model performance index training sample, and the historical data quality index training sample. For each log processing training sample, perform the following steps: Input the current log feature vector training sample, historical log processing operation training sample, historical downstream model performance index training sample, and historical data quality index training sample into the log processing model to be trained to obtain the log processing operation of the training sample. Based on the log processing operations of the training sample, the reward function value is calculated. The reward function value is calculated based on the data quality index, downstream model performance index, downstream model loss function, operation cost, and their respective weights corresponding to the log processing operations of the training sample. Determine whether the reward function value meets the preset training stopping condition; If the conditions are not met, adjust the parameters of the log processing model to be trained, and continue training the log processing model using the log processing training set data until the preset training stopping condition is met, and obtain the trained log processing model.

[0008] In one feasible implementation, the method further includes: For each log processing training sample, obtain the next state space after the log processing operation of the training sample is executed. The next state space includes: new log feature vector training sample, current log processing operation training sample, current downstream model performance index training sample, and current data quality index training sample. Input the current state space and the next state space into the critic model to obtain the current state estimate and the next state estimate; Calculate the advantage estimate based on the current state estimate, the next state estimate, and the reward function value; Based on the advantage estimate and the current state estimate, calculate the value loss function value corresponding to the critic model; Based on the gradient descent algorithm, the parameters of the critic model are updated by minimizing the value loss function.

[0009] In one feasible implementation, the method further includes: Calculate the ratio of the probability of the log processing operation corresponding to the current training sample to the probability of the log processing operation corresponding to the historical training sample, and obtain the policy improvement ratio. Based on the advantage estimate and the strategy improvement ratio, the pruning strategy loss value is calculated using the pruning objective function algorithm. Calculate the entropy value of the actor model's output log processing operation; The strategy loss function value is calculated based on the pruning strategy loss value, entropy value, and the weights corresponding to the entropy value. Based on the gradient ascent algorithm, the parameters of the actor model are updated by minimizing the policy loss function value.

[0010] In one feasible implementation, the method further includes: Obtain validation set data, periodically evaluate the performance of the current log processing model on the validation set, and obtain validation performance metrics; Monitor and verify the changing trends of performance indicators; If the trend of change does not meet the preset trend of change threshold, the preset training stop condition is determined to be met.

[0011] In one feasible implementation, the method further includes: Discrete log processing operations include at least one of the following: field completion, synonym replacement, log truncation, log concatenation, format standardization, field mapping, and regular expression filtering. The continuous parameters include at least one of the following: noise intensity, occlusion rate, regularity threshold sensitivity, field value jitter rate, text truncation rate, and diversity enhancement coefficient.

[0012] Secondly, embodiments of this application provide a log processing apparatus, the apparatus comprising: The extraction module extracts features from the logs to be processed, obtaining the feature vector of the logs to be processed. The acquisition module is used to acquire historical log processing operations, historical downstream model performance indicators, and historical data quality indicators. Among them, the historical downstream model performance indicators are used to characterize the impact of historical log processing operations on the performance of downstream task models, and the historical data quality indicators are used to characterize the impact of historical log processing operations on log quality. The building module is used to construct the current state space based on the feature vector of the log to be processed, historical log processing operations, historical downstream model performance indicators, and historical data quality indicators; The first processing module is used to input the current state space into the log processing model, so that the log processing model selects the target action from the action space based on the current state space; wherein, the action space consists of multiple discrete log processing operations and continuous parameters corresponding to each log processing operation, and the continuous parameters are used to continuously numerically control the parameters required in the discrete log processing operations. Each action in the action space consists of a discrete log processing operation and at least one continuous parameter corresponding to the discrete log processing operation. The second processing module is used to process the logs to be processed using the target action.

[0013] Thirdly, embodiments of this application provide an electronic device, the device including: a processor, and a memory storing computer program instructions; The processor reads and executes computer program instructions to implement a log processing method that achieves any of the first aspects.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement a log processing method as described in any of the first aspects.

[0015] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform a log processing method as described in the first aspect.

[0016] The log processing method, apparatus, device, storage medium, and program product provided in this application can construct a current state space based on the feature vector of the log to be processed, historical log processing operations, historical downstream model performance indicators, and historical data quality indicators. This state space not only covers the core feature vector of the log to be processed, but also integrates historical preprocessing behavior records, feedback on the performance impact of past processing on downstream models, and dynamic changes in log data quality. This enables the log processing model to fully perceive the current processing environment and historical state evolution, providing sufficient contextual information for decision-making. Furthermore, the action space in this application consists of multiple discrete log processing operations and multiple continuous parameters. Therefore, when the current state space is input into the log processing model, the log processing model can simultaneously decide on the processing operation type and the corresponding execution parameters. By decoupling and recombining the operation type selection and parameter intensity adjustment, the flexibility and adaptability of selecting log processing strategies for the log to be processed are effectively improved. Therefore, the log processing operations obtained based on the log processing model can dynamically adapt to the characteristics and processing needs of different log data, thereby effectively improving the accuracy and practicality of log processing. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of a log processing method provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating a log processing method is shown. Figure 3 A framework diagram of the actor critic algorithm is shown; Figure 4 This diagram illustrates a training flowchart for an actor critic model. Figure 5 A schematic diagram of the structure of a log processing device provided in this application is shown; Figure 6 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0019] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0020] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0021] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0022] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0023] Currently, existing technologies for preprocessing logs mainly rely on the prior experience of technical personnel, pre-defining a series of data cleaning and transformation rules. Technical personnel need to select different log processing operations based on different log characteristics. However, in complex log scenarios, the needs of log processing are constantly changing, and pre-defined log processing operations cannot be adapted to all logs, resulting in reduced accuracy and practicality of log processing.

[0024] To address the problems of existing technologies, embodiments of this application provide a log processing method, apparatus, device, storage medium, and program product. By constructing a current state space based on the feature vector of the log to be processed, historical log processing operations, historical downstream model performance indicators, and historical data quality indicators, the log processing model can fully perceive the current processing environment and historical state evolution, providing sufficient contextual information for decision-making. Furthermore, the action space in this application consists of multiple discrete log processing operations and multiple continuous parameters. Therefore, when the current state space is input into the log processing model, the log processing model can simultaneously decide on the processing operation type and the corresponding execution parameters. By decoupling and recombining the operation type selection and parameter intensity adjustment, the flexibility and adaptability of selecting log processing strategies for the log to be processed are effectively improved. Therefore, the log processing operations obtained based on the log processing model can dynamically adapt to the characteristics and processing needs of different log data, thereby effectively improving the accuracy and practicality of log processing.

[0025] The following section first introduces a log processing method provided in an embodiment of this application.

[0026] Figure 1 A flowchart illustrating a log processing method provided in an embodiment of this application is shown. Figure 1 As shown, the method may include the following steps: S101: Extract the features of the logs to be processed to obtain the feature vector of the logs to be processed; S102: Obtain historical log processing operations, historical downstream model performance indicators, and historical data quality indicators. Among them, the historical downstream model performance indicators are used to characterize the impact of historical log processing operations on the performance of downstream task models, and the historical data quality indicators are used to characterize the impact of historical log processing operations on log quality. S103: Construct the current state space based on the feature vector of the log to be processed, historical log processing operations, historical downstream model performance indicators, and historical data quality indicators; S104: Input the current state space into the log processing model, so that the log processing model selects the target action from the action space based on the current state space; S105: Use the target action to process the log to be processed.

[0027] In this embodiment, compared to the prior art which processes logs according to predefined manual rules, resulting in the selected log processing operations being unable to adapt to different log processing needs, this application constructs a current state space based on the feature vector of the log to be processed, historical log processing operations, historical downstream model performance indicators, and historical data quality indicators. This state space not only covers the core feature vector of the log to be processed but also integrates historical preprocessing behavior records, feedback on the performance impact of past processing on downstream models, and dynamic changes in log data quality. This enables the log processing model to fully perceive the current processing environment and historical state evolution, providing sufficient contextual information for decision-making. Furthermore, the action space in this application consists of multiple discrete log processing operations and multiple continuous parameters. Therefore, when the current state space is input into the log processing model, the log processing model can simultaneously decide on the processing operation type and the corresponding execution parameters. By decoupling and recombining the operation type selection and parameter intensity adjustment, the flexibility and adaptability of selecting log processing strategies for the log to be processed are effectively improved. Therefore, the log processing operations obtained based on the log processing model can dynamically adapt to the characteristics and processing needs of different log data, thereby effectively improving the accuracy and practicality of log processing.

[0028] The specific implementation methods for each of the above steps are described below.

[0029] S101: Extract the features of the logs to be processed to obtain the feature vector of the logs to be processed.

[0030] In this embodiment of the application, the original log to be processed is unstructured text. The log processing model cannot directly understand the semantics of the text and needs to convert the text into a computable feature vector of the log to be processed.

[0031] In one example, semantic features of the log to be processed can be extracted using the Bidirectional Encoder Representations from Transformers (BERT) model.

[0032] In another example, numerical characteristics of the log to be processed can be calculated using statistical methods, which include, but are not limited to, one or more of the following: text length statistics, lexical diversity statistics, numerical field statistics, structural feature statistics, and character-level statistics.

[0033] S102: Obtain historical log processing operations, historical downstream model performance indicators, and historical data quality indicators. Among them, the historical downstream model performance indicators are used to characterize the impact of historical log processing operations on the performance of downstream task models, and the historical data quality indicators are used to characterize the impact of historical log processing operations on log quality. In this embodiment, if the log processing model relies solely on the current log feature vector for decision-making, it cannot understand the effects of past processing operations, potentially leading to duplicate or conflicting operations. Therefore, by acquiring historical log processing operations, duplicate or conflicting operations in the preprocessing sequence can be avoided. Historical downstream model performance metrics reflect the impact of historical log processing operations on the performance of downstream task models, while historical data quality metrics reflect the impact of historical log processing operations on log quality. By fusing historical downstream model performance metrics and historical data quality metrics, the log processing model can perceive the consequences of its past actions, thereby learning the sequential relationships between operations.

[0034] In one example, historical log processing operations, historical downstream model performance metrics, and historical data quality metrics can refer to a single operation record generated by the last preprocessing operation and its corresponding performance and quality feedback metrics, or they can refer to an operation sequence composed of multiple past preprocessing operations and its corresponding comprehensive performance and quality evaluation metrics. Specifically, operations at multiple time steps and their corresponding performance and quality feedback metrics can be aggregated into a unified feature representation through sliding window averaging, weighted cumulative averaging, or sequence encoding methods based on recurrent neural networks.

[0035] In one example, a downstream model refers to a machine learning model that receives processed log data as input and performs specific analysis tasks, including but not limited to at least one of anomaly detection models, fault prediction models, security threat identification models, or log classification models.

[0036] Downstream model performance metrics include at least one of precision, recall, F1 score, and AUC, or a combination of precision, recall, F1 score, and AUC. The following explanation uses a combination of precision (P), recall (R), F1 score (F1), and area under the curve (AUC) as the downstream model performance metrics.

[0037] Accuracy definition: (1) in, These are the true positive samples predicted by the downstream model; These are incorrect positive samples predicted by the downstream model.

[0038] Recall rate definition: (2) in, These are the true positive samples predicted by the downstream model; These are incorrect negative samples predicted by the downstream model.

[0039] F1 score definition: (3) Where P is the precision of the downstream model and R is the recall of the downstream model.

[0040] AUC definition: (4) in, The ranking of the true positive samples predicted by the downstream model; These are the true positive samples predicted by the downstream model; These are incorrect negative samples predicted by the downstream model.

[0041] Therefore, the difference between the evaluation metrics of the logs before and after preprocessing by the log processing model and the logs passed to the same downstream model is used as the evaluation metric for the overall performance of the downstream model. The specific expression is: (5) in, This represents the overall performance indicators of the downstream model; The weights represent the precision. This indicates the weight corresponding to the recall rate; This indicates the weight corresponding to the F1 score; This indicates the weight corresponding to the area under the curve; This represents the difference in accuracy metrics obtained from the log processing model before and after preprocessing on the same validation set. This represents the difference in recall rate between the log processing model before and after preprocessing on the same validation set. This represents the difference in F1 scores between the log processing model before and after preprocessing on the same validation set. This represents the difference in the area under the curve before and after preprocessing of the log processing model obtained on the same validation set. Represents the normalization function; It should be noted that: , , , The sum of is 1; , , as well as This refers to directly inputting the original validation set data into the downstream model to obtain the data before preprocessing. , , as well as Then, the original validation set data is input into the log processing model to obtain the corresponding log processing operations. After processing the validation set data based on the log processing operations, the processed validation set data is then input into the downstream model for further preprocessing. , , , The difference between the indicators before and after preprocessing is obtained.

[0042] In one example, a data quality indicator refers to the quality change of log data before and after preprocessing by a log processing model. Data quality indicators include at least one of the following: missing rate, noise ratio, field consistency, and duplication rate, or a combination of missing rate, noise ratio, field consistency, and duplication rate as a data quality indicator. The following explanation uses a combination of missing rate, noise ratio, field consistency, and duplication rate as a data quality indicator.

[0043] Missing rate definition: (6) in, Indicates the missing rate; The total number of missing fields after processing by the log processing model; This represents the total number of fields.

[0044] Noise ratio definition: (7) in, Indicates the noise ratio; The total number of noisy characters after processing by the log processing model; This represents the total number of characters.

[0045] Field Consistency Scoring definition: (8) in, Indicates the consistency score of the field; The total number of records that conform to the normal field format after processing by the log processing model; This represents the total number of records.

[0046] Repetition rate definition: (9) in, Indicates the repetition rate; The total number of duplicate fields after processing by the log processing model; This represents the total number of fields.

[0047] Therefore, data quality metrics of logs before and after preprocessing are obtained through log processing models. The specific expression is: (10) in, The weights corresponding to the missing rate; The weights corresponding to the noise ratio; This indicates the weight corresponding to the field consistency score; This represents the weight corresponding to the repetition rate; Indicates the missing rate; Indicates the noise ratio; Indicates the consistency score of the field; This indicates the repetition rate; it should be noted that: , , , The sum is 1.

[0048] S103: Construct the current state space based on the feature vector of the log to be processed, historical log processing operations, historical downstream model performance indicators, and historical data quality indicators.

[0049] In this embodiment of the application, the feature vector of the log to be processed with a single state is... Expanded into a four-part concatenation: the feature vector of the log to be processed , Historical log processing operations Historical downstream model performance metrics Historical data quality indicators. Among them, Indicate execution The logs obtained after historical log processing operations are input into the downstream model to obtain the downstream model's performance metrics. Indicate execution The data quality of logs obtained after historical log processing operations.

[0050] Specifically, the current state space is defined as: (11) in, Represents the current state space; This is the current log feature vector; The encoding corresponding to historical log processing operations; These are downstream model performance metrics, including precision, recall, F1 score, and AUC. These are data quality metrics, including missing rate, noise ratio, consistency score, and duplication rate.

[0051] This is achieved by storing log feature vectors in the state space in addition to the current state. In addition, it can also process the codes corresponding to historical log operations. Record the historical processing operations of the log processing model to avoid duplicate or conflicting operations in the preprocessing sequence; and use downstream model performance metrics. and data quality indicators The impact of logging operations on downstream model iterations and optimizations is recorded in the log processing model's historical log processing operations, compared to relying solely on... Compared with existing technology designs, this solution integrates information on three aspects: historical behavior, downstream model performance, and the quality of logs generated by log processing operations. This enables the log processing model to have multi-step preprocessing decision-making capabilities that take into account both historical decision feedback and subsequent optimization goals, and can achieve continuous and efficient preprocessing optimization on a single log or a batch of logs.

[0052] S104: Input the current state space into the log processing model, so that the log processing model selects the target action from the action space based on the current state space; In this embodiment, each action in the action space consists of a discrete log processing operation and at least one continuous parameter corresponding to that discrete log processing operation. The current state space is input into the log processing model, which can then select a log processing operation and its corresponding continuous parameter from the action space based on the current state space.

[0053] In one example, the action space consists of multiple discrete log processing operations and multiple continuous parameters; that is, the action space is defined as: ,in Discrete operations refer to a set of predefined log operations that can be switched on and off. Only one of them can be selected and executed at a time, and it is used to complete clearly defined, rule-based tasks. For continuous parameter adjustment, it refers to the continuous numerical control of the parameters required in discrete operations, supporting fine-grained adjustment of enhancement intensity, sampling ratio, etc.

[0054] In one example, one specific way to compose the action space includes: Discrete log processing operations include at least one of the following: field completion, synonym replacement, log truncation, concatenation, format standardization, field mapping, and regular expression filtering.

[0055] In this embodiment, field completion refers to filling missing fields with default values ​​or model predictions to ensure the integrity of the log structure; synonym replacement refers to replacing words in the log text with synonyms to increase sample diversity; log truncation refers to truncating excessively long messages; log concatenation refers to concatenating multiple logs from the same session; format standardization refers to unifying field order, capitalization, time format, etc.; field mapping refers to mapping custom fields to a unified feature namespace; and regular expression filtering refers to removing abnormal characters or retaining or eliminating specific patterns based on preset regular expressions.

[0056] The continuous parameters include at least one of the following: noise intensity, occlusion rate, regularity threshold sensitivity, field value jitter rate, text truncation rate, and diversity enhancement coefficient.

[0057] In the embodiments of this application, noise intensity This refers to the probability of randomly inserting noise into text; masking rate. This refers to the proportion of masked words or characters; regular expression threshold sensitivity. This refers to adjusting the threshold for regular expression filtering or confidence filtering; field value jitter rate. This refers to the scale of adding Gaussian noise to a numerical field; text truncation rate. This refers to retaining the original text before Proportion; Diversity Enhancement Coefficient This refers to the degree of diversity in controlling operations such as synonym substitution and back-translation.

[0058] In one example, the range of values ​​for a continuous parameter can be: .

[0059] In one example, the action space of this application is pluggable, for example, adding or deleting discrete operations and continuous parameters are both supported, without the need to reconstruct the training framework.

[0060] In this embodiment, the action space is divided into two main categories: discrete operations and continuous parameter adjustment. Discrete operations correspond to a set of switchable predefined log processing actions. Each time, a single operation is selected from this set for execution, realizing a clear rule-based processing function. Continuous parameter adjustment, on the other hand, continuously and numerically controls the execution parameters associated with discrete operations, enabling fine-grained adjustment of the operation intensity. By decoupling and recombining operation type selection and parameter intensity adjustment, the flexibility and adaptability of the preprocessing strategy are effectively improved, enabling the generation of more accurate customized log processing operations for log data with different characteristics.

[0061] S105: Use the target action to process the log to be processed.

[0062] In this embodiment, specific log processing operations are performed on the log to be processed through the target action, thereby optimizing the log data quality and providing high-quality input for downstream tasks.

[0063] Figure 2 A flowchart illustrating a log processing method is shown, such as... Figure 2 As shown, S201: Business log data; S202: Agent decision-making; S203: Data preprocessing; S204: Downstream model processing.

[0064] This technical solution optimizes the log data processing flow by introducing a deep reinforcement learning mechanism. By modeling the preprocessing of log samples as a Markov Decision Process (MDP), the intelligent agent is designed to automatically learn the optimal operation strategy under different data quality, context structure, and model feedback conditions, thereby dynamically selecting operations such as enhancement, cleaning, and normalization to ensure the performance of the downstream anomaly detection model.

[0065] S201: Business log data. By acquiring business log data, the business log data is input into the intelligent agent model for processing. The intelligent agent model is the log processing model of this application. The intelligent agent model has adaptive decision-making and strategy optimization capabilities and can autonomously adjust the processing logic based on multi-dimensional state feedback. Therefore, the intelligent agent model can autonomously match and adapt the processing strategy to the current log data. S202: Agent decision-making. The agent can select an action from the action space based on the current state space for the current business log data. Each action corresponds to a log processing operation. S203: Data preprocessing. After the agent selects an action for the current business log data, it preprocesses the business log data by utilizing the log processing operation corresponding to that action. S204: Downstream model processing. The preprocessed log data is input into the downstream model for processing. Since the log data has been preprocessed to eliminate data defects and standardize the format, the data parsing and cleaning cost of the downstream task model can be greatly reduced, the interference of invalid data on the model training or inference process can be reduced, the efficiency of the downstream model in extracting effective features can be improved, and thus the accuracy, reliability and generalization ability of the output results of the downstream model can be guaranteed.

[0066] The above process describes the application of the log processing model. The training process of the log processing model will be explained below. The log processing model in this application is trained and optimized using Deep Reinforcement Learning (DRL) within the Reinforcement Learning (RL) model.

[0067] In one example, one way to train a log processing model is as follows: Obtain the log processing training set data, which includes multiple log processing training samples. Each log processing training sample includes: the current state space, which consists of the current log feature vector training sample, the historical log processing operation training sample, the historical downstream model performance index training sample, and the historical data quality index training sample.

[0068] In this embodiment, the agent can sample data from a log database storing raw business log data in batches. For each interaction, a log batch is randomly or sequentially retrieved from the database. And convert it into a state vector. That is, to transform into the current state space.

[0069] In one example, to ensure the diversity and representativeness of the training samples, different log batches and log sources can be set. By combining batch parallel collection, a small number of labeled logs can be continuously optimized on a large number of unlabeled logs.

[0070] For each log processing training sample, perform the following steps: Input the current log feature vector training sample, historical log processing operation training sample, historical downstream model performance index training sample, and historical data quality index training sample into the log processing model to be trained to obtain the log processing operation of the training sample.

[0071] Based on the log processing operations of the training sample, the reward function value is calculated. The reward function value is calculated based on the data quality index, downstream model performance index, downstream model loss function, operation cost, and their respective weights corresponding to the log processing operations of the training sample.

[0072] In this embodiment, the reward function, as a key module of the iterative strategy of the deep reinforcement learning agent, not only determines the agent's immediate feedback to different behaviors but also indirectly affects the convergence speed and reachability of the optimal strategy throughout the learning process. To avoid problems such as the log processing model's strategy converging to local optima or overfitting preprocessing rules, this application constructs a multi-objective improved reward function design strategy. The formula for the reward function is defined as follows: (12) in, This represents the reward function value; Indicates the data quality after log processing operations; This indicates the impact of log processing operations on downstream model performance metrics; This indicates the difference in loss functions between the preprocessed and downstream models; This indicates the operation cost corresponding to the log processing operation; This indicates the weight corresponding to the data quality after log processing operations; This represents the weight corresponding to the impact of log processing operations on the performance metrics of downstream models. Indicates the weights corresponding to the differences in the loss functions of the downstream models after preprocessing; This indicates the weight of the operation cost corresponding to this log processing operation. It should be noted that: , , , The sum of is 1.

[0073] In one example, the formula for calculating the difference in loss functions between downstream models is: (13) in, This indicates the difference in loss functions between the preprocessed and downstream models; This represents the downstream model loss function when the logs after log processing are input into the downstream model. This represents the downstream model loss function when the original logs are input into the downstream model.

[0074] In another example, the operational cost of a log processing operation can consist of at least one of time cost and resource cost, or both: (14) in, This indicates the operation cost corresponding to the log processing operation; Indicates time cost; This indicates the cost of resources. This indicates the weight corresponding to time cost; This indicates the weight corresponding to the resource cost; it should be noted that: and The sum of is 1.

[0075] In one example, for a batch size of The log, if the operation Average time per log entry (milliseconds), then time cost ;operate The computational load consumed on the CPU or GPU, if the processing time per log entry is [per second]. One Central Processing Unit (CPU) and Each graphics processing unit (GPU) has a batch cost. In one example, and Divide by maximum resource Normalization is performed.

[0076] Determine whether the reward function value meets the preset training stopping condition.

[0077] In this embodiment, the preset training stopping condition can be that the reward function value reaches a preset reward function threshold, or the number of training iterations reaches a preset number of training iterations.

[0078] If the conditions are not met, adjust the parameters of the log processing model to be trained, and continue training the log processing model using the log processing training set data until the preset training stopping condition is met, and obtain the trained log processing model.

[0079] In this embodiment, if the preset training stop condition is not met, the model parameters of the log processing model to be trained can be adjusted based on the currently calculated reward function value, and then the log processing model can be trained again using the log processing training set data until the preset training stop condition is met, thus obtaining the trained log processing model.

[0080] In this embodiment, to address the technical problem of existing log processing methods having a single optimization objective and difficulty in balancing multi-dimensional performance indicators, this embodiment constructs a multi-objective reward function that comprehensively measures data processing quality, downstream model performance, and operational costs. This function considers data quality indicators, downstream model performance indicators, downstream model loss functions, and operational costs. This avoids problems in traditional methods where the log processing model strategy converges to a local optimum due to a single optimization objective, or overfits to specific preprocessing rules. Guided by this multi-objective reward mechanism, the agent can autonomously explore and learn the optimal strategy for balancing data quality, model performance, and resource efficiency during training. This makes log text processing operations under the guidance of the reinforcement learning agent more suitable for log processing needs, thereby improving the accuracy and practicality of log processing.

[0081] In one example, we explore introducing delayed rewards, hierarchical rewards, or adversarial rewards to increase the agent's attention to anomalous samples. Building on this, to avoid low learning efficiency due to sparse rewards, we can start with short-term delayed rewards in a sandbox environment, testing short-to-medium-term delays first, and then gradually expanding the delay range; using an entropy bonus to encourage exploration and prevent the policy from getting stuck in local optima early on. Alternatively, to avoid instability caused by too many hierarchical rewards, we can first verify the effect of each reward layer in stages in a simulation environment, identifying the 1-2 most critical layers; normalizing or weighting the rewards for each layer, and preventing single-layer rewards from dominating policy updates. Or, to avoid excessive adversarial rewards leading to over-adversarial behavior or system collapse, we can design an adversarial failure protection mechanism: when the adversarial reward trigger rate exceeds a threshold, automatically revert to the baseline policy without adversarial rewards. Initially, we can insert adversarial rewards at a small percentage (e.g., 10%), observe system stability, and then gradually increase the weight.

[0082] In another example, a policy network can be pre-trained using an autoencoder or contrastive learning approach, followed by fine-tuning with reinforcement learning to accelerate convergence. Building on this, to avoid catastrophic forgetting due to inconsistencies between pre-training and reinforcement learning objectives, a layered freezing strategy can be employed: fine-tuning only high-level network parameters while keeping low-level features unchanged; retaining the original unsupervised loss term in the initial fine-tuning phase, and optimizing it with a flexible weight (e.g., 0.1) mixed with the RL loss. Alternatively, to avoid poor transfer learning due to significant differences between pre-training data distribution and real logs, small batches of real logs can be introduced for mixed training during the pre-training phase; domain adaptation techniques (e.g., adding a Domain Discriminator in contrastive learning) can be used to reduce the distribution gap. Alternatively, to avoid high computational overhead during pre-training and resource consumption affecting subsequent RL training, single-machine multi-GPU parallel training can be implemented during pre-training, releasing the pre-training environment after completion; controlling the number of pre-training rounds: first using small-scale, rapid iterations to verify the effect, then deciding whether to perform large-scale pre-training.

[0083] In another example, this application specifically uses the Actor-Critic (AC) algorithm of Proximal Policy Optimization (PPO) to adaptively select the corresponding log processing operation. To clearly illustrate the training process of the AC algorithm in this application, this application first explains the framework of the AC algorithm. Figure 3 The framework diagram of the actor critic algorithm is shown, such as... Figure 3 As shown, within the framework of the AC algorithm, the agent algorithm consists of the following three parts: the interaction environment, which in this scheme consists of a log database and the same downstream model; the actor, which plays the role of "decision-maker," whose input is the current state vector. Output the corresponding action The critic, who plays the role of "value assessor," also receives a state vector as input. Output the estimated value of the current state. It is used to evaluate the effectiveness of an actor's current strategy.

[0084] In one example, when the log processing model is an actor critic model, the parameter training process of the critic model is as follows: For each log processing training sample, obtain the next state space after the log processing operation of the training sample is executed. The next state space includes: new log feature vector training sample, current log processing operation training sample, current downstream model performance index training sample, and current data quality index training sample.

[0085] In this embodiment, the critic model needs to evaluate the long-term benefits of the current action. Therefore, it needs to obtain the next state space after executing the current action to provide a basis for long-term benefit evaluation. The next state space includes training samples of new log feature vectors, training samples of the current log processing operation, training samples of the current downstream model performance indicators, and training samples of the current data quality indicators. The next state space is formed by executing the log processing operation of the current training samples, obtaining the processed logs, and extracting new log feature vectors. The currently executed processing operation, the corresponding downstream performance indicators and data quality indicators are combined with the new log feature vectors to form the next state space.

[0086] In one example, a fixed downstream model is used, such as a fixed anomaly detection or classification model (e.g., a Transformer-based log classifier or a traditional random forest model), to handle each batch of preprocessed logs received. Then, a forward evaluation is performed, outputting performance metrics and the loss value for the current batch. These feedbacks serve as the environment's response to the agent's actions. Immediate response, used to construct rewards And drive subsequent states The generation of .

[0087] In one example, an older strategy based on the actor critic model could be used. Data was collected through interaction with the environment. A set of state transitions: ,in, Denotes the current state space in round t. This represents the target action selected in round t. Indicates the action to be performed in round t. The reward function value after that; Indicates the action to be performed in round t. The next state space.

[0088] Input the current state space and the next state space into the critic model to obtain the current state estimate and the next state estimate.

[0089] In this embodiment of the application, the commentator is responsible for estimating the state value. This provides a reliable benchmark for updating actor strategies. By inputting the current state space and the next state space into the critic model, the current state estimate can be obtained. and the next state estimate .

[0090] The advantage estimate is calculated based on the current state estimate, the next state estimate, and the reward function value.

[0091] In this embodiment of the application, the critic model cannot directly fit the target value return during training; it is necessary to calculate the critic's value target. The advantage estimate was calculated previously.

[0092] In one example, this scheme employs generalized advantage estimation, which is suitable for complex log processing tasks and can effectively handle multi-objective rewards. Therefore, the formula for calculating the advantage estimate is: (15) in, This represents the advantage estimate; Indicates time difference error. ,in, Represents the reward function; This represents the estimated value of the next state; This represents the estimated current state. It is a hyperparameter that balances the bias and variance; It is a discount factor; This indicates the time step offset index.

[0093] Based on the advantage estimate and the current state estimate, calculate the value loss function value corresponding to the critic model; In this embodiment, based on the advantage estimate and the current state estimate, the value loss function value corresponding to the critic model is calculated. The value loss function encourages the value estimate output by the critic model. More accurate, thus helping actor models make better strategy adjustments.

[0094] Before calculating the value loss function, the commentator's value objective is calculated using approximate estimations based on generalized advantage estimation and current state estimation. : (16) in, This indicates the value objective of the critic model; This represents the advantage estimate; This represents the current state estimate.

[0095] Therefore, according to value objectives The network parameters of the critic model Updates can be made based on mean squared error loss: (17) in, This represents the value of the loss function corresponding to the critic model; This represents the expectation operator for the training samples; Indicates the current state estimate; This indicates the value objective of the critic model.

[0096] Based on the gradient descent algorithm, the parameters of the critic model are updated by minimizing the value loss function.

[0097] In this embodiment, gradient descent is used to evaluate the value network parameters. Updates are performed by backpropagating to minimize the value loss function, updating the parameters of the critic model. After multiple rounds of updates, the new value network is synchronized to the policy optimization of the next batch of actors. The expression for updating the critic model's parameters is as follows: (18) in, Represents the parameters of the critic model; The value loss function represents the parameter The gradient; This represents the learning rate, which controls the step size for parameter updates.

[0098] In this embodiment, when the log processing model is an actor-critic model, the current state space and the next state space are input into the critic model to obtain the current state estimate and the next state estimate. This enables the critic model to perform value prediction based on temporal difference learning, overcoming the short-sighted limitation of traditional methods that only consider immediate rewards. It achieves accurate prediction of future long-term returns. Furthermore, based on the generalized advantage estimation algorithm, the advantage estimate is calculated by combining the current reward and the value of continuous states, effectively balancing the bias and variance in the estimation process. This provides a stable and reliable value benchmark for policy evaluation. Through the iterative optimization process of minimizing the value loss function value, the accuracy of the critic network's state value prediction is continuously improved, thereby providing an accurate value basis for the policy optimization of the actor network.

[0099] In one example, when the log processing model is the actor critic model, the parameter training process of the actor model is as follows: Calculate the ratio of the probability of the current log processing operation corresponding to the training sample to the probability of the log processing operation corresponding to the training sample in the past, and obtain the policy improvement ratio.

[0100] In this embodiment, to avoid excessive policy updates in the actor model that could lead to training instability, this application limits the update magnitude of the actor model by calculating the policy improvement ratio. The policy improvement ratio is the ratio of the probability that the new actor model outputs the current action to the probability that the old actor model outputs the same action. The calculation formula is as follows: (19) in, Indicates the rate of policy improvement; express; express Based on the advantage estimate and the policy improvement ratio, the pruning policy loss value is calculated using the pruning objective function algorithm.

[0101] In this embodiment of the application, if the policy improvement ratio is too large, it may lead to overfitting of the actor model or a sharp drop in performance. It is necessary to limit the range of the ratio by pruning the objective function to ensure smooth policy improvement. The pruning objective function of the PPO algorithm can be used to calculate the policy loss value after pruning. When the improvement of the new policy is too large, it is pruned to an acceptable range to prevent policy collapse and avoid abnormal fluctuations in the loss function value.

[0102] In one example, the formula for calculating the policy pruning loss is: (20) in, This represents the loss value of the pruning strategy; This represents the approximate advantage function, which measures the performance of the current action relative to the average policy. In this application, a generalized advantage estimation approximation is used. ; Indicates the rate of policy improvement; This represents the expectation operator for the training samples; This represents the hyperparameter that controls the update magnitude; This represents the clipping function.

[0103] Calculate the entropy value of the actor model output log processing operation.

[0104] In this embodiment of the application, in order to prevent the actor model from getting stuck in local optima and to encourage its strategy to explore more diverse action behaviors, an entropy regularization term can be added to the policy objective, that is, to calculate the entropy value of the actor model's output log processing operation: (twenty one) in, Represents the entropy value; Indicates the current state Next, actor strategy Uncertainty regarding action selection; This represents the expectation operator for the training samples.

[0105] The strategy loss function value is calculated based on the pruning strategy loss value, entropy value, and the weights corresponding to the entropy value.

[0106] In this embodiment, the policy loss function needs to take into account both policy improvement and action exploration. The intensity of exploration is adjusted by using entropy weights to ensure that the model optimizes known actions without abandoning potential actions.

[0107] In one example, the formula for calculating the policy loss function value is: (twenty two) in, This represents the value of the policy loss function; This represents the loss value of the pruning strategy; Represents the entropy value; The weight represents the entropy value and is used to control the strength of entropy regularization.

[0108] Based on the gradient ascent algorithm, the parameters of the actor model are updated by minimizing the policy loss function value.

[0109] In this embodiment, the output of the actor model is a set of joint policies, namely a hybrid policy of discrete actions and continuous parameter adjustment. Therefore, the policy network of the actor model can be represented as: (twenty three) in, Represents the joint probability of the target action; Represents the selection probability of a discrete operation; This represents the probability of choosing a continuous parameter.

[0110] By using the gradient ascent algorithm to optimize the policy network parameters of the actor model To update the actor model parameters, backpropagation is used to minimize the policy loss function value. After multiple rounds of optimization, the updated policy is then applied to the old policy. The expression for updating the actor model parameters is as follows: (twenty four) in, Parameters representing the actor model; The policy loss function is expressed as a function of parameters. The gradient; This represents the learning rate, which controls the step size for parameter updates.

[0111] In this embodiment, the policy improvement ratio is obtained by calculating the ratio of the probability of the log processing operation corresponding to the current training sample to the probability of the log processing operation corresponding to the historical training sample. Then, based on the advantage estimate and the policy improvement ratio, the pruning policy loss value is calculated using a pruning objective function algorithm. This effectively limits the magnitude of policy updates, preventing training instability and performance collapse caused by excessively large single policy updates, and ensuring smooth convergence of the training process. Next, the entropy value of the log processing operation output by the actor model is calculated and added as a regularization term to the loss function. This effectively maintains the policy's exploration capability, prevents the actor model from converging prematurely to a local optimum, promotes its exploration of more diverse processing policy combinations in the action space, and enhances the actor model's generalization ability. Finally, the policy loss function value is calculated based on the pruning policy loss value, entropy value, and corresponding weights. The policy loss function value is minimized using a gradient ascent algorithm to update the actor model parameters. This achieves a balance between stable policy performance improvement and sufficient exploration of the action space, thereby improving the accuracy and robustness of the actor model.

[0112] Figure 4 A flowchart for training an actor-critic model is shown, with the following steps: S401: Environment; S402: Storing experience; S403: Calculating the advantage function; S404: Calculating the policy loss; S405: Updating the policy parameters; S406: Iterating the actor model; S407: Calculating the value objective; S408: Calculating the value loss; S409: Updating the value parameters; S410: Iterating the critic model.

[0113] S401: Environment. The actor model, acting as the intelligent agent, interacts with the log processing environment. The environment refers to the overall scenario that includes the business log data to be processed, downstream task models, data quality assessment, etc. The actor model selects a target action from the action space based on the current state space. and add it to the environment; S402: Store experience. After the action is completed, store the state transition experience generated by this interaction into the experience pool. The experience is represented by a state transition quadruple. The data is stored in a format that allows for clear and traceable actions and rewards, while strategy snapshots and decisions are retained through logs for easy retrieval.

[0114] S403: Calculate the advantage function. Based on the state transition quadruple, use the Generalized Advantage Estimation (GAE) to calculate the advantage estimate.

[0115] S404: Calculate the strategy loss. Based on the advantage estimate and the strategy improvement ratio, calculate the pruning strategy loss value.

[0116] S405: Update policy parameters using the gradient ascent algorithm, which updates the actor model parameters by minimizing the policy loss function value.

[0117] S406: Iterate the actor model. After completing the parameter update, the actor model returns to the environment interaction stage, selects new actions based on the updated strategy, and continuously collects experience and optimizes the strategy.

[0118] S407: Calculate the value objective, approximate the value objective using generalized advantage estimation and current state estimation, and calculate the critic's value objective.

[0119] S408: Calculate the value loss. Based on the value target and the current state value prediction, calculate the value loss value.

[0120] S409: Update the value parameters by using gradient descent to update the critic model's parameters by minimizing the value loss function.

[0121] S410: Iterate the critic model. After completing the parameter update, the critic model returns to the intermediate quantity calculation stage. Based on the updated parameters, it outputs a more accurate state value estimate, providing data support for the advantage calculation of the actor model.

[0122] In another example, to avoid overfitting and unnecessary waste of computational resources in the log processing model due to excessive training iterations, the following measures are also included: Obtain validation set data, periodically evaluate the performance of the current log processing model on the validation set, and obtain validation performance metrics.

[0123] In this embodiment of the application, during the training process, the performance of the current log processing model can be periodically evaluated on the validation set to determine whether the current log processing model has been trained.

[0124] In one example, the raw log data can be divided into training set, validation set, and test set data by periodically or conditionally aggregating the raw logs, thus obtaining the validation set data.

[0125] In one example, the raw logs can be aggregated by conditions by hierarchically dividing them based on the business attributes of the logs.

[0126] In one example, the performance metrics for verification may include at least one of the following: log data quality metrics, downstream model task performance metrics, and log processing efficiency metrics.

[0127] Monitor and verify the changing trends of performance indicators.

[0128] In this embodiment, the changing trends over multiple consecutive rounds are monitored to ensure that the model performance steadily improves and tends to converge. This avoids analyzing only a single round's metrics, as fluctuations can lead to inaccurate judgments about the model's training state.

[0129] In one example, a sliding window can be used to calculate the average rate of change of the validation performance metric over multiple consecutive periods.

[0130] If the trend of change does not meet the preset trend of change threshold, the preset training stop condition is determined to be met.

[0131] In this embodiment of the application, the change trend threshold can be the average change rate of the performance index. For example, when the average change rate of the performance index for multiple periods is less than or equal to a preset change threshold, it can be determined that the preset training stop condition is met.

[0132] In another example, it is also possible to determine whether the preset training early stopping conditions are met by monitoring the reward score, such as monitoring the changes in the reward function value over multiple cycles.

[0133] In this embodiment, by acquiring validation set data, the performance of the current log processing model is periodically evaluated on the validation set to obtain validation performance indicators, and the changing trend of the validation performance indicators is monitored. When the changing trend does not meet the preset changing trend threshold, it is determined that the preset training stopping condition is met, and the optimal strategy is directly saved, so as to avoid the log processing model from overfitting and unnecessary waste of computing resources due to too many training times.

[0134] Figure 5 A schematic diagram of a log processing device provided in this application is shown. Figure 5 As shown, the log processing apparatus 500 provided in this application includes: The extraction module 501 extracts features from the logs to be processed to obtain the feature vector of the logs to be processed. The acquisition module 502 is used to acquire historical log processing operations, historical downstream model performance indicators, and historical data quality indicators; among them, the historical downstream model performance indicators are used to characterize the impact of historical log processing operations on the performance of downstream task models, and the historical data quality indicators are used to characterize the impact of historical log processing operations on log quality. Module 503 is used to construct the current state space based on the feature vector of the log to be processed, historical log processing operations, historical downstream model performance indicators, and historical data quality indicators. The first processing module 504 is used to input the current state space into the log processing model, so that the log processing model selects a target action from the action space based on the current state space; wherein, the action space consists of multiple discrete log processing operations and continuous parameters corresponding to each log processing operation, and the continuous parameters are used to continuously numerically control the parameters required in the discrete log processing operations, and each action in the action space consists of a discrete log processing operation and at least one continuous parameter corresponding to the discrete log processing operation. The second processing module 505 is used to process the log to be processed using the target action.

[0135] In one example, the log processing device 500 also includes: The acquisition module is also used to acquire log processing training set data. The log processing training set data includes multiple log processing training samples. Each log processing training sample includes: the current state space, which is composed of the current log feature vector training sample, the historical log processing operation training sample, the historical downstream model performance index training sample, and the historical data quality index training sample. For each log processing training sample, perform the following steps: The input module is used to input the current log feature vector training sample, historical log processing operation training sample, historical downstream model performance index training sample, and historical data quality index training sample into the log processing model to be trained, so as to obtain the log processing operation of the training sample. The calculation module is used to calculate the reward function value based on the log processing operation of the training sample. The reward function value is calculated based on the data quality index, downstream model performance index, downstream model loss function, operation cost and their respective weights corresponding to the log processing operation of the training sample. The judgment module is used to determine whether the reward function value meets the preset training stopping condition; The adjustment module is used to adjust the parameters of the log processing model to be trained when the conditions are not met, and to continue training the log processing model using the log processing training set data until the preset training stopping condition is met, thus obtaining the trained log processing model.

[0136] In one example, the log processing device 500 also includes: The acquisition module is also used to acquire the next state space after the log processing operation of each log processing training sample is executed. The next state space includes: new log feature vector training sample, current log processing operation training sample, current downstream model performance index training sample, and current data quality index training sample. The input module is also used to input the current state space and the next state space into the critic model to obtain the current state estimate and the next state estimate; The calculation module is also used to calculate the advantage estimate based on the current state estimate, the next state estimate, and the reward function value; The calculation module is also used to calculate the value loss function value corresponding to the critic model based on the advantage estimate and the current state estimate; The processing module is also used to update the parameters of the critic model by minimizing the value loss function value based on the gradient descent algorithm.

[0137] In one example, the log processing device 500 also includes: The calculation module is also used to calculate the ratio of the probability of the log processing operation corresponding to the current training sample to the probability of the log processing operation corresponding to the historical training sample, and to obtain the policy improvement ratio. The calculation module is also used to calculate the pruning strategy loss value based on the advantage estimate and the strategy improvement ratio, using the pruning objective function algorithm; The calculation module is also used to calculate the entropy value of the actor model's output log processing operation; The calculation module is also used to calculate the strategy loss function value based on the pruning strategy loss value, entropy value, and the weights corresponding to the entropy value; The processing module is also used to update the parameters of the actor model by minimizing the policy loss function value based on the gradient ascent algorithm.

[0138] In one example, the log processing device 500 also includes: The acquisition module is also used to acquire validation set data, periodically evaluate the performance of the current log processing model on the validation set, and obtain validation performance metrics. The monitoring module is also used to monitor and verify the changing trends of performance indicators; The judgment module is also used to determine whether the preset training stop condition is met when the trend of change does not meet the preset trend of change threshold.

[0139] Figure 6 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0140] An electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0141] Specifically, the processor 601 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0142] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 602 may include removable or non-removable (or fixed) media, or memory 602 may be a non-volatile solid-state memory.

[0143] In one instance, memory 602 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0144] Memory 602 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0145] The processor 601 implements a log processing method in the above-described embodiment by reading and executing computer program instructions stored in the memory 602.

[0146] In one example, the electronic device may also include a communication interface 603 and a bus 604. Wherein, as... Figure 6 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 604 and complete communication with each other.

[0147] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0148] Bus 604 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 604 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0149] Furthermore, in conjunction with one of the log processing methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the log processing methods in the above embodiments.

[0150] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the log processing methods described in the above embodiments.

[0151] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0152] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0153] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0154] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0155] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A log processing method characterized by, The method comprises the following steps: extracting features of a to-be-processed log to obtain a to-be-processed log feature vector; obtaining historical log processing operations, historical downstream model performance indicators, and historical data quality indicators, wherein the historical downstream model performance indicators are used to represent the influence of the historical log processing operations on the performance of a downstream task model, and the historical data quality indicators are used to represent the influence of the historical log processing operations on log quality; constructing a current state space based on the to-be-processed log feature vector, the historical log processing operations, the historical downstream model performance indicators, and the historical data quality indicators; inputting the current state space into a log processing model, so that the log processing model selects a target action from an action space based on the current state space; wherein the action space is composed of a plurality of discrete log processing operations and a plurality of continuous parameters, the continuous parameters are used to continuously numerically control required parameters in the discrete log processing operations, and each action in the action space is composed of a discrete log processing operation and at least one continuous parameter corresponding to the discrete log processing operation; performing log processing on the to-be-processed log by using the target action.

2. The method of claim 1, wherein, Before inputting the current state space into the log processing model, so that the log processing model selects a target action from an action space based on the current state space, the method further comprises the following steps: obtaining log processing training set data, wherein the log processing training set data comprises a plurality of log processing training samples, and each log processing training sample comprises a current state space composed of a current log feature vector training sample, a historical log processing operation training sample, a historical downstream model performance indicator training sample, and a historical data quality indicator training sample; for each log processing training sample, the following steps are performed: inputting the current log feature vector training sample, the historical log processing operation training sample, the historical downstream model performance indicator training sample, and the historical data quality indicator training sample into a to-be-trained log processing model to obtain a log processing operation of the training sample; calculating a reward function value based on the log processing operation of the training sample, wherein the reward function value is calculated based on a data quality indicator, a downstream model performance indicator, a downstream model loss function, and an operation cost corresponding to the log processing operation of the training sample, and respective corresponding weights; determining whether the reward function value meets a preset training stop condition; if the reward function value does not meet the preset training stop condition, adjusting parameters of the to-be-trained log processing model, and continuing to train the log processing model by using the log processing training set data until the preset training stop condition is met, to obtain a trained log processing model.

3. The method of claim 2, wherein, The log processing model is an actor-critic model. The adjusting of the parameters of the to-be-trained log processing model comprises the following steps: For each log processing training sample, a next state space after execution of a log processing operation of the training sample is obtained, and the next state space comprises a new log feature vector training sample, a current log processing operation training sample, a current downstream model performance indicator training sample, and a current data quality indicator training sample; The critic model is input with the current state space and the next state space to obtain a current state estimation value and a next state estimation value; An advantage estimation value is calculated based on the current state estimation value, the next state estimation value, and the reward function value; A value loss function value corresponding to the critic model is calculated based on the advantage estimation value and the current state estimation value; The parameters of the critic model are updated by minimizing the value loss function value based on a gradient descent algorithm.

4. The method of claim 3, wherein, The adjustment of the parameters of the log processing model to be trained comprises: A strategy improvement ratio is obtained by calculating a ratio of a probability of a log processing operation corresponding to the current training sample to a probability of a log processing operation corresponding to a historical training sample; A clipped strategy loss value is calculated based on the advantage estimation value and the strategy improvement ratio by using a clipping objective function algorithm; An entropy value of a log processing operation output by the actor model is calculated; A strategy loss function value is calculated based on the clipped strategy loss value, the entropy value, and a weight corresponding to the entropy value; The parameters of the actor model are updated by minimizing the strategy loss function value based on a gradient ascent algorithm.

5. The method of claim 2, wherein, The adjustment of the parameters of the log processing model to be trained and the continuous training of the log processing model using the log processing training set data are performed until the preset training stop condition is met, and a trained log processing model is obtained, which comprises: Validation set data is obtained, and a performance of a current log processing model is periodically evaluated on the validation set to obtain a validation performance indicator; A change trend of the validation performance indicator is monitored; When the change trend does not meet a preset change trend threshold, it is determined that the preset training stop condition is met.

6. The method of claim 1, wherein, The discrete log processing operation comprises at least one of the following log processing operations: a field completion operation, a synonym replacement operation, a log truncation operation, a log splicing operation, a format standardization operation, a field mapping operation, and a regular filtering operation. The continuous parameter comprises at least one of the following parameters: a noise intensity, a masking rate, a regular threshold sensitivity, a field value jitter rate, a text truncation rate, and a diversity enhancement coefficient. The device comprises:

7. A log processing apparatus characterized by comprising: An extraction module is configured to extract features of a log to be processed to obtain a log feature vector; A obtaining module is configured to obtain historical log processing operations, historical downstream model performance indicators, and historical data quality indicators, wherein the historical downstream model performance indicators are used to represent an influence of the historical log processing operations on a downstream task model performance, and the historical data quality indicators are used to represent an influence of the historical log processing operations on log quality; A construction module is configured to construct a current state space based on the log feature vector to be processed, the historical log processing operations, the historical downstream model performance indicators, and the historical data quality indicators. ​ The first processing module is configured to input the current state space into a log processing model, and cause the log processing model to select a target action from an action space based on the current state space; the action space is composed of a plurality of discrete log processing operations and continuous parameters corresponding to each log processing operation, the continuous parameters are used for continuous numerical control of required parameters in the discrete log processing operations, and each action in the action space is composed of a discrete log processing operation and at least one continuous parameter corresponding to the discrete log processing operation. The second processing module is configured to perform log processing on the log to be processed by using the target action.

8. An electronic device, comprising: The device comprises a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the log processing method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the log processing method according to any one of claims 1-6.

10. A computer program product, characterised in that, The instructions in the computer program product are executed by the processor of the electronic device, so that the electronic device performs the log processing method according to any one of claims 1-6.

Images