Knowledge graph entity identification method, device and equipment and readable storage medium

Abstract

This application discloses a knowledge graph entity recognition method, apparatus, device, and readable storage medium, relating to the field of communication technology. The method includes: extracting at least one field from the log to be predicted based on a log pattern template matching the log to be predicted; fusing spatiotemporal features of the field to generate structured log data; generating a combined embedding vector of textual semantics and structured information based on the structured log data; inputting the fused combined embedding vector into an improved target BERT model for knowledge graph entity prediction, obtaining the entity prediction result output by the target BERT model; performing CRF decoding on the entity prediction result, and filtering the CRF-decoded entity prediction result by confidence level, using entity prediction results with confidence levels higher than a first threshold as the knowledge graph entity recognition result of the log to be predicted, effectively solving the entity fragmentation problem and improving the quality of the prediction results.

Description

Technical Field

[0001] This application belongs to the field of communication technology, specifically relating to a knowledge graph entity recognition method, apparatus, device, and readable storage medium. Background Technology

[0002] In current knowledge graph construction, log data is a key carrier of the business chain, containing knowledge graph entities such as user behavior, API calls, and service capabilities. However, traditional log parsing solutions, such as regular expressions and fixed template parsing, cannot automatically adapt to changes in log format and are prone to ignoring the correlation between temporal contexts in the presence of noise interference, resulting in low accuracy in knowledge graph entity recognition. Alternatively, the conventional Bidirectional Encoder Representations from Transformers (BERT) model's random initialization of weights tends to favor high-frequency knowledge graph entities, resulting in insufficient sensitivity to low-frequency knowledge graph entities and an inability to effectively identify them, which also leads to low accuracy in low-frequency knowledge graph entity recognition. Summary of the Invention

[0003] This application provides a knowledge graph entity recognition method, apparatus, device, and readable storage medium to solve the problem of low accuracy in current knowledge graph entity recognition methods.

[0004] Firstly, a knowledge graph entity recognition method is provided, the method comprising:

[0005] Based on the log pattern template that matches the log to be predicted, extract at least one field from the log to be predicted;

[0006] Structured log data is generated based on a time enhancement factor associated with the timestamp of the log to be predicted, and at least one field in the log to be predicted.

[0007] Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data;

[0008] The combined embedding vector is input into the target BERT model for knowledge graph entity prediction, and the entity prediction result output by the target BERT model is obtained; wherein, the entity prediction result is used to indicate the knowledge graph entity prediction probability of each field;

[0009] The entity prediction results are decoded using a Conditional Random Field (CRF), and the entity prediction results after CRF decoding are filtered by confidence. Entity prediction results with confidence scores higher than a first threshold are used as the knowledge graph entity recognition results of the log to be predicted.

[0010] The step of extracting at least one field from the log to be predicted based on a log pattern template that matches the log to be predicted includes:

[0011] An adaptive parsing function is used to obtain the log pattern template that matches the log to be predicted.

[0012] The adaptive parsing function is used to perform noise-robust parsing on the log to be predicted, and at least one field is extracted from the log to be predicted; the at least one field includes at least one of the following: at least one message text field, at least one level field, and at least one error code field.

[0013] The method further includes:

[0014] The target BERT model is constructed by training the model based on the training log dataset.

[0015] The process of training the model based on the training log dataset to construct the target BERT model includes:

[0016] The training log dataset is parsed and structured to obtain structured log data for each log in the training log dataset;

[0017] Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data;

[0018] The target BERT model is constructed by training the model based on the combined embedding vectors.

[0019] The step of parsing and structuring the training log dataset to obtain structured log data for each log in the training log dataset includes:

[0020] Perform density-based unsupervised clustering on the training log dataset to generate a log pattern template set; the log pattern template set includes at least one log pattern template.

[0021] Based on the log pattern template set, an adaptive parsing function is constructed for each log pattern template; the adaptive parsing function is used to extract at least one field from the log corresponding to the log pattern template.

[0022] The time enhancement factor is determined based on the timestamp of the log corresponding to each log pattern template and the duration of the log stream in which the log is located.

[0023] Based on the time enhancement factor and the adaptive parsing function, structured log data for each log is generated.

[0024] The method further includes:

[0025] The time enhancement factor is determined based on the timestamp of the log to be predicted and the duration of the log stream in which the predicted log is located.

[0026] Specifically, based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data, including:

[0027] Perform BERT encoding on the message text field of the structured log data to generate a text embedding vector;

[0028] Perform categorical embedding and concatenation operations on the categorical and / or numerical fields of the structured log data to generate structured feature embedding vectors;

[0029] Perform a trainable gating operation on the text embedding vector and the structured feature embedding vector to generate gating weights;

[0030] The text embedding vector and the structured feature embedding vector are weighted and fused using the gating weights to generate the combined embedding vector.

[0031] The step of training the model based on the combined embedding vectors to construct the target BERT model includes:

[0032] Perform global mean and low-frequency knowledge graph entity correction operations on the combined embedding vectors to generate the initial weight matrices of each layer of the target BERT model;

[0033] During the multi-head attention calculation process, the time decay factor and entity relevance gating matrix are injected into the attention score calculation to generate the attention output matrix of the target BERT model;

[0034] Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model, and perform an adaptive activation operation on the linear features to generate the output feature matrix of the Ith layer of the target BERT model; I is an integer greater than 1;

[0035] Perform a joint calculation operation of focus loss and distribution smoothing on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model;

[0036] The target BERT model is constructed based on the initial weight matrix of each layer of the target BERT model, the attention output matrix of the target BERT model, the output feature matrix of each layer of the target BERT model, and the total loss.

[0037] The step of performing global mean and low-frequency knowledge graph entity correction operations on the combined embedding vector to generate the initial weight matrix of each layer of the target BERT model includes:

[0038] Calculate the arithmetic mean of the combined embedding vectors of all logs in the training log dataset to obtain the global mean vector of the combined embeddings;

[0039] Based on the number of samples of each knowledge graph entity category in the training log dataset, determine the frequency weight of each knowledge graph entity category;

[0040] The entity distribution correction increment matrix is ​​determined based on the arithmetic mean of the combined embedding vectors of all logs of the same knowledge graph entity category and the frequency weight of the knowledge graph entity category.

[0041] Based on the entity distribution correction increment matrix and the combined embedded global mean vector, the initial weight matrix of each layer of the target BERT model is generated.

[0042] Specifically, in the multi-head attention calculation process, the time decay factor and entity relevance gating matrix are injected into the attention score calculation to generate the attention output matrix of the target BERT model, including:

[0043] The time decay factor is determined based on the absolute time difference between the current log and other logs in the log stream;

[0044] Based on the trainable weight matrix and the combined embedding vector, the entity relevance gating matrix is ​​determined.

[0045] After calculating and scaling the dot product of the query vector and the key vector, the result is multiplied element-wise by the time decay factor and the entity relevance gating matrix at the corresponding positions, and then normalized and weighted summed to generate the attention output matrix of the target BERT model.

[0046] Specifically, a linear transformation operation is performed on the output feature matrix of the (I-1)th layer of the target BERT model, and an adaptive activation operation is performed on the linear features to generate the output feature matrix of the I-th layer of the target BERT model, including:

[0047] Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model to generate an activation input vector that represents the linear features;

[0048] Based on the entity frequency weights of the knowledge graph, a category adaptive coefficient is determined; wherein, the category adaptive coefficient is inversely correlated with the entity frequency weights of the knowledge graph.

[0049] An adaptive activation function is constructed based on the activation input vector and the category adaptive coefficients; the adaptive activation function includes: a Gaussian error linear unit (GELU) and a hyperbolic tangent function modulated by the knowledge graph entity category hint vector; the knowledge graph entity category hint vector is used to provide knowledge graph entity category information;

[0050] The output of the adaptive activation function is used as the output feature matrix of the I layer of the target BERT model.

[0051] Specifically, the focus loss and distribution smoothing joint calculation operation is performed on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model, including:

[0052] The focus loss is determined based on the predicted probability of the knowledge graph entity category by the target BERT model.

[0053] Based on the difference between the predicted distribution and the prior distribution of knowledge graph entity categories obtained from the training log dataset, a distribution smoothing term is determined.

[0054] Based on the focus loss and the distribution smoothing term, the total loss of the target BERT model is generated.

[0055] Secondly, a knowledge graph entity recognition device is provided, including a processor and a transceiver, wherein the transceiver receives and transmits data under the control of the processor, and the processor is used to perform the following operations:

[0056] Based on the log pattern template that matches the log to be predicted, extract at least one field from the log to be predicted;

[0057] Structured log data is generated based on a time enhancement factor associated with the timestamp of the log to be predicted, and at least one field in the log to be predicted.

[0058] Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data;

[0059] The combined embedding vector is input into the target BERT model for knowledge graph entity prediction, and the entity prediction result output by the target BERT model is obtained; wherein, the entity prediction result is used to indicate the knowledge graph entity prediction probability of each field;

[0060] The entity prediction results are decoded using a Conditional Random Field (CRF), and the entity prediction results after CRF decoding are filtered by confidence. Entity prediction results with confidence scores higher than a first threshold are used as the knowledge graph entity recognition results of the log to be predicted.

[0061] The processor is also used to perform the following operations:

[0062] An adaptive parsing function is used to obtain the log pattern template that matches the log to be predicted.

[0063] The adaptive parsing function is used to perform noise-robust parsing on the log to be predicted, and at least one field is extracted from the log to be predicted; the at least one field includes at least one of the following: at least one message text field, at least one level field, and at least one error code field.

[0064] The processor is also used to perform the following operations:

[0065] The target BERT model is constructed by training the model based on the training log dataset.

[0066] The processor is also used to perform the following operations:

[0067] The training log dataset is parsed and structured to obtain structured log data for each log in the training log dataset;

[0068] Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data;

[0069] The target BERT model is constructed by training the model based on the combined embedding vectors.

[0070] The processor is also used to perform the following operations:

[0071] Perform density-based unsupervised clustering on the training log dataset to generate a log pattern template set; the log pattern template set includes at least one log pattern template.

[0072] Based on the log pattern template set, an adaptive parsing function is constructed for each log pattern template; the adaptive parsing function is used to extract at least one field from the log corresponding to the log pattern template.

[0073] The time enhancement factor is determined based on the timestamp of the log corresponding to each log pattern template and the duration of the log stream in which the log is located.

[0074] Based on the time enhancement factor and the adaptive parsing function, structured log data for each log is generated.

[0075] The processor is also used to perform the following operations:

[0076] The time enhancement factor is determined based on the timestamp of the log to be predicted and the duration of the log stream in which the predicted log is located.

[0077] The processor is also used to perform the following operations:

[0078] Perform BERT encoding on the message text field of the structured log data to generate a text embedding vector;

[0079] Perform categorical embedding and concatenation operations on the categorical and / or numerical fields of the structured log data to generate structured feature embedding vectors;

[0080] Perform a trainable gating operation on the text embedding vector and the structured feature embedding vector to generate gating weights;

[0081] The text embedding vector and the structured feature embedding vector are weighted and fused using the gating weights to generate the combined embedding vector.

[0082] The processor is also used to perform the following operations:

[0083] Perform global mean and low-frequency knowledge graph entity correction operations on the combined embedding vectors to generate the initial weight matrices of each layer of the target BERT model;

[0084] During the multi-head attention calculation process, the time decay factor and entity relevance gating matrix are injected into the attention score calculation to generate the attention output matrix of the target BERT model;

[0085] Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model, and perform an adaptive activation operation on the linear features to generate the output feature matrix of the Ith layer of the target BERT model; I is an integer greater than 1;

[0086] Perform a joint calculation operation of focus loss and distribution smoothing on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model;

[0087] The target BERT model is constructed based on the initial weight matrix of each layer of the target BERT model, the attention output matrix of the target BERT model, the output feature matrix of each layer of the target BERT model, and the total loss.

[0088] The processor is also used to perform the following operations:

[0089] Calculate the arithmetic mean of the combined embedding vectors of all logs in the training log dataset to obtain the global mean vector of the combined embeddings;

[0090] Based on the number of samples of each knowledge graph entity category in the training log dataset, determine the frequency weight of each knowledge graph entity category;

[0091] The entity distribution correction increment matrix is ​​determined based on the arithmetic mean of the combined embedding vectors of all logs of the same knowledge graph entity category and the frequency weight of the knowledge graph entity category.

[0092] Based on the entity distribution correction increment matrix and the combined embedded global mean vector, the initial weight matrix of each layer of the target BERT model is generated.

[0093] The processor is also used to perform the following operations:

[0094] The time decay factor is determined based on the absolute time difference between the current log and other logs in the log stream;

[0095] Based on the trainable weight matrix and the combined embedding vector, the entity relevance gating matrix is ​​determined.

[0096] After calculating and scaling the dot product of the query vector and the key vector, the result is multiplied element-wise by the time decay factor and the entity relevance gating matrix at the corresponding positions, and then normalized and weighted summed to generate the attention output matrix of the target BERT model.

[0097] The processor is also used to perform the following operations:

[0098] Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model to generate an activation input vector that represents the linear features;

[0099] Based on the entity frequency weights of the knowledge graph, a category adaptive coefficient is determined; wherein, the category adaptive coefficient is inversely correlated with the entity frequency weights of the knowledge graph.

[0100] An adaptive activation function is constructed based on the activation input vector and the category adaptive coefficients; the adaptive activation function includes: a Gaussian error linear unit (GELU) and a hyperbolic tangent function modulated by the knowledge graph entity category hint vector; the knowledge graph entity category hint vector is used to provide knowledge graph entity category information;

[0101] The output of the adaptive activation function is used as the output feature matrix of the I layer of the target BERT model.

[0102] Specifically, the focus loss and distribution smoothing joint calculation operation is performed on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model, including:

[0103] The focus loss is determined based on the predicted probability of the knowledge graph entity category by the target BERT model.

[0104] Based on the difference between the predicted distribution and the prior distribution of knowledge graph entity categories obtained from the training log dataset, a distribution smoothing term is determined.

[0105] Based on the focus loss and the distribution smoothing term, the total loss of the target BERT model is generated.

[0106] Thirdly, a knowledge graph entity recognition device is provided, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the knowledge graph entity recognition method as described above.

[0107] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the knowledge graph entity recognition method described above.

[0108] Fifthly, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the knowledge graph entity recognition method as described above.

[0109] The above-mentioned technical solution of this application has at least the following beneficial effects:

[0110] In the knowledge graph entity recognition method, apparatus, device, and readable storage medium of this application embodiment, at least one field is extracted from the log to be predicted using a log pattern template matched with the log to be predicted, which can solve the problem of variable log formats; and the fields are fused with spatiotemporal features to generate structured log data; based on the structured log data, a combined embedding vector that integrates the text semantics and structured information of the structured log data is generated; the fused combined embedding vector is input into an improved target BERT model for knowledge graph entity prediction, and the entity prediction result output by the target BERT model is obtained; the entity prediction result is decoded using a Conditional Random Field (CRF), and the entity prediction result after CRF decoding is filtered by confidence, and the entity prediction result with a confidence level higher than a first threshold is taken as the knowledge graph entity recognition result of the log to be predicted, which effectively solves the entity fragmentation problem and improves the quality of the prediction result. Attached Figure Description

[0111] Figure 1 This is an example diagram of a dual-active architecture for the knowledge graph entity recognition method provided in this application embodiment;

[0112] Figure 2 This diagram illustrates the steps of the knowledge graph entity recognition method provided in this application embodiment.

[0113] Figure 3 Example graph showing the comparison of entity recognition accuracy of different log parsing methods;

[0114] Figure 4 Example graph showing the impact of different training measurements on the recall rate of hard-case samples;

[0115] Figure 5 Example graph showing the impact of confidence threshold on prediction results;

[0116] Figure 6 This is a schematic diagram showing the structure of the knowledge graph entity recognition device provided in the embodiments of this application. Detailed Implementation

[0117] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0118] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and are not used to describe a specified order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0119] It is worth noting that the technologies described in this application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA), and other systems. The terms "system" and "network" in this application are often used interchangeably, and the described technologies can be used with the systems and radio technologies mentioned above, as well as with other systems and radio technologies. However, the following description describes New Radio (NR) systems for illustrative purposes, and NR terminology is used in most of the following description. These technologies can also be applied to applications beyond NR systems, such as 6th generation (6G) radio systems. th Generation 6G communication system.

[0120] like Figure 1 The diagram illustrates a dual-active architecture for the knowledge graph entity recognition method provided in this application embodiment. This dual-active architecture includes data center A and data center B; both data center A and data center B contain gateways, applications (application 1, application 2, application 3), and databases (Oracle, Redis, RabbitMQ). Service registration and discovery are achieved through a registry center, and data consistency is ensured through synchronous data replication. The dual-active architecture complicates application system call relationships, requiring the retrieval of relationships between callers and callees (i.e., knowledge graph entities) from logs to ensure the efficiency and reliability of daily operations.

[0121] The features of the above-mentioned active-active architecture include:

[0122] Because it is a dual-active architecture, both data centers A and B have complete business processing capabilities. The application call relationships within each data center are complex, and they also need to interact and synchronize data with components in other data centers to ensure the high availability and reliability of the entire system.

[0123] In a dual-active scenario, when channels and institutions call for corresponding capabilities, they are distributed to different data centers according to rules. There may also be mutual calls between different data centers. Such complex call relationships require accurate information from logs to obtain the relationship between the caller and the callee, which is particularly important in daily operations to facilitate monitoring, troubleshooting, and business optimization.

[0124] Furthermore, to address the effective extraction of knowledge graph entities (including "capabilities," "interfaces," and "callers") in the aforementioned architecture, this application provides a knowledge graph entity recognition method, such as... Figure 2 As shown, the knowledge graph recognition method includes:

[0125] Step 201: Based on the log pattern template that matches the log to be predicted, extract at least one field from the log to be predicted;

[0126] Optionally, the log pattern template is a log pattern template from a pre-trained log pattern template library.

[0127] Optionally, when a new log format that cannot match the existing log pattern template is encountered, an asynchronous template library update mechanism will be triggered to generate a new log pattern template through offline clustering, thereby ensuring that the system continuously adapts to changes in log format.

[0128] Optionally, the log pattern template is used to identify key fields in the logs to be predicted using a log pattern template generated based on clustering.

[0129] For example, when the predictable log "2025-06-25 14:30:22 WARN Caller UserB's request to interface / api / user / info timed out" is received, this embodiment of the application automatically matches the log pattern template "Caller's request to interface timed out", extracts "UserB" as the caller entity and " / api / user / info" as the interface entity, and records the log level "WARN" and the timestamp.

[0130] Step 202: Generate structured log data based on the time enhancement factor associated with the timestamp of the log to be predicted, and at least one field in the log to be predicted;

[0131] Optionally, the extracted fields are combined with an enhancement factor calculated based on timestamps to finally output structured log data that integrates the extracted fields and time context information, thereby improving noise robustness and the continuity of entity recognition.

[0132] This step involves injecting a time enhancement factor, which specifically strengthens or weakens the parsing results at a particular point in time, thereby injecting time-related information and improving the continuity of entity recognition, especially enhancing the entity correlation among log entries that are temporally adjacent. For example, a time enhancement factor is calculated based on the time difference between the current log and neighboring logs; a 5-second time difference corresponds to a decay value of 0.6, strengthening the correlation between logs that are temporally adjacent.

[0133] Step 203: Based on the structured log data, generate a combined embedding vector that integrates the text semantics and structured information of the structured log data;

[0134] Optionally, the structured log data with injected spatiotemporal information is processed to generate a combined embedding vector that integrates the textual semantics and structured information of the structured log data, thereby more effectively supporting entity recognition.

[0135] Step 204: Input the combined embedding vector into the target BERT model for knowledge graph entity prediction, and obtain the entity prediction result output by the target BERT model; wherein, the entity prediction result is used to indicate the knowledge graph entity prediction probability of each field;

[0136] Optionally, the target BERT model employs an enhanced attention mechanism, incorporating a time decay factor and entity relevance gating on top of standard multi-head attention, making the model more focused on key context fragments. The forward propagation process uses an adaptive activation function, automatically enhancing nonlinear transformation capabilities for low-frequency entities such as "caller". The model outputs the entity prediction probability for each field; for example, in this case, the probability of "UserB" being identified as "caller" reaches 0.85.

[0137] Step 205: Perform Conditional Random Field (CRF) decoding on the entity prediction results, and perform confidence filtering on the entity prediction results after CRF decoding. Entity prediction results with confidence scores higher than the first threshold are used as the knowledge graph entity recognition results of the log to be predicted.

[0138] Optionally, the entity prediction results are optimized by conditional random field decoding, and the entity fragmentation problem is solved by using pre-learned label transition constraints. Then, the confidence of the entire log is calculated (for example, the confidence of the log is the average of the highest prediction probabilities of all fields). In this example, a high confidence of 0.90 is obtained. Finally, a preset threshold of 0.7 is applied for filtering, and only high confidence results are output.

[0139] In one implementation, for logs with low confidence, this embodiment of the application marks them and feeds them back into the training process for subsequent optimization of the target BERT model.

[0140] In this embodiment of the application, the target BERT model outputs the entity probability of each field. Conventional BIO sequence decoding methods ignore the context transition constraints between labels and the overall prediction confidence, which can easily lead to fragmented predicted entities and fail to filter out erroneous predictions with low confidence, such as incorrectly splitting "caller" into multiple fragments.

[0141] This embodiment first calculates an overall prediction confidence score for the logs to be predicted, then uses conditional random field decoding to optimize the sequence labeling results and solve the fragmentation problem by utilizing the learned inter-label transition constraints, and finally applies a confidence threshold filter to discard the prediction results of logs with low overall confidence and retain only the entity sequences with high confidence as the final output. The specific steps are as follows:

[0142] 1) Sample confidence calculation

[0143] For each log entry to be predicted, a confidence scalar is calculated. This confidence score is based on the model's predicted entity class probabilities for all tokens in that log entry. Specifically, it is calculated as the average of the highest predicted class probabilities for each token. The higher this value, the higher the model's consistency in predicting the entire log entry, and the higher the confidence score. This is represented as:

[0144]

[0145] In the formula, Indicates the first The confidence scalar of each log entry takes values ​​[0, 1], with values ​​closer to 1 indicating higher prediction consistency;

[0146] This indicates taking the maximum value of the entity category dimension;

[0147] For the model to token category The predicted probability.

[0148] 2) Conditional Random Field Decoding

[0149] The Conditional Random Field (CRF) decoder is applied to process the token-level entity probability matrix output by BERT. CRF utilizes a learnable transition matrix that encodes the rationality and constraints of transitions between different entity category labels, optimizing the entity sequence labeling results and resolving the label fragmentation problem that may occur with simple BIO decoding. This generates a more complete entity sequence that conforms to the contextual logic, represented as:

[0150]

[0151] In the formula, The entity sequence after decoding a CRF (Conditional Random Field);

[0152] This represents the decoding function for a conditional random field.

[0153] This is the token probability matrix output by BERT;

[0154] It is a trainable transition matrix that encodes the transition constraints between entity categories.

[0155] 3) Confidence threshold filtering

[0156] Confidence filtering is applied to the entity sequence prediction results after CRF decoding. The calculated overall confidence scalar of the log is compared with a preset confidence threshold. If the confidence is lower than the threshold, the entire entity prediction sequence of that log is invalidated, thus filtering out results with low model prediction quality and insufficient confidence, avoiding the propagation of erroneous predictions, and retaining only the entity recognition results with high confidence as the final output, as shown below:

[0157]

[0158] In the formula, For the final entity prediction sequence;

[0159] For indicator functions, when Output 1 when At that time, the entire log prediction is set to zero to prevent error propagation;

[0160] For confidence thresholds, such as, =0.7.

[0161] In summary, in this embodiment, by using a log pattern template matching the log to be predicted, at least one field in the log to be predicted can be extracted, which can solve the problem of variable log formats; and the fields are fused with spatiotemporal features to generate structured log data; based on the structured log data, a combined embedding vector integrating the text semantics and structured information of the structured log data is generated; the fused combined embedding vector is input into the improved target BERT model for knowledge graph entity prediction, and the entity prediction result output by the target BERT model is obtained; the entity prediction result is decoded using a Conditional Random Field (CRF), and the entity prediction result after CRF decoding is filtered by confidence, and the entity prediction result with a confidence level higher than a first threshold is taken as the knowledge graph entity recognition result of the log to be predicted, which effectively solves the entity fragmentation problem and improves the quality of the prediction result.

[0162] It should be noted that the knowledge graph entities mentioned in the embodiments of this application can also be simply referred to as entities, and will not be explained again hereafter.

[0163] This application's embodiments target massive and complex log data, accurately identifying key entities within the logs, such as users, interfaces, and callers. Operators can gain deep insights into the operational status of business systems, user behavior patterns, and business process interactions, thereby achieving refined management and optimization of business processes, improving business efficiency and service quality. For example, it can quickly locate the source of system failures, optimize interface call chains, reduce business downtime, and enhance user experience.

[0164] In at least one embodiment of this application, step 201 includes:

[0165] An adaptive parsing function is used to obtain the log pattern template that matches the log to be predicted.

[0166] The adaptive parsing function is used to perform noise-robust parsing on the log to be predicted, and at least one field is extracted from the log to be predicted; the at least one field includes at least one of the following: at least one message text field, at least one level field, and at least one error code field.

[0167] Optionally, the adaptive parsing function dynamically assigns weights to each log pattern template by calculating the similarity between each log and each log pattern template, and performs a weighted combination of the field extraction results of each log pattern template based on these weights, thereby achieving noise-robust parsing of the log to be predicted, automatically attenuating the influence of outlier patterns during pattern matching, and generating noise-robust field extraction results, which include at least one of the above-mentioned fields.

[0168] In at least one embodiment of this application, step 203 includes:

[0169] Perform BERT encoding on the message text field of the structured log data to generate a text embedding vector;

[0170] Perform categorical embedding and concatenation operations on the categorical and / or numerical fields of the structured log data to generate structured feature embedding vectors;

[0171] Perform a trainable gating operation on the text embedding vector and the structured feature embedding vector to generate gating weights;

[0172] The text embedding vector and the structured feature embedding vector are weighted and fused using the gating weights to generate the combined embedding vector.

[0173] In this embodiment, text embedding vectors for message text fields and structured feature embedding vectors for categorical and / or numerical fields are generated respectively. The two types of heterogeneous embedding vectors are weighted and fused using a dynamically calculated gating weight to generate a combined embedding vector that can simultaneously express text semantics and structured information, thereby more effectively supporting entity recognition and improving the accuracy of entity recognition.

[0174] For example, the extracted fields are processed by applying the standard BERT word embedding layer to generate text embedding vectors. At the same time, structured fields such as log level and error code are processed by categorical and numerical embedding respectively. The text quality is automatically evaluated by a dynamic gating mechanism. When the text noise is low, the text embedding is given a higher weight. In this example, the gating weight is calculated to be 0.6. Finally, a combined feature vector that integrates text semantics and structured information is generated.

[0175] Optionally, the method further includes:

[0176] The time enhancement factor is determined based on the timestamp of the log to be predicted and the duration of the log stream in which the predicted log is located.

[0177] For example, a time enhancement factor is calculated based on the timestamp of the log to be predicted and the total time span of the entire log stream. Specifically, it enhances or weakens the strength of the parsing results at a specific time point, thereby injecting time-related information and improving the continuity of entity recognition, especially enhancing the entity correlation among log entries that are temporally adjacent. This is expressed as:

[0178]

[0179] In the formula, For time enhancement factor;

[0180] The timestamp of the log to be predicted;

[0181] This represents the total time span of the log stream;

[0182] For example, the time amplitude coefficient, =0.1, controls the temporal correlation strength.

[0183] In at least one embodiment of this application, the method further includes:

[0184] The target BERT model is constructed by training the model based on the training log dataset.

[0185] It's important to note that log data generated by application systems is semi-structured, often exhibiting high noise levels and varied patterns. Entity boundaries within log entries are frequently unclear, and entity identification is highly context-dependent. For example, log entries like "Caller UserA failed to call DataFetch via interface / api / get" are difficult to identify directly and accurately. Conventional processing techniques typically use regular expressions or fixed templates for parsing, but these methods cannot automatically adapt to changes in log format and, in the presence of noise, easily overlook temporal contextual relationships, resulting in low entity identification accuracy.

[0186] In this embodiment of the application, model training is performed based on the training log dataset to construct the target BERT model, including:

[0187] The training log dataset is parsed and structured to obtain structured log data for each log in the training log dataset;

[0188] Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data;

[0189] The target BERT model is constructed by training the model based on the combined embedding vectors.

[0190] The step of parsing and structuring the training log dataset to obtain structured log data for each log in the training log dataset includes:

[0191] S11, Perform density-based unsupervised clustering on the training log dataset to generate a log pattern template set; the log pattern template set includes at least one log pattern template;

[0192] S12, Based on the log pattern template set, construct an adaptive parsing function for each log pattern template; the adaptive parsing function is used to extract at least one field from the log corresponding to the log pattern template;

[0193] S13, determine the time enhancement factor based on the timestamp of the log corresponding to each log mode template and the duration of the log stream in which the log is located;

[0194] S14. Based on the time enhancement factor and the adaptive parsing function, generate structured log data for each log.

[0195] This application embodiment automatically generates a log pattern template library by dynamically clustering log content, and constructs an adaptive parsing function based on template similarity weighting to extract log fields. Simultaneously, it combines an enhancement factor calculated based on timestamps to ultimately output structured log data that integrates content parsing results and temporal context information, thereby improving noise robustness and the continuity of entity recognition. The specific steps are as follows:

[0196] S11 performs density-based unsupervised clustering on the training log dataset to generate a set of log pattern templates, thus solving the problem of variable log formats.

[0197] In one implementation, unsupervised clustering is performed on the training log dataset. A density-based clustering algorithm groups log entries according to the similarity of their content patterns. This operation automatically generates a set of log pattern templates, which serve as the basis for subsequent parsing, thus adapting to the problem of varying log formats and providing the core pattern basis for normalization. This is represented as follows:

[0198]

[0199] In the formula, This represents the set of generated log pattern templates, which serve as the basis for parsing.

[0200] This represents a density-based clustering algorithm.

[0201] For training log dataset;

[0202] For the first A log pattern template for each cluster;

[0203] The total number of clusters determined automatically;

[0204] This represents the total number of log entries.

[0205] S12, Based on the log pattern template set, construct an adaptive parsing function for each log pattern template; the adaptive parsing function is used to extract at least one field from the log corresponding to the log pattern template; that is, based on the log pattern template library, perform similarity-weighted parsing operation on the log content to generate noise-robust field extraction results and suppress outlier pattern interference.

[0206] In one implementation, a parsing function is constructed based on the log pattern template library generated in the previous step. This function dynamically assigns weights to each template by calculating the similarity between each original log and each pattern template. Then, the field extraction results of each template are weighted and combined according to these weights, thereby achieving robust parsing of the original log for noise and automatically attenuating the influence of outlier patterns during pattern matching. This is expressed as:

[0207]

[0208] In the formula, It is an adaptive analytic function. The field extraction results characterizing noise robustness include at least one field;

[0209] Indicates the first The cluster's log mode context;

[0210] For template-based Field extraction operation;

[0211] Template weights are calculated as follows: ;

[0212] For the first Cluster centers;

[0213] For example, the weight decay coefficient. =0.5, control mode similarity sensitivity, increase this value to strengthen the main mode when noise increases.

[0214] S13. Based on the timestamp of the log corresponding to each log pattern template and the duration of the log stream in which the log is located, determine the time enhancement factor; that is, perform a cosine period enhancement factor calculation operation on the log timestamp to generate a time enhancement factor (also known as a time context enhancement factor) to improve the entity continuity recognition capability.

[0215] In one implementation, a time enhancement factor is calculated based on the log timestamps and the total time span of the entire log stream. Specifically, this factor enhances or weakens the strength of the parsing results at specific time points, thereby injecting time-related information and improving the continuity of entity recognition. In particular, it enhances the entity association between log entries that are temporally adjacent, as expressed below:

[0216]

[0217] In the formula, For time enhancement factor;

[0218] For the first The timestamp of each log entry;

[0219] This represents the total time span of the log stream;

[0220] For example, the time amplitude coefficient, =0.1, controls the temporal correlation strength.

[0221] S14. Based on the time enhancement factor and the adaptive parsing function, generate structured log data for each log; that is, perform a fusion operation between the parsing result (i.e., the field extraction result) and the time enhancement factor to output the normalized log (i.e., structured log data) injected with spatiotemporal information.

[0222] In one implementation, the output of the adaptive parsing function, i.e., the field extraction result, is integrated with the time context enhancement factor. This operation generates the final structured log entries, which serve as input to subsequent models. The output is normalized log data integrating content parsing and time-series information, represented as follows:

[0223]

[0224] In the formula, For the normalized first 1 log entry, used as model input;

[0225] For the training log dataset, the first 1 log entry.

[0226] In one embodiment, the accuracy performance of different log parsing methods in entity recognition tasks is compared. The experiment places four methods—"regular expression", "fixed template", "long short-term memory network parsing" and "this application (also referred to as this technology)"—in the same coordinate system, and uses four sets of bars with different textures to represent their performance on three types of entities—"capabilities", "interfaces", and "callers"—as well as the "macro average" metric. Figure 3 As can be seen, the dark bars representing this technology are significantly superior to other methods across all entity types, especially for low-frequency entities such as "callers". Figure 3 The paper visually demonstrates that the dynamic clustering parsing and temporal context enhancement techniques of this application effectively solve the shortcomings of traditional methods in low-frequency entity recognition and noise adaptation, and verifies the comprehensive advantages of the technical solution in entity recognition accuracy.

[0227] In summary, this application's embodiments automatically generate a log pattern template library using a density-based clustering algorithm and construct a parsing function weighted by template similarity, combined with a timestamp enhancement factor, effectively improving the noise robustness of log data and the continuity of entity recognition. Specifically, by using a density-based clustering algorithm to automatically generate a log pattern template library and constructing a parsing function weighted by template similarity, combined with an enhancement factor calculated using timestamps, this effectively improves the noise robustness of log data and the continuity of entity recognition. For example, when processing log content such as "Caller UserA failed to call the DataFetch capability via interface / api / get", this application can more accurately identify the entities within it, while traditional methods may ignore the relationships between entities due to noise interference.

[0228] In at least one embodiment of this application, during model training, a combined embedding vector integrating the textual semantics and structured information of the structured log data is generated based on the structured log data, including:

[0229] S21, Perform BERT encoding on the message text field of the structured log data to generate a text embedding vector;

[0230] S22, Perform categorical embedding and concatenation operations on the categorical fields and / or numerical fields of the structured log data to generate a structured feature embedding vector;

[0231] S23, Perform a trainable gating operation on the text embedding vector and the structured feature embedding vector to generate gating weights;

[0232] S24, the text embedding vector and the structured feature embedding vector are weighted and fused using the gating weights to generate the combined embedding vector.

[0233] In log data, the identification of key entities such as "interface" often relies on combined features. However, conventional technical processing methods simply use text word embeddings, ignoring the semantic contribution of structured fields, making it difficult to effectively distinguish entities such as "caller" and "capability" in sparse logs.

[0234] This application embodiment generates text embedding vectors for log message text and structured feature embedding vectors for structured fields, respectively. These two heterogeneous embedding vectors are then weighted and fused using a dynamically calculated gating weight to generate a combined embedding vector that simultaneously expresses text semantics and structured information, thereby more effectively supporting entity recognition. The specific steps are as follows:

[0235] S21, perform BERT encoding on the message text field of the structured log data to generate a text embedding vector; that is, perform BERT encoding on the structured log data injected with spatiotemporal information to generate a text embedding vector and capture text semantic features.

[0236] In one implementation, the standard BERT word embedding layer is used to process the log message text field. This operation converts the text information into a high-dimensional semantic vector representation, i.e., a text embedding vector, and then extracts the semantic features from the log message, represented as:

[0237]

[0238] In the formula, Indicates the first The text embedding vector of each log entry;

[0239] This represents the standard BERT word embedding layer;

[0240] For the first The message text field of the log entry.

[0241] S22, perform categorical embedding and concatenation operations on the categorical and / or numerical fields of the structured log data to generate a structured feature embedding vector; that is, perform categorical embedding and concatenation operations on the categorical / numerical fields of the structured log data to generate a structured feature embedding vector while preserving the semantic information of the fields.

[0242] In one implementation, embedding operations are performed on categorical and numeric fields in the logs separately. Categorical fields are transformed using a categorical field embedding function, and numeric fields are transformed using a numeric field embedding function. Then, the embedding vectors of these two types of fields are concatenated to generate a structured feature embedding vector representing the structured information of the logs, expressed as:

[0243]

[0244] In the formula, Indicates the first Structured feature embedding vectors for each log entry;

[0245] Embed functions for categorical fields;

[0246] Embed functions for numeric fields;

[0247] This represents a vector concatenation operation;

[0248] For the first The level field of the log entry;

[0249] For the first Error code field for each log entry.

[0250] S23, perform a trainable gating operation on the text embedding vector and the structured feature embedding vector to generate gating weights; that is, perform a trainable gating operation on the text embedding vector and the structured feature embedding vector to generate adaptive weight scalars and dynamically balance the contribution ratio of the two modes.

[0251] A gated weight scalar is calculated based on the text embedding vector and the structured feature embedding vector. The concatenated two-class embedding vectors are processed through a trainable layer, and then a sigmoid activation function is applied to obtain a weight value between 0 and 1. Based on the relevance of the current log features, the contribution ratio of the text embedding and the structured feature embedding in the final combination is adaptively determined. For example, the weight of the text embedding is automatically reduced when there is high text noise, as expressed as:

[0252]

[0253] In the formula, The gating weight takes values ​​[0, 1]. When the message text is noisy, Automatically reduce to enhance structured features;

[0254] Use the Sigmoid activation function;

[0255] The weight matrix is ​​trainable.

[0256] It is a trainable bias vector;

[0257] This indicates a vector concatenation operation.

[0258] S24, the text embedding vector and the structured feature embedding vector are weighted and fused using the gating weights to generate the combined embedding vector. That is, the text embedding vector and the structured feature embedding vector are weighted and fused using the calculated gating weight scalars to generate a combined embedding vector that integrates text semantics and structured information.

[0259] In one implementation, the text embedding vector and the structured feature embedding vector are weighted and fused using a calculated gating weight scalar. Specifically, the text embedding vector is multiplied by the gating weight, the structured feature embedding vector is multiplied by 1 and subtracted from the gating weight, and then the two results are added together to generate a combined embedding vector that integrates text semantics and structured information, represented as:

[0260]

[0261] In the formula, For the first A combined embedding vector of log entries.

[0262] In summary, this application generates semantic embedding vectors for log message text and feature embedding vectors for structured fields, respectively, and then performs weighted fusion using dynamic gating weights to generate a combined embedding vector that simultaneously expresses textual semantics and structured information, thus improving the accuracy of entity recognition. Specifically, it generates semantic embedding vectors for log message text and feature embedding vectors for structured fields, respectively, and then performs weighted fusion using dynamic gating weights to generate a combined embedding vector that simultaneously expresses textual semantics and structured information. This feature representation method can more comprehensively reflect the semantic information of log data, improving the accuracy of entity recognition. For example, when identifying entities such as "caller" and "capability," this application can more accurately distinguish them, while traditional methods may confuse them due to insufficient feature representation.

[0263] In at least one embodiment of this application, the step of training the model based on the combined embedding vectors to construct the target BERT model includes:

[0264] S31, Perform global mean and low-frequency knowledge graph entity correction operations on the combined embedding vector to generate the initial weight matrix of each layer of the target BERT model;

[0265] S32, In the multi-head attention calculation process, the time decay factor and entity relevance gating matrix are injected into the attention score calculation to generate the attention output matrix of the target BERT model;

[0266] S33, perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model, and perform an adaptive activation operation on the linear features to generate the output feature matrix of the Ith layer of the target BERT model; I is an integer greater than 1;

[0267] S34, Perform a joint calculation operation of focus loss and distribution smoothing on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model;

[0268] S35. Based on the initial weight matrix of each layer of the target BERT model, the attention output matrix of the target BERT model, the output feature matrix of each layer of the target BERT model, and the total loss, construct the target BERT model.

[0269] As an optional embodiment, S31 includes:

[0270] Calculate the arithmetic mean of the combined embedding vectors of all logs in the training log dataset to obtain the global mean vector of the combined embeddings;

[0271] Based on the number of samples of each knowledge graph entity category in the training log dataset, determine the frequency weight of each knowledge graph entity category;

[0272] The entity distribution correction increment matrix is ​​determined based on the arithmetic mean of the combined embedding vectors of all logs of the same knowledge graph entity category and the frequency weight of the knowledge graph entity category.

[0273] Based on the entity distribution correction increment matrix and the combined embedded global mean vector, the initial weight matrix of each layer of the target BERT model is generated.

[0274] S31 is used to perform global mean and low-frequency entity correction operations on the combined embedded vector to generate distribution-aware initial weights and alleviate long-tail bias.

[0275] Because log data exhibits a significant long-tail distribution, such as sparse "caller" entity samples, and the conventional BERT method of randomly initializing weights tends to favor high-frequency entities such as "interfaces," resulting in a severe lack of sensitivity to low-frequency entities such as "capabilities."

[0276] This embodiment first calculates the global mean of the combined embedding vectors of all training samples, and then calculates a correction factor that focuses more on low-frequency entity features by combining the entity frequency-based weights. The two are added together to generate the initial weight matrix of each layer of the BERT model, so that the initial state of the model is more in line with the distribution of the actual data, especially enhancing the attention to low-frequency entities. The specific steps are as follows:

[0277] 1) Combined embedding global mean calculation

[0278] Calculate the arithmetic mean of the combined embedding vectors of all training samples. This operation involves summing the combined embedding vectors of all samples and dividing by the total number of samples. The result is a global mean vector representing the center of the combined embedding distribution of the entire training dataset, expressed as:

[0279]

[0280] In the formula, Represents the global mean vector of the combined embedding;

[0281] This represents the total number of training log samples in the training log dataset.

[0282] 2) Calculation of entity frequency weights

[0283] Calculate the frequency weight scalar for each entity category. This weight is based on the number of samples of the entity category in the training set. Specifically, it is calculated by taking the reciprocal of the number of samples for each category and then normalizing it so that low-frequency entity categories automatically receive higher weight values ​​and high-frequency entity categories receive lower weight values. This is expressed as:

[0284]

[0285] In the formula, Indicates entity category Frequency weight scalar;

[0286] Index the entity categories: 1 = "Capability", 2 = "Interface", 3 = "Caller";

[0287] For category The number of samples in the training set, the denominator The sum of the inverse frequencies of all categories is used, and low-frequency entities automatically receive higher weights.

[0288] 3) Calculation of entity distribution correction increment

[0289] Calculate an entity distribution correction increment matrix. This operation involves grouping by entity category, summing the combined embedding vectors of all samples belonging to the same category, multiplying the sum for each category by its corresponding entity frequency weight scalar, accumulating the weighted sums for all categories, and finally multiplying by an entity correction rate coefficient to obtain a correction amount. This correction amount focuses more on the combined embedding features of low-frequency entity categories, and is expressed as:

[0290]

[0291] In the formula, The incremental matrix is ​​corrected for entity distribution; low-frequency entities are processed through... Gain higher weight;

[0292] For entity correction rate, e.g., =0.05;

[0293] For the first The entity label of a log entry.

[0294] 4) Initialize weight generation

[0295] The combined embedding of the global mean vector and the entity distribution correction increment matrix are added to generate the initial weight matrix for all layers of the BERT model. This integrates global distribution information and corrections for long-tail entity distributions, making the model's initial weights more adapted to the distribution of actual data and enhancing sensitivity to low-frequency entities. This is expressed as:

[0296]

[0297] In the formula, Initialize the weight matrix for all layers of the BERT model;

[0298] Compensate for distribution bias during random initialization;

[0299] Enhanced sensitivity to low-frequency entities for long-tailed distributions.

[0300] As an optional embodiment, S32 includes:

[0301] The time decay factor is determined based on the absolute time difference between the current log and other logs in the log stream;

[0302] Based on the trainable weight matrix and the combined embedding vector, the entity relevance gating matrix is ​​determined.

[0303] After calculating and scaling the dot product of the query vector and the key vector, the result is multiplied element-wise by the time decay factor and the entity relevance gating matrix at the corresponding positions, and then normalized and weighted summed to generate the attention output matrix of the target BERT model.

[0304] S32 is used to inject time decay and entity gating operations into the attention score, generate context-focused attention output, and enhance key sequence segments.

[0305] It should be noted that log entities such as "interface" need to be able to capture long-distance call chain context information, but the standard multi-head attention mechanism is sensitive to local noise and cannot prioritize key sequence segments, such as the starting point of the call chain, resulting in relatively scattered context information.

[0306] In this embodiment of the application, during the standard multi-head attention computation process, a time decay factor that decreases with increasing time interval and a gating value for predicting entity relevance are injected into each position. This enables the attention mechanism to simultaneously perceive the temporal proximity between logs and their relevance to the current entity task, thereby focusing more on key contextual fragments. The specific steps are as follows:

[0307] 1) Calculation of time decay factor.

[0308] A time decay factor scalar is calculated based on the absolute time difference between the current log entry and other entries in the sequence. An exponential function is used to produce a value between 0 and 1, thereby suppressing the influence of log entries with longer time intervals on the attention calculation. This is expressed as:

[0309]

[0310] In the formula, This represents a scalar value for the time decay factor, with a value of (0, 1].

[0311] The time difference (in seconds) between the current log entry and other entries in the sequence is calculated by... calculate;

[0312] For example, the attenuation coefficient, =0.1, controlling the intensity of time decay;

[0313] Represents an exponential function;

[0314] This represents absolute value operations.

[0315] 2) Generation of entity relevance gating matrix.

[0316] A gating matrix for entity relevance is calculated by multiplying a trainable weight matrix with a combined embedding vector to generate a gating value in the range [0, 1]. This gating value then predicts and amplifies the importance of log positions highly relevant to the current entity recognition task in the attention calculation, as shown below:

[0317]

[0318] In the formula, This is the entity relevance gating matrix, with values ​​[0, 1]. This gating amplifies the attention score at entity relevance locations.

[0319] The weight matrix is ​​trainable.

[0320] This is a combined embedding vector.

[0321] 3) Location-aware attention computation.

[0322] In the standard multi-head attention calculation process, the time decay factor and entity relevance gating matrix are injected into the attention score calculation. Specifically, after calculating and scaling the dot product of the query vector and the key vector, the result is multiplied element-wise by the time decay factor and entity relevance gating value at the corresponding positions, followed by Softmax normalization and weighted summation. This operation is performed synchronously on all attention heads, enabling the attention mechanism to simultaneously perceive temporal proximity and entity relevance, focusing more on key and context-related sequence segments, as shown below:

[0323]

[0324] In the formula, For improved attention output matrix;

[0325] For attention head index, ;

[0326] For the total number of attention heads;

[0327] For query matrix;

[0328] The key matrix;

[0329] It is a value matrix;

[0330] The dimension of the key vector;

[0331] It is a normalized exponential function;

[0332] This represents the matrix transpose operation;

[0333] This is the time decay factor function.

[0334] It should be noted that, The attention score characterizes the injection time decay and the entity gating operation. Attention scores characterize the effects of uninjected time decay and entity gating operations.

[0335] As an optional embodiment, S33 includes:

[0336] Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model to generate an activation input vector that represents the linear features;

[0337] Based on the entity frequency weights of the knowledge graph, a category adaptive coefficient is determined; wherein, the category adaptive coefficient is inversely correlated with the entity frequency weights of the knowledge graph.

[0338] An adaptive activation function is constructed based on the activation input vector and the category adaptive coefficients; the adaptive activation function includes: a Gaussian error linear unit (GELU) and a hyperbolic tangent function modulated by the knowledge graph entity category hint vector; the knowledge graph entity category hint vector is used to provide knowledge graph entity category information;

[0339] The output of the adaptive activation function is used as the output feature matrix of the I layer of the target BERT model.

[0340] S33 is used to perform a linear transformation operation on the output features of the previous BERT layer to obtain linear features, and then perform a GELU+entity modulation tanh activation operation on the linear features to generate feature outputs that enhance nonlinear expression and improve low-frequency entity sensitivity.

[0341] It should be noted that the deep semantics of log entities require efficient non-linear transformations, but the standard BERT activation function is prone to saturation under sparse data, which leads to gradient vanishing and ignores the contextual relationships of entities, making it difficult to effectively capture the complex features of low-frequency entities.

[0342] In this embodiment, after the linear transformation of each layer of BERT, an adaptive activation function is applied. This function, based on the standard GELU, adds a hyperbolic tangent transform term that is cued by entity category information and adjusted by frequency adaptive coefficients to enhance nonlinear expressive power, especially improving the feature extraction effect for low-frequency entities. The specific steps are as follows:

[0343] 1) Basic Feature Transformation

[0344] A linear transformation operation is performed on the output features of the previous BERT layer. Specifically, the output features of the previous layer are multiplied by the weight matrix of the current layer and then added to the bias vector of the current layer to generate the activation input vector, which provides the basic input for subsequent nonlinear activation, as shown below:

[0345]

[0346] In the formula, This represents the activation input vector, which characterizes linear features;

[0347] For the first Layer weight matrix;

[0348] For the first The output features of the layer, when l is 1, ;

[0349] For the first The bias vector of the layer.

[0350] 2) Calculation of Category Adaptive Coefficient

[0351] A categorical adaptive coefficient scalar is calculated based on the entity frequency weight scalar. A sigmoid function is used for mapping, resulting in larger adaptive coefficient values ​​for low-frequency entity categories. This allows low-frequency entities to acquire stronger nonlinear transformation capabilities in subsequent activations, expressed as:

[0352]

[0353] In the formula, Indicate category Adaptive coefficient scalar, low-frequency entities through To achieve stronger nonlinear transformation capabilities;

[0354] The entity frequency weights are applied, with lower-frequency entities receiving greater weights.

[0355] To adjust parameters, such as, =1.0, controlling the nonlinear enhancement amplitude.

[0356] 3) Adaptive activation function execution

[0357] A custom adaptive activation function is applied, which combines the standard GELU activation function with a hyperbolic tangent function modulated by an entity category cue vector. The entity category cue vector provides entity category information, while the category adaptive coefficient scalar calculated in the previous step is used to adjust the contribution strength of the hyperbolic tangent function term. This enhances the feature representation of low-frequency entities, thereby overcoming the problems of saturation and gradient vanishing of the standard activation function under sparse data, and incorporating entity context information. This is expressed as:

[0358]

[0359] In the formula, An adaptive activation function;

[0360] It is a standard Gaussian error linear unit;

[0361] It is the hyperbolic tangent function;

[0362] The entity category hint vector is a trainable parameter initialized to the average entity label embedding.

[0363] modulation The contribution intensity of the item, low-frequency entities through and Enhanced features are obtained.

[0364] 4) Generation of layer feature outputs

[0365] The output of the adaptive activation function is used as the output feature matrix of the current BERT layer to generate a feature representation that has undergone enhanced nonlinear transformation and entity context awareness, thereby improving the ability to represent low-frequency entities, as shown below:

[0366]

[0367] In the formula, For the first The output feature matrix of the last BERT layer is defined as follows: ;

[0368] pass Enhance the ability to express the features of low-frequency entities such as "callers".

[0369] As an optional embodiment, S34 includes:

[0370] The focus loss is determined based on the predicted probability of the knowledge graph entity category by the target BERT model.

[0371] Based on the difference between the predicted distribution and the prior distribution of knowledge graph entity categories obtained from the training log dataset, a distribution smoothing term is determined.

[0372] Based on the focus loss and the distribution smoothing term, the total loss of the target BERT model is generated.

[0373] S34 is used to perform joint calculation operations of focus loss and distribution smoothing on the prediction results, generating an imbalance-aware total loss function to enhance hard example learning.

[0374] It should be noted that the number of samples for the three types of entities—capabilities, interfaces, and callers—is significantly unbalanced. "Callers" are scarce, and conventional cross-entropy loss is prone to overfitting on high-frequency entities and is not sensitive to difficult-to-distinguish samples with blurred entity boundaries, resulting in a low recognition rate for low-frequency entities.

[0375] The overall loss function designed in this step consists of two parts: first, the focus loss, which dynamically increases the weight of hard-to-classify samples based on the model's prediction confidence; and second, the distribution smoothing term, which constrains the model prediction to approach the true class distribution of the training data. The two are combined to optimize the model parameters to address the problems of imbalanced samples and hard-case identification. The specific steps are as follows:

[0376] 1) Calculation of focal loss

[0377] A focus loss scalar is calculated for each field in each log entry. Its core is to calculate a difficulty-aware weighting factor based on the model's predicted probability of the true entity category for that field. The lower the predicted probability, the larger the weighting factor, thus assigning higher weights to difficult samples in the loss, thereby reducing the contribution of easily classified samples to the total loss, allowing the model to focus more on learning difficult samples. This is expressed as:

[0378]

[0379] In the formula, The loss scalar is the focal point;

[0380] For the model to the first Log entry number The entity category prediction probability vector of each field is obtained by... The result is obtained by inputting the data into the preset Softmax function;

[0381] The index is the actual label, with values ​​{1, 2, 3};

[0382] This represents the model's predicted probability of the true class.

[0383] For focusing parameters, such as, =2.0, controlling the magnitude of weight increase for difficult examples;

[0384] This is the difficulty weighting factor.

[0385] 2) Construction of distribution smoothing term

[0386] A distribution smoothing scalar is computed, which measures the difference between the model's predicted distribution and the prior distribution of entity classes obtained based on training set statistics. This scalar uses one-hot encoding of the true labels and the prior probabilities of the entity classes to constrain the model's predicted distribution to approach the true data distribution, correcting for prediction bias caused by imbalanced samples. It is expressed as:

[0387]

[0388] In the formula, For distribution smoothing scalars;

[0389] One-hot encoding for the real label, when the category is The value is 1 when the time is right, and 0 otherwise.

[0390] For category The prior probability;

[0391] It is the natural logarithm.

[0392] 3) Total Loss Integration

[0393] The total loss scalar for model training is calculated. This loss is the sum of the focus loss for all fields in each log entry and the scalar of the distribution smoothing term. The distribution smoothing term is multiplied by a smoothing term weight coefficient for weighting. Then, the loss for each log entry across all training samples is summed, integrating the focus loss for learning hard examples and the distribution smoothing term for long-tail distribution correction. This combined optimizes the model parameters to address the imbalance problem in entity recognition, and is expressed as:

[0394]

[0395] In the formula, This is the total loss scalar;

[0396] This represents summing over all samples;

[0397] Indicates the first Sum of all tokens in the log entries;

[0398] For the first The number of tokens in each log entry;

[0399] For example, the weighting coefficients of the smoothing term. =0.2.

[0400] In one embodiment, the impact of different training strategies on the recognition performance of hard example samples is analyzed. For example... Figure 4As shown in the figure, the experiment uses four strategies—"uniform sampling," "static hard example sampling," "focus loss," and "dynamic sampling plus loss optimization"—as rows, and three entity categories—"capabilities," "interfaces," and "callers"—as columns, with color depth representing recall levels. The figure shows a clear color gradient: from light in the upper left corner to dark in the lower right corner, indicating that the entity recognition strategy of this embodiment achieves optimal results across all entity categories. Particularly noteworthy is the example of the "caller" entity, where the dark block containing this application stands in stark contrast to the other strategies, directly verifying the synergistic effect of the dynamic hard example sampling mechanism and the focus loss function, effectively addressing the core pain point of traditional training methods' insufficient learning of low-frequency entities and samples with blurred boundaries.

[0401] In summary, the embodiments of this application significantly improve the model's ability to recognize low-frequency entities by adaptively initializing weights, improving the attention mechanism, adaptive activation function, and optimizing the loss function, particularly for low-frequency entities and long-tailed distribution problems. Specifically, the initial weight matrix of the BERT model is generated by calculating the combined embedding global mean and the weight adjustment based on entity frequency, making the initial state of the model more closely match the actual data distribution and enhancing the sensitivity to low-frequency entities. In addition, a total loss function including focus loss and distribution smoothing term is designed to optimize model parameters to address the problems of sample imbalance and hard example recognition. For example, when dealing with low-frequency entities such as "caller," the embodiments of this application can significantly improve their recognition accuracy, while traditional methods may ignore low-frequency entities due to sample imbalance.

[0402] In one embodiment, the effect of the confidence threshold on the quality control of prediction results is analyzed. The changes in precision and recall are analyzed by adjusting the threshold (0.5-0.95). A dual-axis line graph is shown below. Figure 5 As shown, when the threshold increases, the precision curve continues to rise (up to 15%), while the recall curve gradually declines. The two curves intersect at a threshold of 0.7, forming the optimal balance point. Experimental results verify the effectiveness of the confidence filtering mechanism. When the threshold is too low, the recall is high but contains a large number of false predictions; when the threshold is too high, although the precision is improved, valid entities are missed. This application achieves the optimal balance between precision and recall at the intersection point, where approximately 85% of low-confidence predictions are filtered out, with only a small number of valid identifications lost. Figure 5 The optimal balance point marked by the dashed line (threshold 0.7) perfectly matches the theoretical value of the scheme design, proving that the synergistic effect of temporal context enhancement and conditional random field decoding can stably output high-confidence entity sequences and significantly reduce fragmentation errors of entities such as "caller".

[0403] like Figure 6As shown in the illustration, this application also provides a knowledge graph entity recognition device, including a processor 600 and a transceiver 610. The transceiver 610 receives and transmits data under the control of the processor 600, and the processor 600 is used to perform the following operations:

[0404] Based on the log pattern template that matches the log to be predicted, extract at least one field from the log to be predicted;

[0405] Structured log data is generated based on a time enhancement factor associated with the timestamp of the log to be predicted, and at least one field in the log to be predicted.

[0406] Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data;

[0407] The combined embedding vector is input into the target BERT model for knowledge graph entity prediction, and the entity prediction result output by the target BERT model is obtained; wherein, the entity prediction result is used to indicate the knowledge graph entity prediction probability of each field;

[0408] The entity prediction results are decoded using a Conditional Random Field (CRF), and the entity prediction results after CRF decoding are filtered by confidence. Entity prediction results with confidence scores higher than a first threshold are used as the knowledge graph entity recognition results of the log to be predicted.

[0409] In some embodiments of this application, the processor is also configured to perform the following operations:

[0410] An adaptive parsing function is used to obtain the log pattern template that matches the log to be predicted.

[0411] The adaptive parsing function is used to perform noise-robust parsing on the log to be predicted, and at least one field is extracted from the log to be predicted; the at least one field includes at least one of the following: at least one message text field, at least one level field, and at least one error code field.

[0412] In some embodiments of this application, the processor is also configured to perform the following operations:

[0413] The target BERT model is constructed by training the model based on the training log dataset.

[0414] In some embodiments of this application, the processor is also configured to perform the following operations:

[0415] The training log dataset is parsed and structured to obtain structured log data for each log in the training log dataset;

[0416] Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data;

[0417] The target BERT model is constructed by training the model based on the combined embedding vectors.

[0418] In some embodiments of this application, the processor is also configured to perform the following operations:

[0419] Perform density-based unsupervised clustering on the training log dataset to generate a log pattern template set; the log pattern template set includes at least one log pattern template.

[0420] Based on the log pattern template set, an adaptive parsing function is constructed for each log pattern template; the adaptive parsing function is used to extract at least one field from the log corresponding to the log pattern template.

[0421] The time enhancement factor is determined based on the timestamp of the log corresponding to each log pattern template and the duration of the log stream in which the log is located.

[0422] Based on the time enhancement factor and the adaptive parsing function, structured log data for each log is generated.

[0423] In some embodiments of this application, the processor is also configured to perform the following operations:

[0424] The time enhancement factor is determined based on the timestamp of the log to be predicted and the duration of the log stream in which the predicted log is located.

[0425] In some embodiments of this application, the processor is also configured to perform the following operations:

[0426] Perform BERT encoding on the message text field of the structured log data to generate a text embedding vector;

[0427] Perform categorical embedding and concatenation operations on the categorical and / or numerical fields of the structured log data to generate structured feature embedding vectors;

[0428] Perform a trainable gating operation on the text embedding vector and the structured feature embedding vector to generate gating weights;

[0429] The text embedding vector and the structured feature embedding vector are weighted and fused using the gating weights to generate the combined embedding vector.

[0430] In some embodiments of this application, the processor is also configured to perform the following operations:

[0431] Perform global mean and low-frequency knowledge graph entity correction operations on the combined embedding vectors to generate the initial weight matrices of each layer of the target BERT model;

[0432] During the multi-head attention calculation process, the time decay factor and entity relevance gating matrix are injected into the attention score calculation to generate the attention output matrix of the target BERT model;

[0433] Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model, and perform an adaptive activation operation on the linear features to generate the output feature matrix of the Ith layer of the target BERT model; I is an integer greater than 1;

[0434] Perform a joint calculation operation of focus loss and distribution smoothing on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model;

[0435] The target BERT model is constructed based on the initial weight matrix of each layer of the target BERT model, the attention output matrix of the target BERT model, the output feature matrix of each layer of the target BERT model, and the total loss.

[0436] In some embodiments of this application, the processor is also configured to perform the following operations:

[0437] Calculate the arithmetic mean of the combined embedding vectors of all logs in the training log dataset to obtain the global mean vector of the combined embeddings;

[0438] Based on the number of samples of each knowledge graph entity category in the training log dataset, determine the frequency weight of each knowledge graph entity category;

[0439] The entity distribution correction increment matrix is ​​determined based on the arithmetic mean of the combined embedding vectors of all logs of the same knowledge graph entity category and the frequency weight of the knowledge graph entity category.

[0440] Based on the entity distribution correction increment matrix and the combined embedded global mean vector, the initial weight matrix of each layer of the target BERT model is generated.

[0441] In some embodiments of this application, the processor is also configured to perform the following operations:

[0442] The time decay factor is determined based on the absolute time difference between the current log and other logs in the log stream;

[0443] Based on the trainable weight matrix and the combined embedding vector, the entity relevance gating matrix is ​​determined.

[0444] After calculating and scaling the dot product of the query vector and the key vector, the result is multiplied element-wise by the time decay factor and the entity relevance gating matrix at the corresponding positions, and then normalized and weighted summed to generate the attention output matrix of the target BERT model.

[0445] In some embodiments of this application, the processor is also configured to perform the following operations:

[0446] Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model to generate an activation input vector that represents the linear features;

[0447] Based on the entity frequency weights of the knowledge graph, a category adaptive coefficient is determined; wherein, the category adaptive coefficient is inversely correlated with the entity frequency weights of the knowledge graph.

[0448] An adaptive activation function is constructed based on the activation input vector and the category adaptive coefficients; the adaptive activation function includes: a Gaussian error linear unit (GELU) and a hyperbolic tangent function modulated by the knowledge graph entity category hint vector; the knowledge graph entity category hint vector is used to provide knowledge graph entity category information;

[0449] The output of the adaptive activation function is used as the output feature matrix of the I layer of the target BERT model.

[0450] In some embodiments of this application, a joint calculation operation of focal loss and distribution smoothing is performed on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model, including:

[0451] The focus loss is determined based on the predicted probability of the knowledge graph entity category by the target BERT model.

[0452] Based on the difference between the predicted distribution and the prior distribution of knowledge graph entity categories obtained from the training log dataset, a distribution smoothing term is determined.

[0453] Based on the focus loss and the distribution smoothing term, the total loss of the target BERT model is generated.

[0454] In the data preprocessing stage, this application employs dynamic clustering of log content to automatically generate a pattern template library and constructs an analytical function based on template similarity weighting, combined with an enhancement factor calculated using timestamps, effectively improving noise robustness and entity recognition continuity. At the feature engineering layer, semantic embedding vectors of log message text and feature embedding vectors of structured fields are generated, and weighted fusion is performed using dynamically calculated gating weights to generate combined embedding vectors, providing richer feature representations for entity recognition. Regarding model construction and training, this application makes several improvements to the BERT model. The improvements include adaptive initialization of weights, which generates an initial weight matrix by calculating and combining the global mean of the embedding with the weight adjustment based on entity frequency, making the initial state of the model more closely match the actual data distribution and enhancing sensitivity to low-frequency entities; improved BERT attention mechanism by injecting a time decay factor and entity relevance gating value, making attention more focused on key context fragments; adoption of an adaptive activation function to enhance feature extraction of low-frequency entities; design of a total loss function including focus loss and distribution smoothing terms to optimize model parameters to address sample imbalance and hard example identification problems; and training optimization based on dynamic hard example sampling to ensure that the model continuously focuses on hard-to-identify samples. These improvements work together to form an efficient target BERT model for entity identification of log data. In the prediction and feedback phase, this embodiment first calculates the overall prediction confidence score of the log, then uses a conditional random field decoder to optimize the sequence labeling results, and finally applies a confidence threshold filter to retain high-confidence entity sequences as output. For example, the synergistic effect of the dynamic hard example sampling mechanism and the focus loss function effectively solves the problem of insufficient learning of low-frequency entities and blurred boundary samples by traditional training methods; the synergistic effect of temporal context enhancement and conditional random field decoding can stably output high-confidence entity sequences and significantly reduce entity fragmentation errors. These components are interdependent and jointly improve the accuracy and robustness of entity recognition.

[0455] It should be noted that the knowledge graph entity recognition device provided in this application embodiment is a device capable of executing the above-described knowledge graph entity recognition method. Therefore, all embodiments of the above-described knowledge graph entity recognition method are applicable to this device and can achieve the same or similar beneficial effects, which will not be repeated here.

[0456] This application also provides a knowledge graph entity recognition device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the various processes in the knowledge graph entity recognition method embodiments described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0457] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the various processes described above in the knowledge graph entity recognition method embodiments, achieving the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0458] This application also provides a computer program product, including computer instructions. When executed by a processor, the computer instructions implement the various processes of the knowledge graph entity recognition method embodiment described above and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0459] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0460] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 A device for one or more processes and / or the functions specified in one or more boxes.

[0461] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce a paper article including an instruction means, the instruction means being implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0462] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment, causing the computer or other programmable equipment to perform a series of operational steps to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0463] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A knowledge graph entity recognition method, characterized in that, The method includes: Based on the log pattern template that matches the log to be predicted, extract at least one field from the log to be predicted; Structured log data is generated based on a time enhancement factor associated with the timestamp of the log to be predicted, and at least one field in the log to be predicted. Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data; The combined embedding vector is input into the target BERT model for knowledge graph entity prediction, and the entity prediction result output by the target BERT model is obtained; wherein, the entity prediction result is used to indicate the knowledge graph entity prediction probability of each field; The entity prediction results are decoded using a Conditional Random Field (CRF), and the entity prediction results after CRF decoding are filtered by confidence. Entity prediction results with confidence scores higher than a first threshold are used as the knowledge graph entity recognition results of the log to be predicted.

2. The method according to claim 1, characterized in that, The step of extracting at least one field from the log to be predicted based on a log pattern template that matches the log to be predicted includes: An adaptive parsing function is used to obtain the log pattern template that matches the log to be predicted. The adaptive parsing function is used to perform noise-robust parsing on the log to be predicted, and at least one field is extracted from the log to be predicted; the at least one field includes at least one of the following: at least one message text field, at least one level field, and at least one error code field.

3. The method according to claim 1, characterized in that, The method further includes: The target BERT model is constructed by training the model based on the training log dataset.

4. The method according to claim 3, characterized in that, The target BERT model is constructed by training the model based on the training log dataset, including: The training log dataset is parsed and structured to obtain structured log data for each log in the training log dataset; Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data; The target BERT model is constructed by training the model based on the combined embedding vectors.

5. The method according to claim 4, characterized in that, The step of parsing and structuring the training log dataset to obtain structured log data for each log in the training log dataset includes: Perform density-based unsupervised clustering on the training log dataset to generate a log pattern template set; the log pattern template set includes at least one log pattern template. Based on the log pattern template set, an adaptive parsing function is constructed for each log pattern template; the adaptive parsing function is used to extract at least one field from the log corresponding to the log pattern template. The time enhancement factor is determined based on the timestamp of the log corresponding to each log pattern template and the duration of the log stream in which the log is located. Based on the time enhancement factor and the adaptive parsing function, structured log data for each log is generated.

6. The method according to claim 1 or 2, characterized in that, The method further includes: The time enhancement factor is determined based on the timestamp of the log to be predicted and the duration of the log stream in which the predicted log is located.

7. The method according to claim 1 or 4, characterized in that, Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data, including: Perform BERT encoding on the message text field of the structured log data to generate a text embedding vector; Perform categorical embedding and concatenation operations on the categorical and / or numerical fields of the structured log data to generate structured feature embedding vectors; Perform a trainable gating operation on the text embedding vector and the structured feature embedding vector to generate gating weights; The text embedding vector and the structured feature embedding vector are weighted and fused using the gating weights to generate the combined embedding vector.

8. The method according to claim 4, characterized in that, The step of training the model based on the combined embedding vectors to construct the target BERT model includes: Perform global mean and low-frequency knowledge graph entity correction operations on the combined embedding vectors to generate the initial weight matrices of each layer of the target BERT model; During the multi-head attention calculation process, the time decay factor and entity relevance gating matrix are injected into the attention score calculation to generate the attention output matrix of the target BERT model; Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model, and perform an adaptive activation operation on the linear features to generate the output feature matrix of the Ith layer of the target BERT model; I is an integer greater than 1; Perform a joint calculation operation of focus loss and distribution smoothing on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model; The target BERT model is constructed based on the initial weight matrix of each layer of the target BERT model, the attention output matrix of the target BERT model, the output feature matrix of each layer of the target BERT model, and the total loss.

9. The method according to claim 8, characterized in that, The step of performing global mean and low-frequency knowledge graph entity correction operations on the combined embedding vectors to generate the initial weight matrices for each layer of the target BERT model includes: Calculate the arithmetic mean of the combined embedding vectors of all logs in the training log dataset to obtain the global mean vector of the combined embeddings; Based on the number of samples of each knowledge graph entity category in the training log dataset, determine the frequency weight of each knowledge graph entity category; The entity distribution correction increment matrix is ​​determined based on the arithmetic mean of the combined embedding vectors of all logs of the same knowledge graph entity category and the frequency weight of the knowledge graph entity category. Based on the entity distribution correction increment matrix and the combined embedded global mean vector, the initial weight matrix of each layer of the target BERT model is generated.

10. The method according to claim 8, characterized in that, In the multi-head attention calculation process, the time decay factor and entity relevance gating matrix are injected into the attention score calculation to generate the attention output matrix of the target BERT model, including: The time decay factor is determined based on the absolute time difference between the current log and other logs in the log stream; Based on the trainable weight matrix and the combined embedding vector, the entity relevance gating matrix is ​​determined. After calculating and scaling the dot product of the query vector and the key vector, the result is multiplied element-wise by the time decay factor and the entity relevance gating matrix at the corresponding positions, and then normalized and weighted summed to generate the attention output matrix of the target BERT model.

11. The method according to claim 8, characterized in that, A linear transformation operation is performed on the output feature matrix of the (I-1)th layer of the target BERT model, and an adaptive activation operation is performed on the linear features to generate the output feature matrix of the Ith layer of the target BERT model, including: Perform a linear transformation operation on the output feature matrix of the (I-1)th layer of the target BERT model to generate an activation input vector that represents the linear features; Based on the entity frequency weights of the knowledge graph, a category adaptive coefficient is determined; wherein, the category adaptive coefficient is inversely correlated with the entity frequency weights of the knowledge graph. An adaptive activation function is constructed based on the activation input vector and the category adaptive coefficients; the adaptive activation function includes: a Gaussian error linear unit (GELU) and a hyperbolic tangent function modulated by the knowledge graph entity category hint vector; the knowledge graph entity category hint vector is used to provide knowledge graph entity category information; The output of the adaptive activation function is used as the output feature matrix of the I layer of the target BERT model.

12. The method according to claim 8, characterized in that, Perform a joint calculation operation of focal loss and distribution smoothing on the output feature matrix of the last layer of the target BERT model to generate the total loss of the target BERT model, including: The focus loss is determined based on the predicted probability of the knowledge graph entity category by the target BERT model. Based on the difference between the predicted distribution and the prior distribution of knowledge graph entity categories obtained from the training log dataset, a distribution smoothing term is determined. Based on the focus loss and the distribution smoothing term, the total loss of the target BERT model is generated.

13. A knowledge graph entity recognition device, comprising a processor and a transceiver, wherein the transceiver receives and transmits data under the control of the processor, characterized in that, The processor is used to perform the following operations: Based on the log pattern template that matches the log to be predicted, extract at least one field from the log to be predicted; Structured log data is generated based on a time enhancement factor associated with the timestamp of the log to be predicted, and at least one field in the log to be predicted. Based on the structured log data, a combined embedding vector is generated that integrates the textual semantics and structured information of the structured log data; The combined embedding vector is input into the target BERT model for knowledge graph entity prediction, and the entity prediction result output by the target BERT model is obtained; wherein, the entity prediction result is used to indicate the knowledge graph entity prediction probability of each field; The entity prediction results are decoded using a Conditional Random Field (CRF), and the entity prediction results after CRF decoding are filtered by confidence. Entity prediction results with confidence scores higher than a first threshold are used as the knowledge graph entity recognition results of the log to be predicted.

14. A knowledge graph entity recognition device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that, When the processor executes the program, it implements the knowledge graph entity recognition method as described in any one of claims 1-12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the knowledge graph entity recognition method as described in any one of claims 1-12.

16. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the knowledge graph entity recognition method as described in any one of claims 1-12.

Images