An abnormal data processing method, device, equipment and medium

Abstract

The present application relates to the technical field of data processing, and in particular to an abnormal data processing method and device, equipment and medium. It can be applied to the bank supervision scene in the financial field and the diagnosis and treatment system monitoring scene in the medical field, combined with multi-dimensional information when an abnormal event occurs, to realize hierarchical progressive abnormal analysis, gradually narrow the abnormal analysis range, and improve the accuracy of the abnormal analysis result. By calculating the comprehensive adaptation degree through the two dimensions of disposal effect score and usage frequency, the disposal strategy that is both excellent in practical verification effect and meets the current abnormal scene matching demand can be screened out, avoiding the one-sidedness caused by relying on a single index to screen the strategy, effectively improving the success rate of abnormal disposal, and reducing the computing power and time cost consumed by repeated disposal. The whole process of abnormal data from analysis to disposal is automatically processed without manual intervention to sort out abnormal information and match disposal schemes, which can shorten the response cycle of abnormal processing, thereby improving the data processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an abnormal data processing method, apparatus, device, and medium. Background Technology

[0002] In financial and healthcare scenarios, business systems generate massive amounts of operational data, which often contains various hidden system anomalies. Timely and accurate identification and handling of these anomalies are crucial for ensuring stable system operation. Current anomaly handling methods primarily rely on manual analysis of business experience, configuration of fixed verification rules, and the use of basic statistical verification models for anomaly screening and rectification. This approach is highly dependent on manual experience, has a long rule configuration cycle, and cannot promptly detect hidden or emerging anomalies, leading to delayed anomaly handling. Therefore, improving processing efficiency is a pressing issue that needs to be addressed in anomaly data processing. Summary of the Invention

[0003] This invention provides an abnormal data processing method, apparatus, device, and medium to solve the problem of low processing efficiency in the abnormal data processing process.

[0004] Firstly, an abnormal data processing method is provided, the abnormal data processing method comprising: Obtain anomaly event records when target data is abnormal, the anomaly event records include the anomaly type, anomaly severity, anomaly fields and associated context corresponding to the anomaly event; Based on the anomaly type, the anomaly degree, the anomaly field and the associated context, anomaly classification and anomaly root cause inference are performed to obtain anomaly classification results and anomaly root cause results. Based on the anomaly type, the anomaly degree, the anomaly field, the associated context, the anomaly classification result, and the anomaly root cause result, anomaly pattern recognition is performed on the anomaly event to obtain the target anomaly pattern; Based on the anomaly type, the anomaly root cause result, and the target anomaly pattern, determine a set of candidate handling strategies that match the anomaly event; Obtain the treatment effect score and usage frequency of each candidate treatment strategy. Calculate the comprehensive suitability score of each candidate treatment strategy based on the treatment effect score and usage frequency. Determine the candidate treatment strategy with the highest comprehensive suitability score as the optimal treatment strategy and use the optimal treatment strategy to handle the abnormal event.

[0005] Secondly, an anomaly data processing device is provided, the anomaly data processing device comprising: The acquisition module is used to acquire records of abnormal events when the target data is abnormal. The abnormal event records include the abnormal type, abnormality, abnormal fields and associated context of the abnormal event. The module is used to perform anomaly classification and anomaly root cause reasoning based on the anomaly type, the anomaly degree, the anomaly field and the associated context, and to obtain anomaly classification results and anomaly root cause results. The identification module is used to identify the abnormal event based on the abnormality type, the abnormality degree, the abnormal field, the associated context, the abnormality classification result and the abnormal root cause result, so as to obtain the target abnormality pattern. The determination module is used to determine a set of candidate handling strategies that match the abnormal event based on the abnormality type, the abnormality root cause result and the target abnormality pattern; The calculation module is used to obtain the treatment effect score and usage frequency of each candidate treatment strategy, calculate the comprehensive suitability score of each candidate treatment strategy based on the treatment effect score and usage frequency, determine the candidate treatment strategy corresponding to the largest comprehensive suitability score as the optimal treatment strategy, and use the optimal treatment strategy to process the abnormal event.

[0006] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described abnormal data processing method.

[0007] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described abnormal data processing method.

[0008] The advantages of this application compared to the prior art are: This application employs a layered, progressive anomaly analysis by combining multi-dimensional information from the occurrence of abnormal events. From classification and localization to root cause analysis and pattern recognition, the scope of anomaly analysis is gradually narrowed, avoiding judgment biases caused by single-dimensional analysis and improving the accuracy of the results. By calculating the comprehensive suitability through two dimensions—treatment effectiveness score and usage frequency—it can select treatment strategies that have been proven effective in practice and meet the matching needs of the current anomaly scenario. This avoids the one-sidedness caused by relying solely on a single indicator for strategy selection, effectively improving the success rate of anomaly handling and reducing the computational and time costs of repeated handling. The entire process of anomaly data processing, from analysis to handling, is automated, eliminating the need for manual intervention in sorting anomaly information and matching treatment plans. This shortens the response cycle for anomaly handling, thereby improving data processing efficiency. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of an application environment for an abnormal data processing method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an abnormal data processing method provided in an embodiment of the present invention; Figure 3 yes Figure 2 A flowchart illustrating a specific implementation of step S22; Figure 4 yes Figure 2 A flowchart illustrating a specific implementation of step S23; Figure 5 yes Figure 4 A schematic diagram of a specific implementation method for step S42; Figure 6 yes Figure 2 A flowchart illustrating a specific implementation of step S24; Figure 7 yes Figure 2 A schematic diagram of a specific implementation method for step S25; Figure 8 This is a schematic diagram of an abnormal data processing device according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0013] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0014] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0015] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0016] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0017] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0018] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0019] An embodiment of the present invention provides an abnormal data processing method that can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can be a cluster of one or more servers used to acquire records of abnormal events when target data is abnormal. These records include the abnormal event type, severity, abnormal fields, and associated context. Based on the abnormal event type, severity, fields, and context, the server performs abnormal classification and root cause inference to obtain classification and root cause results. A multi-dimensional feature vector is constructed based on these parameters. The abnormal event is then subjected to abnormal pattern recognition using the feature vector to obtain the target abnormal pattern. Based on the abnormal event type, root cause results, and target abnormal pattern, a set of candidate handling strategies matching the abnormal event is determined. The server obtains the handling effect score and usage frequency of each candidate strategy. Based on these scores, a comprehensive suitability score is calculated for each candidate strategy. The candidate strategy with the highest comprehensive suitability score is determined as the optimal handling strategy, and the optimal strategy is used to process the abnormal event. Clients can be, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0020] Please see Figure 2 , Figure 2 A flowchart illustrating an abnormal data processing method provided in an embodiment of the present invention includes the following steps.

[0021] S21: Obtain the exception event record when the target data is abnormal. The exception event record includes the exception type, exception degree, exception field and associated context corresponding to the exception event.

[0022] In step S21, the target data is the monitored data, and the abnormal event record is the information recorded when the target data is abnormal. The abnormal event record includes the abnormal type, abnormality level, abnormal field, and associated context corresponding to the abnormal event. Among them, the abnormal type represents the specific form of the data deviating from the normal state, the abnormality level quantifies the severity of the data deviating from the normal state, the abnormal field is the name of the specific data field where the abnormality occurred, and the associated context represents the surrounding business status and system operating environment when the abnormal event occurred.

[0023] In this embodiment, abnormal event records are acquired when target data is abnormal. The target data can be data from the financial and medical fields. For example, in a banking supervision scenario in the financial field, the target data can be EAST (Examination and Analysis System Technology) regulatory data; acquiring abnormal event records when the target data is abnormal means acquiring abnormal event records when EAST regulatory data is abnormal. In a medical diagnosis and treatment system monitoring scenario, the target data can be patient electronic medical record data, clinical laboratory test data, etc., generated during the diagnosis and treatment process; acquiring abnormal event records when the target data is abnormal means acquiring abnormal event records of patient electronic medical record data generated during the diagnosis and treatment process.

[0024] The abnormal event log includes the abnormal event type, abnormality level, abnormal fields, and associated context. The abnormality type can be data missing, out-of-bounds values, or format errors; the abnormality level is divided into minor, moderate, and severe levels based on preset thresholds; the abnormal fields are the names of the specific data fields where the abnormality occurred, such as device temperature, heart rate values, or image transmission rates, thus accurately pinpointing the source of the fault; the associated context mainly covers the operational status of surrounding related services at the time of the abnormal event, the basic operating environment parameters of the system before and after the abnormality, and related records of similar historical abnormalities.

[0025] It should be noted that when obtaining abnormal event records when the target data is abnormal, the target data needs to be monitored in real time. If an abnormal alarm is triggered, the target data is determined to be abnormal, and the corresponding abnormal event is recorded. That is, the abnormal type, abnormality, abnormal fields and associated context of the abnormal event are recorded to obtain the abnormal event record.

[0026] When monitoring target data in real time, anomaly detection algorithms based on streaming computing can be used. For example, a sliding window mechanism can be used to calculate the statistical characteristics of the target data stream in real time, such as mean, variance, and distribution pattern, and dynamically compare them with historical benchmark models. Once a deviation of the statistical characteristics from a preset confidence interval is detected, it is determined that there is an anomaly in the current target data, and an alarm mechanism is triggered to record the anomaly type, anomaly severity, anomaly fields, and associated context of the anomaly event.

[0027] It should be noted that the abnormal event log may also include an abnormal event record identifier. This identifier is used to uniquely distinguish each triggered abnormal alarm, enabling precise location during subsequent data processing and tracing. The abnormal event record identifier is typically generated using a globally unique identifier, such as an ID generated by the UUID algorithm, ensuring the uniqueness and orderliness of the identifier in high-concurrency scenarios. The abnormal event log may also include the monitoring time point, which precisely records the exact moment the abnormality was triggered, providing a precise time anchor for subsequent backtracking of related contexts. The abnormal event log may also include the data source system, i.e., clearly identifying the specific business system or device terminal from which the abnormal data originated, such as the HIS system, LIS system, or specific medical imaging equipment.

[0028] In this embodiment, the anomaly type, anomaly degree, anomaly fields, and associated context when the target data is abnormal are obtained, so as to facilitate accurate location and rapid repair of the abnormal data in the future.

[0029] S22: Based on the anomaly type, anomaly degree, anomaly fields, and associated context, perform anomaly classification and anomaly root cause reasoning to obtain anomaly classification results and anomaly root cause results.

[0030] In step S22, the anomaly classification result is an anomaly category mapped to a preset knowledge base based on anomaly type, anomaly degree and associated context, and the anomaly root cause result is the anomaly cause derived based on multidimensional feature association analysis.

[0031] In this embodiment, when classifying and reasoning about the root causes of anomalies based on anomaly type, anomaly severity, anomaly fields, and associated context, anomaly classification can be performed using a preset classification model, and root cause reasoning can be performed using a knowledge graph. Specifically, when classifying anomalies using the preset classification model, the model is a multi-feature fusion hierarchical classification model adapted to four specific anomalies: data entry errors, system bugs, logical defects, and specification changes. Its input dimensions include anomaly type, anomaly severity, anomaly fields, and associated context, accurately capturing the feature boundaries and distribution patterns of each of the four anomalies. That is, the anomaly type, anomaly severity, anomaly fields, and associated context are input into the preset classification model, and the anomaly classification result is output. The anomaly classification result can include data entry errors, system bugs, logical defects, specification changes, etc.

[0032] It should be noted that the preset classification model can be a large language model, such as a deep learning model based on the Transformer architecture. During training of the preset classification model, training samples and their corresponding ground truth labels are obtained. The training samples include the sample anomaly type, anomaly degree, anomaly fields, and associated context. The ground truth labels are the actual anomaly classification results. The model parameters are optimized using a backpropagation algorithm to minimize the loss between the predicted results and the ground truth labels until the model converges, resulting in a trained classification model.

[0033] When performing anomaly root cause reasoning using knowledge graphs, the anomaly type, anomaly severity, anomaly field, and associated context are input into the corresponding knowledge graph, and the anomaly root cause result is output. Specifically, centered on the anomaly field, the relationships within the knowledge graph are traversed, and a subgraph of the associated links is extracted. The anomaly type, anomaly severity, anomaly field, and associated context are used as constraints to filter valid causal paths, and the anomaly root cause is determined based on these valid causal paths.

[0034] It's important to note that when using knowledge graphs for anomaly root cause reasoning, a corresponding knowledge graph needs to be pre-constructed. This graph consists of three parts: entities, relationships, and rule triples. Entities can include anomaly entities, context entities, and root cause entities. Anomaly entities include anomaly type, anomaly severity, and anomaly fields. Context entities include data sources, upstream interfaces, dependency tables, operational procedures, and business dimensions. Root cause entities include issues such as missing field synchronization, incorrect interface parameters, unreasonable threshold configuration, inconsistent encoding, failed scheduled tasks, and upstream data truncation. The relationship layer defines directional causal, dependency, and association relationships. For example, an anomaly field might depend on an upstream interface, and a format error might be rooted in a mismatched encoding format. The rule triple layer stores industry validation rules and prior knowledge of faults, such as numerical out-of-bounds conditions and excessively low field threshold limits.

[0035] The anomaly type, anomaly severity, anomaly fields, and associated context are input into the corresponding knowledge graph. The input ternary information is broken down, the associated context is loaded, entities are aligned and matched, and a set of query starting nodes is generated. Starting from the starting node, the graph is traversed bidirectionally along the directed relation edges in both the forward and reverse directions to uncover all potential causal paths, find all intermediate triggers and underlying root causes that can trigger the current anomaly, and determine the root cause of the anomaly.

[0036] In this embodiment, anomaly classification and root cause reasoning are performed using multi-dimensional features to quickly locate the source of the fault and shorten the troubleshooting cycle. This method not only improves operational efficiency but also significantly reduces the misjudgment rate and time cost of manual troubleshooting.

[0037] Among them, such as Figure 3As shown, step S22, which involves classifying and reasoning about the root causes of anomalies based on their type, severity, fields, and context, yields the classification and root cause results, including: S31: Obtain the preset instruction template, fill in the exception type, exception level, exception fields and related context into the instruction template, and generate structured reasoning instructions; S32: Input the structured reasoning instructions into the preset large language model and output the anomaly classification results and anomaly root cause results.

[0038] In this embodiment, a preset instruction template is obtained. This template is a fixed text frame with a predefined format. The frame reserves positions for filling corresponding anomaly feature parameters, enabling the organization of scattered anomaly feature information into a unified natural language description. This adapts to the input format requirements of large language models, facilitating accurate reading of information from various anomaly dimensions and avoiding feature omissions or semantic confusion. The preset instruction template may include placeholders for anomaly type, anomaly severity, anomaly fields, and associated context. The anomaly type, anomaly severity, anomaly fields, and associated context are filled into the instruction template to generate structured inference instructions. For example, if the anomaly type is "memory overflow," the anomaly severity is "high," the anomaly field is "JVM heap memory," and the associated context is "no recent code releases but traffic surge," the corresponding generated structured inference instructions would be: Anomaly Type - Memory Overflow, Anomaly Severity - High, Anomaly Field - JVM Heap Memory, Associated Context - No recent code releases but traffic surge.

[0039] The structured inference instructions are input into a pre-defined large language model, which outputs anomaly classification results and anomaly root cause results. The pre-defined large language model is used to parse key semantic elements in the structured instructions and perform anomaly classification and root cause inference based on these semantic elements. The pre-defined large language model can be a deep learning model based on the Transformer architecture. During training, training samples and ground truth labels are obtained. The training samples are the sample structured inference instructions, and the ground truth labels are the corresponding ground truth anomaly classification results and ground truth anomaly root cause results. The sample structured inference instructions include the anomaly type, anomaly degree, anomaly field, and associated context of the sample anomaly event. Supervised learning is used to optimize the model parameters. When the error between the model's prediction and the ground truth labels converges to a pre-defined threshold, the model training is complete, and the trained large language model, i.e., the pre-defined large language model, is obtained.

[0040] In this embodiment, by integrating abnormal information into standardized structured reasoning instructions and inputting them into the large language model, the input format can be standardized, helping the large language model to quickly and accurately extract key information and reduce semantic bias. At the same time, it can make full use of the strong semantic understanding and reasoning capabilities of the large language model, improving the accuracy and efficiency of abnormal classification and root cause localization. There is no need to build complex reasoning models separately for specific business scenarios, which reduces the development and adaptation costs of abnormal handling scenarios. It can also adapt to the data abnormal handling needs of different types and fields, and has strong versatility.

[0041] S23: Based on the anomaly type, anomaly degree, anomaly fields, associated context, anomaly classification results, and anomaly root cause results, perform anomaly pattern recognition on the anomaly event to obtain the target anomaly pattern.

[0042] In step S23, the abnormal pattern is a standardized summary of similar abnormal phenomena, occurrence patterns, triggering characteristics, and propagation paths.

[0043] In this embodiment, when performing anomaly pattern recognition on anomaly events based on anomaly type, anomaly degree, anomaly fields, associated context, anomaly classification results, and anomaly root cause results, a preset anomaly pattern recognition model can be used for recognition. The preset anomaly pattern recognition model can be a classifier based on a deep neural network. The anomaly type, anomaly degree, anomaly fields, associated context, anomaly classification results, and anomaly root cause results are input into the preset anomaly pattern recognition model, and the target anomaly pattern is output.

[0044] It should be noted that when training the anomaly pattern recognition model, corresponding training samples and their ground truth labels are obtained. The training samples include the anomaly type, anomaly severity, anomaly fields, associated context, anomaly classification result, and root cause result of the anomaly event. The ground truth labels are the corresponding real anomaly patterns. Supervised learning is used to optimize the model parameters until the error between the prediction result and the ground truth label converges to a preset threshold. This completes the model training, resulting in the trained anomaly pattern recognition model, which is the preset anomaly pattern recognition model.

[0045] In this embodiment, the trained anomaly pattern recognition model can accurately capture the nonlinear correlation between multidimensional features, thereby achieving in-depth analysis of complex anomaly evolution paths and improving the accuracy and robustness of anomaly pattern recognition.

[0046] Among them, such as Figure 4 As shown, step S23, which involves performing anomaly pattern recognition on the abnormal event based on the anomaly type, anomaly degree, anomaly fields, associated context, anomaly classification result, and anomaly root cause result, to obtain the target anomaly pattern, includes: S41: Construct a multidimensional feature vector based on the anomaly type, anomaly degree, anomaly field, associated context, anomaly classification result, and anomaly root cause result; S42: Obtain a preset exception pattern library, which contains multiple exception patterns and the center vector of each exception pattern; S43: Calculate the cosine similarity between the multidimensional feature vector and each center vector, and select the anomaly pattern with the highest similarity as the target anomaly pattern.

[0047] In this embodiment, when performing anomaly pattern recognition on anomaly events based on anomaly type, anomaly degree, anomaly fields, associated context, anomaly classification results, and anomaly root cause results, a multi-dimensional feature vector can be constructed based on these factors. A clustering algorithm is then used for anomaly pattern recognition to obtain the target anomaly pattern. Specifically, when constructing the multi-dimensional feature vector based on anomaly type, anomaly degree, anomaly fields, associated context, anomaly classification results, and anomaly root cause results, the data for each dimension can be numerically encoded and normalized to obtain the multi-dimensional feature vector, ensuring that the data for each dimension participates in the calculation under a unified scale.

[0048] A pre-defined anomaly pattern library is obtained, containing multiple anomaly patterns and a center vector for each pattern. These center vectors represent the core distribution of various typical anomaly patterns in the feature space. The cosine similarity between the multidimensional feature vectors and each center vector is calculated, and the anomaly pattern with the highest similarity is selected as the target anomaly pattern. The target anomaly pattern can include pattern categories such as systematic anomalies, local anomalies, or sporadic anomalies.

[0049] It should be noted that if the calculated highest cosine similarity is lower than a preset threshold, the current anomalous event is determined not to belong to any typical anomalous patterns already existing in the database. In this case, the anomalous event can be marked as a new anomalous pattern to be explored and stored in a temporary candidate pattern set. Technical personnel will periodically review and label the anomalous events in the set manually. Once it is confirmed that the event belongs to a new anomalous pattern, the corresponding center vector will be extracted and added to the anomalous pattern database to complete the dynamic iteration of the anomalous pattern database and improve the coverage and accuracy of subsequent anomalous pattern recognition.

[0050] In this embodiment, abnormal events are identified based on their abnormality type, degree, fields, context, classification results, and root cause results. This allows for accurate identification of existing abnormal patterns and rapid matching of corresponding solutions. Furthermore, it enables automatic identification of new abnormalities and autonomous updates to the abnormality pattern library, avoiding the tedious manual maintenance of the library. This also allows the accuracy of abnormality identification to gradually improve with the number of identifications, reducing the probability of misjudging unfamiliar abnormalities.

[0051] Among them, such as Figure 5 As shown, step S42, which is to obtain the preset exception mode library, includes: S51: Obtain multiple historical abnormal event samples and the corresponding abnormal feature vector samples for each historical abnormal event sample; S52: Calculate the distance matrix between all pairs of anomalous feature vector samples. Based on the distance matrix, find the K nearest anomalous feature vector samples for each anomalous feature vector sample, where K is an integer greater than zero. S53: For any target abnormal feature vector sample, take the abnormal feature vector sample that is farthest from the target abnormal feature vector sample among the K abnormal feature vector samples as the boundary reference of the target abnormal feature vector sample, and construct the local density distribution of the target abnormal feature vector sample based on the boundary reference. S54: Based on each local density distribution, cluster all historical anomalous event samples to obtain multiple anomalous pattern clusters; S55: For any abnormal pattern cluster, calculate the mean vector of all abnormal feature vector samples within the abnormal pattern cluster to obtain the center vector of the corresponding abnormal pattern, and construct a preset abnormal pattern library based on multiple abnormal pattern clusters and the center vector corresponding to each abnormal pattern cluster.

[0052] In this embodiment, when constructing a preset abnormal pattern library, multiple historical abnormal event samples and corresponding abnormal feature vector samples are obtained. The historical abnormal event samples are real abnormal data recorded by the system over a past period, and the abnormal feature vector samples are numerical representations obtained after feature extraction and vectorization of the historical abnormal events, including multi-dimensional features such as the abnormality type, abnormality degree, abnormal fields, associated context, abnormality classification results, and root cause analysis results corresponding to the historical abnormal event samples.

[0053] Calculate the pairwise distance matrix between all anomalous feature vector samples. Based on the distance matrix, find the K nearest anomalous feature vector samples for each anomalous feature vector sample. The choice of K should balance the refinement of the local structure with computational efficiency. When calculating the pairwise distance matrix, Euclidean distance, cosine similarity, or Mahalanobis distance can be used; this example does not impose specific limitations and can be flexibly selected based on the actual data distribution characteristics. The distance matrix is ​​a symmetric matrix, where each element represents the distance between any two anomalous feature vector samples. When finding the K nearest anomalous feature vector samples for each anomalous feature vector sample, KD-trees, ball trees, or approximate nearest neighbor search algorithms can be used to accelerate the retrieval process, thereby significantly reducing computational complexity in large-scale data scenarios.

[0054] For any target anomaly feature vector sample, the anomaly feature vector sample farthest from the target anomaly feature vector sample among K anomaly feature vector samples is used as the boundary reference of the target anomaly feature vector sample. The boundary reference is a key anchor point defining the local neighborhood range of this sample; that is, the radius of the local neighborhood is determined by this anchor point, thereby quantifying the sparsity of the target anomaly feature vector sample in the feature space. Based on the boundary reference, when constructing the local density distribution of the target anomaly feature vector sample, a kernel density estimation method can be introduced. Using the radius determined by the boundary reference as the bandwidth parameter, the probability density function around the target anomaly feature vector sample is smoothly estimated. That is, the density value of the target anomaly feature vector sample in the local neighborhood is obtained through integration. This density value intuitively reflects the clustering state of the target anomaly feature vector sample in the feature space; the higher the density, the stronger the similarity between the target anomaly feature vector sample and its surrounding neighbors.

[0055] Based on each local density distribution, all historical anomalous event samples are clustered to obtain multiple anomalous pattern clusters. When clustering all historical anomalous event samples based on each local density distribution, a density-based clustering algorithm, such as HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise), can be used. All historical anomalous event samples are marked as unvisited. Each unvisited sample is traversed sequentially, and its local density value is used as the core to determine if it meets the core object condition. The core object condition is that its local density value is greater than a preset minimum density threshold. If the core object condition is met, all locally reachable sample sets are found from that historical anomalous event sample. This set is divided into an independent anomalous pattern cluster, and all samples within the cluster are marked as visited. After traversal, all samples not assigned to any cluster are marked as noise points. Finally, multiple anomalous pattern clusters corresponding to different anomalous characteristics are obtained, completing the clustering process.

[0056] For any anomalous pattern cluster, the mean vector of all anomalous feature vector samples within the anomalous pattern cluster is calculated to obtain the center vector of the corresponding anomalous pattern. This center vector is then used to characterize the core features of the anomalous pattern, thereby constructing an anomalous pattern library.

[0057] In this embodiment, by performing density clustering on the features of historical abnormal events, different types of abnormal patterns are automatically identified. This eliminates the need for manually labeled abnormal types, reducing the cost of manual labeling and improving the efficiency of building the abnormal pattern library. At the same time, by constructing a local density distribution through boundary references, it is possible to better distinguish abnormal clusters of different densities, avoid deviations in clustering results when density differences are large, and improve the accuracy of abnormal pattern classification.

[0058] S24: Based on the anomaly type, the root cause of the anomaly, and the target anomaly pattern, determine a set of candidate handling strategies that match the anomaly event.

[0059] In step S24, the candidate disposal strategy set is a set of recommended solutions generated based on historical disposal records and expert knowledge base, including multiple candidate disposal strategies.

[0060] In this embodiment, when determining the set of candidate handling strategies matching the abnormal event based on the abnormality type, the root cause result of the abnormality, and the target abnormality pattern, a multi-dimensional hierarchical matching combined with strategy screening and verification method can be adopted. First, a preset abnormality strategy mapping database is constructed. The mapping database pre-stores one-to-one correspondences and many-to-many relationships between various abnormality types, abnormal root causes, abnormal patterns, and handling strategies. Each preset handling strategy is bound to a corresponding abnormality adaptation tag. Based on the abnormality type, the root cause result of the abnormality, and the target abnormality pattern, a search is performed in the preset abnormality strategy mapping database to filter out all initial handling strategies that match single-dimensional, two-dimensional, and three-dimensional features. That is, handling strategies that match any dimension of the abnormality type, abnormal root cause, or target abnormality pattern are selected, or handling strategies that match the abnormality type and abnormal root cause, the abnormal root cause and the abnormal pattern, or the abnormality type and the abnormal pattern in pairs, or handling strategies that completely match the three dimensions of abnormality type, root cause, and pattern, thus obtaining the initial handling strategy set.

[0061] After selecting all initial treatment strategy sets that match single-dimensional, two-dimensional, and three-dimensional features, a dynamic weighted scoring mechanism is introduced. Differential weights are assigned to the matching confidence scores of anomaly type, anomaly root cause, and anomaly pattern. For example, a higher weight is assigned to the matching confidence score of anomaly pattern, a medium weight to the matching confidence score of anomaly root cause, and a basic weight to the matching confidence score of anomaly type. For instance, a weight of 0.6 is assigned to the matching confidence score of anomaly pattern, a weight of 0.3 to the matching confidence score of anomaly root cause, and a weight of 0.1 to the matching confidence score of anomaly type, etc. Other weight values ​​can also be used; this embodiment is not limited. Based on the corresponding weight values, the treatment strategies in the initial treatment strategy set are weighted and summed to obtain the comprehensive matching score for each strategy. The initial treatment strategy set is then sorted in descending order according to the comprehensive matching score, and the top N high-scoring treatment strategies are selected as the candidate treatment strategy set. In this embodiment, by jointly matching the three dimensions of anomaly type, anomaly root cause, and target anomaly pattern, the appropriate handling strategy for the current anomaly scenario can be accurately identified. At the same time, by using quantitative scoring and priority ranking, the effectiveness, rationality, and feasibility of the candidate handling strategy set are ensured, which greatly improves the accuracy and efficiency of anomaly event handling.

[0062] Among them, such as Figure 6 As shown, step S24, which involves determining a set of candidate handling strategies matching the abnormal event based on the anomaly type, the root cause of the anomaly, and the target anomaly pattern, includes: S61: Obtain the mapping table between anomaly type, anomaly root cause result, target anomaly pattern and handling strategy; S62: Based on the anomaly type, the root cause of the anomaly, and the target anomaly pattern, and in conjunction with the mapping relationship table, determine the initial set of handling strategies that match the anomaly event; S63: Obtain the treatment effect score of each initial treatment strategy, and filter the initial treatment strategies with treatment effect scores greater than the preset score threshold as candidate treatment strategies to obtain a set of candidate treatment strategies.

[0063] In this embodiment, a mapping table is obtained between anomaly type, root cause result, target anomaly pattern, and handling strategy. This mapping table pre-stores the association data between multi-dimensional features accumulated during historical anomaly event handling and corresponding handling strategies. When determining the initial handling strategy set matching the anomaly event based on the anomaly type, root cause result, and target anomaly pattern, and in conjunction with the mapping table, the initial handling strategy set is obtained by retrieving the handling strategy that perfectly matches the current anomaly features from the mapping table. Specifically, the initial handling strategy set is obtained by retrieving the handling strategy that matches the anomaly type, root cause result, and target anomaly pattern from the mapping table.

[0064] Obtain the handling effect score for each initial handling strategy. The handling effect score is a quantitative value calculated based on indicators such as the frequency of successful resolution of similar abnormal events, average response time, and resource consumption of the strategy in historical handling records. That is, the handling effect score equals the success frequency multiplied by the time efficiency coefficient and then divided by the resource consumption coefficient. The success frequency is the ratio of the number of times the handling strategy successfully resolved the problem in similar abnormal events in history to the total number of calls. The time efficiency coefficient is the reciprocal of the average response time of the handling strategy in similar abnormal events in history. The resource consumption coefficient is the ratio of the system resources used during the execution of the handling strategy to the standard resource threshold.

[0065] Initial treatment strategies with performance scores exceeding a preset threshold are selected as candidate treatment strategies, resulting in a candidate strategy set. This selection mechanism effectively eliminates inefficient or resource-intensive strategies, ensuring that the candidate strategy set combines high success rate with low resource consumption.

[0066] In this embodiment, the initial strategy is precisely screened based on the handling effect, and invalid handling strategies with low success rate, slow response and high resource consumption are automatically filtered out, so as to avoid wasting system resources by executing inefficient strategies in the future, while greatly increasing the probability of matching effective handling solutions, and ensuring the efficiency and effectiveness of subsequent abnormal handling.

[0067] S25: Obtain the handling effect score and usage frequency of each candidate handling strategy. Based on the handling effect score and usage frequency, calculate the comprehensive suitability score of each candidate handling strategy. Determine the candidate handling strategy with the highest comprehensive suitability score as the optimal handling strategy and use the optimal handling strategy to handle the abnormal event.

[0068] In step S25, the handling effect score is a strategy effectiveness index quantified based on historical data. The usage frequency is the ratio of the cumulative number of times the strategy was actually invoked to the total number of invocations within a historical period, reflecting the popularity and stability of the handling strategy in actual operation and maintenance scenarios. The comprehensive adaptability score characterizes the overall level of the handling strategy's effectiveness and acceptance in actual application. The candidate handling strategy corresponding to the highest comprehensive adaptability score is determined as the optimal handling strategy, and the optimal handling strategy is used to handle the abnormal event.

[0069] In this embodiment, the treatment effect score and usage frequency of each candidate treatment strategy are obtained. Based on the treatment effect score and usage frequency, the comprehensive suitability score of each candidate treatment strategy is calculated. The formula for calculating the comprehensive suitability score is as follows: in, The overall fit score for handling strategy s, As a handling strategy, Candidate disposal strategy set, Score the effectiveness of the treatment strategy s. The frequency of use of disposal strategy s is given, and disposal strategy s is a candidate disposal strategy. For parameters, The value of is determined based on the actual situation. It should be noted that, in order to unify the dimensions of the treatment effect score and the frequency of use, the treatment effect score and the frequency of use are normalized respectively, and the values ​​of the two indicators are mapped to the interval [0,1].

[0070] The candidate handling strategy corresponding to the highest comprehensive fit score is determined as the optimal handling strategy, and the optimal handling strategy is used to handle the abnormal event.

[0071] In this embodiment, by combining the two dimensions of handling effect score and usage frequency for calculation, it is ensured that the selected optimal strategy itself has excellent handling performance and also conforms to the common application preferences in actual operation and maintenance scenarios. This can further improve the success rate and stability of this anomaly handling and ensure the reliability of anomaly handling.

[0072] Among them, such as Figure 7 As shown, step S25 involves obtaining the treatment effect score and usage frequency of each candidate treatment strategy, and calculating the comprehensive suitability score of each candidate treatment strategy based on the treatment effect score and usage frequency, including: S71: For any candidate treatment strategy, the treatment effect score is graded to obtain the corresponding effect level; S72: Determine the weighting coefficients that match the effectiveness level, and sum the treatment effectiveness score and the frequency of use based on the weighting coefficients to obtain the comprehensive suitability score.

[0073] In this embodiment, when classifying the treatment effect score for any candidate treatment strategy, a pre-defined scoring range can be established for different levels. For example, the score can be divided into four levels: excellent, good, satisfactory, and unsatisfactory. Each level corresponds to a pre-defined scoring range. Then, the treatment effect score to be calculated is matched to the corresponding range to determine its effect level. A weighting coefficient matching the effect level is determined. Based on the weighting coefficient, the treatment effect score and usage frequency are weighted and summed. That is, based on the effect level, the weight value of the treatment effect score is determined. The difference between the weight value of the treatment effect score and 1 is determined as the weight value of the usage frequency. The treatment effect score and usage frequency are weighted and summed to obtain the comprehensive suitability score.

[0074] It should be noted that, in order to unify the dimensions of the treatment effect score and the frequency of use, the treatment effect score and the frequency of use are normalized separately, and the values ​​of the two indicators are mapped to the range of [0,1] before subsequent weighted calculation. This avoids the impact of one indicator on the overall fit score being unreasonably amplified or reduced due to the large difference in the original value range of the two indicators, and ensures that the final overall fit score can accurately reflect the actual fit level of the candidate treatment strategy.

[0075] In this embodiment, the influence ratio of the two indicators on the overall suitability is dynamically adjusted according to the actual handling needs. When the handling effect is good, the weight coefficient corresponding to the handling effect score can be increased to amplify its influence on the overall score. When the handling effect is poor, the weight coefficient of the handling effect score can be decreased to increase the proportion of the usage frequency on the overall score, so that the final selected handling strategy is more in line with the current abnormal data handling needs and improves the flexibility and adaptability of abnormal handling.

[0076] Optionally, after handling the abnormal event using the optimal handling strategy, the process also includes: Update the mapping table based on the anomaly type, root cause, target anomaly pattern, and optimal handling strategy corresponding to the anomaly event.

[0077] In this embodiment, the mapping table is a core data table used to store the correlation between anomaly characteristics and handling strategies. After the optimal handling strategy is executed and monitoring confirms that the current anomaly has been eliminated and business operations have returned to normal, the system automatically collects complete anomaly characteristic parameters and handling data, including key information such as the identified anomaly type, root cause result, target anomaly pattern, and optimal handling strategy. Based on the anomaly type, root cause result, and target anomaly pattern, the system re-clusters and updates the mapping table. This allows the system to dynamically correct the mapping relationships between anomaly type, root cause, anomaly pattern, and handling strategies based on real feedback data. This provides more accurate data support for rapid response to similar anomalies in the future.

[0078] This application employs a layered, progressive anomaly analysis by combining multi-dimensional information from the occurrence of abnormal events. From classification and localization to root cause analysis and pattern recognition, the scope of anomaly analysis is gradually narrowed, avoiding judgment biases caused by single-dimensional analysis and improving the accuracy of the results. By calculating the comprehensive suitability through two dimensions—treatment effectiveness score and usage frequency—it can select treatment strategies that have been proven effective in practice and meet the matching needs of the current anomaly scenario. This avoids the one-sidedness caused by relying solely on a single indicator for strategy selection, effectively improving the success rate of anomaly handling and reducing the computational and time costs of repeated handling. The entire process of anomaly data processing, from analysis to handling, is automated, eliminating the need for manual intervention in sorting anomaly information and matching treatment plans. This shortens the response cycle for anomaly handling, thereby improving data processing efficiency.

[0079] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0080] In one embodiment, an abnormal data processing apparatus is provided, which corresponds one-to-one with the abnormal data processing methods described in the above embodiments. For example... Figure 8 As shown, the abnormal data processing device 80 includes an acquisition module 81, a obtaining module 82, an identification module 83, a determination module 84, and a calculation module 85.

[0081] The acquisition module 81 is used to acquire abnormal event records when the target data is abnormal. The abnormal event records include the abnormal type, abnormality, abnormal fields and associated context of the abnormal event.

[0082] Module 82 is used to classify anomalies and infer root causes based on anomaly type, anomaly degree, anomaly fields and associated context, and obtain anomaly classification results and anomaly root cause results.

[0083] The identification module 83 is used to identify the abnormal pattern of the abnormal event based on the abnormal type, abnormal degree, abnormal field, associated context, abnormal classification result and abnormal root cause result, so as to obtain the target abnormal pattern. The determination module 84 is used to determine a set of candidate handling strategies that match the abnormal event based on the abnormality type, the root cause of the abnormality, and the target abnormality pattern.

[0084] The calculation module 85 is used to obtain the treatment effect score and usage frequency of each candidate treatment strategy. Based on the treatment effect score and usage frequency, the comprehensive suitability score of each candidate treatment strategy is calculated. The candidate treatment strategy with the highest comprehensive suitability score is determined as the optimal treatment strategy, and the optimal treatment strategy is used to handle the abnormal event.

[0085] Optionally, module 82 includes: The first acquisition unit is used to acquire a preset instruction template, fill in the exception type, exception degree, exception fields and associated context into the instruction template, and generate structured reasoning instructions.

[0086] The output unit is used to input structured reasoning instructions into a preset large language model and output anomaly classification results and anomaly root cause results.

[0087] Optionally, the recognition module 83 includes: The building unit is used to construct a multidimensional feature vector based on the anomaly type, anomaly degree, anomaly field, associated context, anomaly classification result, and anomaly root cause result.

[0088] The second acquisition unit is used to acquire a preset abnormal pattern library, which contains multiple abnormal patterns and the center vector of each abnormal pattern.

[0089] The calculation unit is used to calculate the cosine similarity between the multidimensional feature vector and each center vector, and select the anomaly pattern with the highest similarity as the target anomaly pattern.

[0090] Optionally, the second acquisition unit includes: The acquisition sub-unit is used to acquire multiple historical abnormal event samples and the corresponding abnormal feature vector samples for each historical abnormal event sample.

[0091] The first calculation subunit is used to calculate the distance matrix between each pair of all anomalous feature vector samples. Based on the distance matrix, it finds the K nearest anomalous feature vector samples for each anomalous feature vector sample, where K is an integer greater than zero.

[0092] A sub-unit is constructed to take the K abnormal feature vector samples that are farthest from the target abnormal feature vector sample as the boundary reference of the target abnormal feature vector sample, and construct the local density distribution of the target abnormal feature vector sample based on the boundary reference.

[0093] The clustering subunit is used to cluster all historical anomalous event samples according to each local density distribution, resulting in multiple anomalous pattern clusters.

[0094] The second calculation subunit is used to calculate the mean vector of all abnormal feature vector samples within any abnormal pattern cluster, obtain the center vector of the corresponding abnormal pattern, and construct a preset abnormal pattern library based on multiple abnormal pattern clusters and the center vector corresponding to each abnormal pattern cluster.

[0095] Optionally, the determining module 84 includes: The third acquisition unit is used to acquire a mapping table between anomaly type, anomaly root cause result, target anomaly pattern and handling strategy.

[0096] The second determining unit is used to determine the initial handling strategy set that matches the abnormal event based on the abnormality type, the root cause of the abnormality, and the target abnormality pattern, combined with the mapping relationship table.

[0097] The filtering unit is used to obtain the treatment effect score of each initial treatment strategy, and filter the initial treatment strategies with treatment effect scores greater than a preset score threshold as candidate treatment strategies to obtain a set of candidate treatment strategies.

[0098] Optionally, the computing module 85 includes: The division unit is used to classify the treatment effect score for any candidate treatment strategy and obtain the corresponding effect level.

[0099] The summation unit is used to determine the weighting coefficients that match the effectiveness level. Based on the weighting coefficients, the treatment effectiveness score and the frequency of use are weighted and summed to obtain the comprehensive suitability score.

[0100] Optionally, the abnormal data processing device 80 further includes: The update module is used to update the mapping relationship table based on the exception type, root cause result, target exception mode, and optimal handling strategy corresponding to the exception event.

[0101] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0102] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 9 As shown, the computer device of this embodiment includes: at least one processor ( Figure 9 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, wherein the processor executes the computer program to implement the steps in any of the above-described exception data processing method embodiments.

[0103] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 9 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0104] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0105] Memory includes non-volatile and / or volatile readable storage media, internal memory, etc., wherein internal memory may be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media may be the hard disk of a computer device, or in other embodiments, it may be an external storage device of the computer device, such as a plug-in hard disk, a Smart MediaCard (SMC), a Secure Digital (SD) card, a Flash Card, etc., provided on the computer device. Furthermore, memory may include both internal storage units and external storage devices of the computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code of computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, disks, or optical discs. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0107] The present invention can implement all or part of the processes in the above embodiments of the method, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0108] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0109] Those skilled in the art will understand that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0110] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of apparatuses or units may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0113] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for processing abnormal data, characterized in that, The abnormal data processing method includes: Obtain anomaly event records when target data is abnormal, the anomaly event records include the anomaly type, anomaly severity, anomaly fields and associated context corresponding to the anomaly event; Based on the anomaly type, the anomaly degree, the anomaly field and the associated context, anomaly classification and anomaly root cause inference are performed to obtain anomaly classification results and anomaly root cause results. Based on the anomaly type, the anomaly degree, the anomaly field, the associated context, the anomaly classification result, and the anomaly root cause result, anomaly pattern recognition is performed on the anomaly event to obtain the target anomaly pattern; Based on the anomaly type, the anomaly root cause result, and the target anomaly pattern, determine a set of candidate handling strategies that match the anomaly event; Obtain the treatment effect score and usage frequency of each candidate treatment strategy. Calculate the comprehensive suitability score of each candidate treatment strategy based on the treatment effect score and usage frequency. Determine the candidate treatment strategy with the highest comprehensive suitability score as the optimal treatment strategy and use the optimal treatment strategy to handle the abnormal event.

2. The abnormal data processing method as described in claim 1, characterized in that, The step of performing anomaly classification and root cause inference based on the anomaly type, the anomaly degree, the anomaly field, and the associated context to obtain anomaly classification results and anomaly root cause results includes: Obtain a preset instruction template, fill in the exception type, the exception degree, the exception field and the associated context into the instruction template, and generate a structured reasoning instruction; The structured reasoning instructions are input into a preset large language model, and the anomaly classification results and anomaly root cause results are output.

3. The abnormal data processing method as described in claim 1, characterized in that, The step of identifying anomaly patterns in the abnormal event based on the anomaly type, the anomaly degree, the anomaly fields, the associated context, the anomaly classification result, and the anomaly root cause result to obtain the target anomaly pattern includes: A multidimensional feature vector is constructed based on the anomaly type, the anomaly degree, the anomaly field, the associated context, the anomaly classification result, and the anomaly root cause result; Obtain a preset abnormal pattern library, which contains multiple abnormal patterns and a center vector for each abnormal pattern; Calculate the cosine similarity between the multidimensional feature vector and each center vector, and select the anomaly pattern with the highest similarity as the target anomaly pattern.

4. The abnormal data processing method as described in claim 3, characterized in that, The process of obtaining the preset abnormal pattern library includes: Obtain multiple historical abnormal event samples and the corresponding abnormal feature vector samples for each historical abnormal event sample; Calculate the distance matrix between all pairs of anomalous feature vector samples. Based on the distance matrix, find the K nearest anomalous feature vector samples for each anomalous feature vector sample, where K is an integer greater than zero. For any target anomaly feature vector sample, the anomaly feature vector sample that is farthest from the target anomaly feature vector sample among K anomaly feature vector samples is used as the boundary reference of the target anomaly feature vector sample. Based on the boundary reference, the local density distribution of the target anomaly feature vector sample is constructed. Based on each local density distribution, all historical anomalous event samples are clustered to obtain multiple anomalous pattern clusters; For any abnormal pattern cluster, the mean vector of all abnormal feature vector samples within the abnormal pattern cluster is calculated to obtain the center vector of the corresponding abnormal pattern. A preset abnormal pattern library is constructed based on multiple abnormal pattern clusters and the center vector corresponding to each abnormal pattern cluster.

5. The abnormal data processing method as described in claim 1, characterized in that, The step of determining a set of candidate handling strategies matching the abnormal event based on the abnormality type, the root cause result of the abnormality, and the target abnormality pattern includes: Obtain a mapping table between the anomaly type, the root cause result of the anomaly, the target anomaly pattern, and the handling strategy; Based on the anomaly type, the anomaly root cause result, and the target anomaly pattern, and in conjunction with the mapping relationship table, determine the initial handling strategy set that matches the anomaly event; Obtain the treatment effect score for each initial treatment strategy, and filter the initial treatment strategies with treatment effect scores greater than a preset score threshold as candidate treatment strategies to obtain a set of candidate treatment strategies.

6. The abnormal data processing method as described in claim 1, characterized in that, The process involves obtaining the treatment effect score and usage frequency of each candidate treatment strategy, and calculating the comprehensive suitability score of each candidate treatment strategy based on the treatment effect score and usage frequency, including: For any candidate treatment strategy, the treatment effect score is divided into levels to obtain the corresponding effect level; Determine the weighting coefficient that matches the effect level, and sum the treatment effect score and the usage frequency according to the weighting coefficient to obtain the comprehensive suitability score.

7. The abnormal data processing method as described in claim 5, characterized in that, After processing the abnormal event using the optimal handling strategy, the method further includes: The mapping table is updated based on the anomaly type corresponding to the anomaly event, the root cause of the anomaly, the target anomaly pattern, and the optimal handling strategy.

8. An abnormal data processing device, characterized in that, The abnormal data processing device includes: The acquisition module is used to acquire records of abnormal events when the target data is abnormal. The abnormal event records include the abnormal type, abnormality, abnormal fields and associated context of the abnormal event. The module is used to perform anomaly classification and anomaly root cause reasoning based on the anomaly type, the anomaly degree, the anomaly field and the associated context, and to obtain anomaly classification results and anomaly root cause results. The identification module is used to identify the abnormal event based on the abnormality type, the abnormality degree, the abnormal field, the associated context, the abnormality classification result and the abnormal root cause result, so as to obtain the target abnormality pattern. The determination module is used to determine a set of candidate handling strategies that match the abnormal event based on the abnormality type, the abnormality root cause result and the target abnormality pattern; The calculation module is used to obtain the treatment effect score and usage frequency of each candidate treatment strategy, calculate the comprehensive suitability score of each candidate treatment strategy based on the treatment effect score and usage frequency, determine the candidate treatment strategy corresponding to the largest comprehensive suitability score as the optimal treatment strategy, and use the optimal treatment strategy to process the abnormal event.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the abnormal data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the abnormal data processing method as described in any one of claims 1 to 7.

Images