Data exception identification method and device, computer equipment and storage medium

Abstract

The invention discloses a data exception identification method and device, computer equipment and a storage medium, and belongs to the technical field of big data. Firstly, to-be-processed data is extracted, and the to-be-processed data is decomposed into a plurality of data point sets through a data segmentation technology; then, a risk assessment model of a plurality of preset isolated trees is adopted, the model is composed of the isolated trees with the number equal to that of the data point sets, and each tree is abandoned to specially process one data point set; by calculating the path length of the data points in each isolated tree, the isolation degree of the data points is evaluated, and a corresponding score is generated. And finally, according to a preset abnormal score threshold, automatically identifying an isolated tree with abnormally high isolation degree, and judging the corresponding data point set as an abnormal data point set. The invention further relates to the technical field of block chains, and the data stream is stored on the block chain node. According to the method, the advantages of data decomposition and the multi-isolated tree model are effectively combined, accurate identification of data exception is realized, and the data processing efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application belongs to the field of big data technology, and specifically relates to a data anomaly identification method, device, computer equipment and storage medium. Background Art

[0002] The high-risk model is a risk assessment model used to identify and predict customers or entities that may engage in high-risk behaviors (such as frequent claims, fraud, etc.) in the insurance field (especially employer liability insurance and group accident insurance). This model is usually built based on a series of data features (such as corporate credit code, historical claims records, business type, etc.) and evaluates the risk level through complex algorithm analysis. In practical applications, the high-risk model can help insurance companies manage risks more effectively, reduce potential claims losses, and improve business operation efficiency.

[0003] Existing high-risk models are usually optimized using isolated trees, but isolated trees generally only contain one isolated tree, resulting in a two-level differentiation in the accuracy of high-risk models. Because the basic unit of the isolated tree model is an isolated tree (iTree), and a single isolated tree has limited ability to capture data complexity and abnormal characteristics, and high-risk models usually involve diverse data sources and complex business scenarios. These data may contain multiple types of abnormal behaviors, and a single isolated tree is difficult to fully cover all types of abnormalities, resulting in insufficient ability to capture complex data features. The prediction of high-risk behaviors is either too conservative, missing potential threats, or too aggressive, with frequent false positives. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a data anomaly identification method, device, computer equipment and storage medium to solve the technical problems that the existing high-risk flow model is usually optimized using isolated trees and has insufficient ability to capture complex data features, and the accuracy of the prediction results is differentiated into two levels.

[0005] In order to solve the above technical problems, the present application provides a method for identifying data anomalies, which adopts the following technical solutions:

[0006] A method for identifying data anomalies, comprising:

[0007] Receive anomaly identification instructions and obtain the data stream to be processed from the preset multi-source database;

[0008] Perform data segmentation on the data stream to be processed to obtain several sets of data points;

[0009] Input each data point set into a preset risk assessment model in sequence, wherein the risk assessment model includes a number of isolated trees, and the number of isolated trees is equal to the number of data point sets;

[0010] Determine an isolation tree matching each set of data points, and import each set of data points into the matching isolation tree;

[0011] Calculate the path length of each isolated tree, and generate an isolation score of each isolated tree based on the path length of each isolated tree;

[0012] Based on the isolation score of each isolated tree and a preset abnormal score threshold, an abnormal isolated tree is determined, and a set of data points corresponding to the abnormal isolated tree is obtained as an abnormal data point set.

[0013] Furthermore, the data stream to be processed is segmented to obtain several data point sets, including:

[0014] Parse the data stream to be processed and identify the timestamp and data type of each data point in the data stream to be processed;

[0015] The data stream to be processed is segmented based on the preset time window and timestamp to obtain all data points of the data stream to be processed;

[0016] All data points in the data stream to be processed are clustered based on the data type to obtain several sets of data points.

[0017] Furthermore, before each set of data points is sequentially input into the preset risk assessment model, it also includes:

[0018] Count the number of data point sets to obtain the number of data point sets;

[0019] An isolated tree construction algorithm is used to generate a number of isolated trees matching the number of data points in the set by recursive partitioning to obtain an isolated tree set.

[0020] The weight parameter of each isolated tree is calculated, and a risk assessment model is constructed based on the weight parameter of each isolated tree and the isolated tree set.

[0021] Further, determining an isolation tree matching each set of data points, and importing each set of data points into the matching isolation tree, including:

[0022] Identify the data point structure for each set of data points;

[0023] Search the isolated tree structure that matches the data structure of each data point set in the isolated tree set to obtain the matching isolated tree of each data point set;

[0024] According to the tree structure of the matching isolation tree, the data points in each data point set are mapped to the matching isolation tree according to the data point structure.

[0025] Further, according to the tree structure of the matching isolation tree, the data points in each data point set are mapped to the matching isolation tree according to the data point structure, including:

[0026] Identify tree nodes, leaf nodes, and edges in matching isolation trees;

[0027] Identify a data point matching the tree node in the data point set, obtain a first data point, and map a content field of the first data point to the tree node;

[0028] Identify a data point matching the leaf node in the data point set, obtain a second data point, and map a content field of the second data point to the leaf node;

[0029] Identify the association relationship between the first data point and the second data point, obtain a first association relationship, and map the first association relationship to an edge between a tree node and a leaf node;

[0030] Identify the association relationship between the two second data points, obtain the second association relationship, and map the second association relationship to the edge between the two leaf nodes.

[0031] Furthermore, the path length of each isolated tree is calculated, and the isolation score of each isolated tree is generated based on the path length of each isolated tree, including:

[0032] For each isolated tree in the isolated tree set, traverse the entire isolated tree starting from the tree node of each isolated tree until the last leaf node;

[0033] Count the sum of the edges from the tree node to the last leaf node of each isolated tree to get the path length of each isolated tree;

[0034] The number of branches of each isolated tree is identified, and the isolation score of each isolated tree is calculated based on the path length of each isolated tree and the number of branches of each isolated tree.

[0035] Furthermore, before determining the abnormal isolated tree based on the isolation score of each isolated tree and the preset abnormal score threshold, and obtaining the data point set corresponding to the abnormal isolated tree as the abnormal data point set, the method further includes:

[0036] Based on the isolation score of each isolated tree in the isolated tree set, construct an isolation score distribution;

[0037] Calculate the mean and standard deviation of the isolation score distribution;

[0038] Setting anomaly score thresholds based on the mean and standard deviation of the isolation score distribution;

[0039] Based on the isolation score of each isolated tree and the preset abnormal score threshold, the abnormal isolated tree is determined, including:

[0040] Determine in turn whether the isolation score of each isolated tree is greater than the abnormal score threshold;

[0041] If the isolation score of an isolated tree is greater than the abnormal score threshold, the isolated tree with the isolation score greater than the abnormal score threshold is determined as an abnormal isolated tree.

[0042] In order to solve the above technical problems, the embodiment of the present application also provides a data anomaly identification device, which adopts the following technical solution:

[0043] A data anomaly identification device, comprising:

[0044] A data stream acquisition module is used to receive anomaly identification instructions and obtain the data stream to be processed from a preset multi-source database;

[0045] A data segmentation module is used to segment the data stream to be processed to obtain a set of data points;

[0046] A data input module, used to input each data point set into a preset risk assessment model in sequence, wherein the risk assessment model includes a plurality of isolated trees, and the number of isolated trees is equal to the number of data point sets;

[0047] A data point import module, used to determine an isolated tree matching each data point set, and import each data point set into the matching isolated tree;

[0048] An isolation score module, used for calculating the path length of each isolated tree, and generating an isolation score of each isolated tree based on the path length of each isolated tree;

[0049] The anomaly identification module is used to determine the abnormal isolated tree based on the isolation score of each isolated tree and a preset anomaly score threshold, and obtain the data point set corresponding to the abnormal isolated tree as the abnormal data point set.

[0050] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0051] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any one of the above-mentioned data anomaly identification methods when executing the computer-readable instructions.

[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0053] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of any one of the above-mentioned data anomaly identification methods.

[0054] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0055] The present application discloses a data anomaly identification method, device, computer equipment and storage medium, which belongs to the field of big data technology. First, the data to be processed is extracted and decomposed into multiple data point sets by data segmentation technology. Subsequently, a risk assessment model with multiple isolated trees is used, which is composed of isolated trees with the same number as the data point set, and each tree is allocated to process a data point set. By calculating the path length of the data points in each isolated tree, its isolation degree is evaluated and a corresponding score is generated. Finally, according to the preset abnormal score threshold, the isolated tree with abnormally high isolation degree is automatically identified, and its corresponding data point set is determined as an abnormal data point set. The present application realizes efficient anomaly identification of complex data streams through precise data segmentation and matching isolated tree processing, and uses the risk assessment model with multiple isolated trees to effectively reduce the computational complexity and improve the accuracy of anomaly detection. By setting the abnormal score threshold, the abnormal data point set can be automatically screened out to ensure data quality and security. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 An exemplary system architecture diagram is shown in which the present application can be applied;

[0058] Figure 2 A flow chart of an embodiment of a method for identifying data anomalies according to the present application is shown;

[0059] Figure 3 A schematic diagram showing the structure of an embodiment of a data anomaly identification device according to the present application is shown;

[0060] Figure 4 A schematic structural diagram of an embodiment of a computer device according to the present application is shown. DETAILED DESCRIPTION

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0062] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0063] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0064] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0065] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0066] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers and desktop computers, etc.

[0067] Server 105 can be a server that provides various services, such as a background server that provides support for pages displayed on terminal devices 101, 102, and 103. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms.

[0068] It should be noted that the data anomaly identification method provided in the embodiment of the present application is generally executed by a server, and accordingly, the data anomaly identification device is generally arranged in the server.

[0069] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0070] Continue to refer Figure 2 , shows a flow chart of an embodiment of the data anomaly identification method according to the present application. The embodiment of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0071] The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The data anomaly recognition method includes the following steps:

[0072] S201, receiving an abnormality identification instruction, and obtaining a data stream to be processed from a preset multi-source database.

[0073] Specifically, after the instruction is triggered, the system retrieves the data stream to be processed from the preset multi-source database, which can integrate data from data sources of different channels, formats or systems. For example, in an e-commerce risk control scenario, the system may simultaneously obtain data from multiple databases such as user behavior logs, transaction records, and product information to comprehensively analyze user behavior and identify potential risky transactions. By integrating these multi-source data, the system can more accurately construct isolation trees, calculate isolation scores, and ultimately identify abnormal isolation trees, that is, high-risk transactions or abnormal behavior patterns.

[0074] S202, performing data segmentation on the data stream to be processed to obtain a plurality of data point sets.

[0075] Specifically, in the data processing process, the data stream needs to be segmented to obtain a set of data points so that the data stream can be decomposed into a data form that is easier to manage and analyze. This step not only improves processing efficiency, but also makes subsequent analysis work more detailed and accurate. Taking financial risk control as an example, the system receives a data stream containing a large number of transaction records. By segmenting these data streams, multiple data point sets divided by different dimensions such as time, user, and transaction type can be obtained. Each set contains transaction data within a specific range, which is convenient for subsequent construction of isolation trees, calculation of isolation scores, and identification of abnormal isolation trees. In this way, the system can locate abnormal transactions more quickly and take corresponding risk control measures.

[0076] S203, inputting each data point set into a preset risk assessment model in sequence, wherein the risk assessment model includes a plurality of isolated trees, and the number of isolated trees is equal to the number of data point sets.

[0077] Specifically, by assigning a corresponding isolation tree to each data point set for risk assessment, the computational complexity is effectively reduced and the accuracy of anomaly detection is improved. This one-to-one mapping relationship ensures that each data point set can be analyzed in a targeted manner, improving the accuracy and efficiency of risk assessment. Taking fraud detection as an example, each data point set may represent the transaction behavior of a user over a period of time. These sets are input into an equal number of isolation trees. Each tree independently performs anomaly detection on the corresponding set based on its own structure and parameters. If the isolation score of an isolation tree exceeds the threshold, it indicates that the transaction behavior in the set is abnormal and may involve fraud risk, effectively reducing the false positive and false negative rates and improving the accuracy of fraud detection.

[0078] S204, determining an isolated tree matching each data point set, and importing each data point set into the matching isolated tree.

[0079] Specifically, by finding the most appropriate isolation tree for each data point set, it can be ensured that each data set is analyzed and processed in a targeted manner, thereby improving the accuracy and efficiency of risk assessment. By structurally matching the data point set with the corresponding isolation tree, the data points are effectively imported into the corresponding isolation tree. First, by identifying the data point structure of the data point set, the accuracy of subsequent matching is ensured. Then, the isolation tree that matches the data structure of each data point set is searched in the isolation tree set, and the diversity of the isolation tree structure is utilized to achieve customized matching between data points and isolation trees. Finally, according to the tree structure of the matching isolation tree, the data points in the data point set are mapped to the tree, ensuring the effective representation of the data in the isolation tree.

[0080] S205, calculating the path length of each isolated tree, and generating an isolation score of each isolated tree based on the path length of each isolated tree.

[0081] Specifically, by calculating the path length of the isolation tree to generate an isolation score, a quantitative indicator is provided for evaluating the complexity of the data structure and identifying anomalies. The statistics of the path length not only reflect the depth distribution of the data in the tree, but also reveal the intrinsic connection and complexity between the data. Taking financial transaction analysis as an example, each isolation tree may represent a collection of transaction patterns. Calculating the path length of these trees, that is, counting the number of edges from the root node to each leaf node, can reflect the complexity and length of the transaction sequence. The isolation score generated based on the path length can quantitatively evaluate the degree of abnormality of each transaction pattern. A high score may indicate that the transaction pattern is more likely to have abnormal behavior, such as fraud, money laundering, etc., thereby helping financial institutions take timely measures to control risks.

[0082] S206, based on the isolation score of each isolated tree and a preset abnormal score threshold, determine the abnormal isolated tree, and obtain the data point set corresponding to the abnormal isolated tree as the abnormal data point set.

[0083] Specifically, by comparing the isolation score with the preset anomaly score threshold, the accurate identification of abnormal isolated trees is achieved, and the abnormal data point set is extracted accordingly, which not only improves the accuracy of anomaly detection, but also simplifies the subsequent data processing process. When the isolation score of the isolated tree is greater than the anomaly score threshold, the isolated tree with an isolation score greater than the anomaly score threshold is determined as an abnormal isolated tree; when the isolation score of the isolated tree is less than or equal to the anomaly score threshold, the isolated tree with an isolation score less than or equal to the anomaly score threshold is determined as a normal isolated tree. The data point set corresponding to the abnormal isolated tree is obtained as the abnormal data point set, and the specific location of these abnormal data point sets is determined in the data stream to be processed, the abnormal data location information is generated, and the abnormal data location information is output for reference by the tester.

[0084] In the above embodiments, the present application discloses a data anomaly identification method, which belongs to the field of big data technology. The present application realizes efficient anomaly identification of complex data streams through precise data segmentation and matching isolation tree processing, and uses a risk assessment model with multiple isolation trees preset to effectively reduce the computational complexity and improve the accuracy of anomaly detection. By setting an anomaly scoring threshold, a set of abnormal data points can be automatically screened out, ensuring data quality and security.

[0085] Furthermore, the data stream to be processed is segmented to obtain several data point sets, including:

[0086] Parse the data stream to be processed and identify the timestamp and data type of each data point in the data stream to be processed;

[0087] The data stream to be processed is segmented based on the preset time window and timestamp to obtain all data points of the data stream to be processed;

[0088] All data points in the data stream to be processed are clustered based on the data type to obtain several sets of data points.

[0089] In this embodiment, in the data preprocessing stage, the efficiency and accuracy of subsequent anomaly identification are significantly improved through the data segmentation strategy. First, by parsing the data stream, the timestamp and data type of each data point are accurately identified. Subsequently, the data stream is flexibly divided into a set of data points in multiple time periods by combining a preset time window with a timestamp, ensuring the timeliness of data processing. Finally, the data points are clustered according to the data type to form a set of data points with similar data types, which facilitates the risk assessment model to perform targeted analysis.

[0090] Suppose there is an order data stream of an e-commerce platform, which records the user's order behavior in real time, including order ID, user ID, product name, order timestamp, etc. When parsing the data stream, the orders are assigned to the corresponding time period according to the preset time window (such as every hour) and the order timestamp. For example, if the order timestamp of an order shows that it was placed at 10:30 am, then the order will be included in the data point set of the time period "10:00-10:59".

[0091] In data anomaly identification, clustering data points according to data types can significantly improve the accuracy of analysis. Taking bank transaction data as an example, transaction records contain multiple data types, such as transaction amount, transaction time, transaction type (transfer, deposit, withdrawal, etc.). Through clustering algorithms, banks can divide transaction data points into multiple sets according to transaction amount, transaction time, and transaction type. For example, all data points of transfer transaction types are aggregated into one set, while deposit and withdrawal transaction types form their own sets.

[0092] In the above embodiment, through time window segmentation and data type clustering, complex data streams are effectively converted into an ordered and clearly classified set of data points, which not only simplifies the data processing process, but also enhances the accuracy of anomaly detection. The time window ensures the timeliness of data processing, and data type clustering improves the targetedness of data analysis.

[0093] Furthermore, before each set of data points is sequentially input into the preset risk assessment model, it also includes:

[0094] Count the number of data point sets to obtain the number of data point sets;

[0095] An isolated tree construction algorithm is used to generate a number of isolated trees matching the number of data points in the set by recursive partitioning to obtain an isolated tree set.

[0096] The weight parameter of each isolated tree is calculated, and a risk assessment model is constructed based on the weight parameter of each isolated tree and the isolated tree set.

[0097] In this embodiment, first, the number of data point sets is counted to ensure the accurate matching of model construction, that is, the number of isolated trees corresponds to the data point set one by one. Subsequently, an isolated tree construction algorithm is used to recursively partition the data space to generate an isolated tree set, so that the advantages of the isolation forest algorithm can be used to effectively identify abnormal points in the data. Finally, by calculating the weight parameters of each isolated tree and integrating them into the risk assessment model, not only the sensitivity of the model to abnormal data is improved, but also the robustness and adaptability of the model are enhanced.

[0098] The Isolation Tree (iTree) algorithm is the core component of the Isolation Forest (iForest) algorithm. The algorithm recursively partitions the data set into smaller subsets by randomly selecting data features and their partition values ​​until the termination condition is met (such as the tree reaches a preset height, there is only one sample on a node, or the samples on the node have the same features). The Isolation Tree algorithm aims to build a tree structure in which abnormal data points are isolated earlier due to large feature differences, thus having a shorter path length in the tree.

[0099] In this embodiment, in order to optimize model performance or handle specific scenarios, different importance or "implicit weights" need to be assigned to different isolated trees. When building isolated trees, the sampling strategy or feature selection strategy can be adjusted so that some trees pay more attention to specific aspects or regions of the data during training. For example, by changing the number of samples or feature subsets used when training each tree, the contribution of each tree to anomaly detection can be indirectly affected.

[0100] Suppose there is a financial transaction dataset that contains a large number of normal transactions and a small number of fraudulent transactions. We detect these fraudulent transactions by constructing an isolation forest model (containing multiple isolated trees). The initial weight of each isolated tree in the isolation forest model can be set to "0.5". Since fraudulent transactions often have characteristics that are significantly different from normal transactions, it is necessary to ensure that the isolation forest model can capture these differences.

[0101] Financial transaction data is usually unbalanced, that is, the number of normal transactions far exceeds that of fraudulent transactions. In order to improve the model's ability to detect fraudulent transactions, oversampling or undersampling strategies can be used when constructing isolated trees. For example, fraudulent transactions can be oversampled, that is, fraudulent transaction samples can be copied multiple times to increase their proportion in the training set. Through the oversampling operation, some isolated trees will pay more attention to the characteristics of fraudulent transactions during the training process. After training is completed, these isolated trees that can learn more fraudulent transaction features are identified, and the weights of these isolated trees are increased according to the number or proportion of learned fraudulent transaction features. For example, every time an isolated tree learns a fraudulent transaction feature, its initial weight "0.5" is increased by "0.1".

[0102] In the above embodiment, the efficiency of the risk assessment model is ensured by accurately matching the data point set with the number of isolated trees. The isolated tree set is constructed by recursive segmentation, and the model is optimized using weight parameters, which improves the accuracy and robustness of anomaly detection.

[0103] Further, determining an isolation tree matching each set of data points, and importing each set of data points into the matching isolation tree, including:

[0104] Identify the data point structure for each set of data points;

[0105] Search the isolated tree structure that matches the data structure of each data point set in the isolated tree set to obtain the matching isolated tree of each data point set;

[0106] According to the tree structure of the matching isolation tree, the data points in each data point set are mapped to the matching isolation tree according to the data point structure.

[0107] In this embodiment, the data point set is structurally matched with the corresponding isolated tree, and the data points are effectively imported into the corresponding isolated tree. First, by identifying the data point structure of the data point set, the accuracy of subsequent matching is ensured. Then, the isolated tree matching the data structure of each data point set is searched in the isolated tree set, and the diversity of the isolated tree structure is utilized to achieve customized matching between the data point and the isolated tree. Finally, according to the tree structure of the matching isolated tree, the data points in the data point set are mapped to the tree, ensuring the effective representation of the data in the isolated tree, which not only improves the efficiency of data processing, but also enhances the adaptability of the model to complex data structures.

[0108] In the process of identifying the data point structure of each data point set, it is first necessary to analyze each data point in the data point set to determine its data type (such as integer, floating point number, string, etc.) and structured information (such as timestamp, geographic location, etc.). For example, if the data point set contains transaction records with timestamps, the time dimension of the data can be determined by identifying the timestamp field in each record (such as "2024-07-28T12:34:56"). At the same time, other fields (such as transaction amount, transaction type) are analyzed to determine the specific structure and meaning of the data.

[0109] When searching for an isolated tree that matches the data structure of each data point set in the isolated tree set, the characteristics of the data points in the data point set can be analyzed first, such as data type, timestamp format, number of fields, etc. Then, based on these characteristics, the isolated tree set is traversed to find isolated trees that are designed with the same or similar data structure in mind. For example, if the data point set contains transaction records with timestamps, isolated trees whose branch nodes are split according to timestamps can be found, because these isolated trees are more likely to effectively handle data point sets of this type.

[0110] In the above embodiment, by accurately matching the data point set with the isolated tree structure, efficient classification and processing of data is achieved, which not only improves the accuracy of data processing, but also enhances the adaptability of the model to the data structure. Data points are mapped to matching isolated trees according to the structure, which optimizes resource allocation and improves the efficiency and effect of anomaly detection and risk assessment.

[0111] Further, according to the tree structure of the matching isolation tree, the data points in each data point set are mapped to the matching isolation tree according to the data point structure, including:

[0112] Identify tree nodes, leaf nodes, and edges in matching isolation trees;

[0113] Identify a data point matching the tree node in the data point set, obtain a first data point, and map a content field of the first data point to the tree node;

[0114] Identify a data point matching the leaf node in the data point set, obtain a second data point, and map a content field of the second data point to the leaf node;

[0115] Identify the association relationship between the first data point and the second data point, obtain a first association relationship, and map the first association relationship to an edge between a tree node and a leaf node;

[0116] Identify the association relationship between the two second data points, obtain the second association relationship, and map the second association relationship to the edge between the two leaf nodes.

[0117] In this embodiment, the content fields of the data points in the data point set are deeply associated with the nodes, leaf nodes and edges in the matching isolated tree through the field mapping strategy. First, the structure of the isolated tree is parsed to clarify the roles and positions of the tree nodes, leaf nodes and edges. Then, according to the characteristics of the data points, the data points that match the tree nodes and leaf nodes are identified in the data point set, and the content fields of these data points are directly mapped to the corresponding tree structure, which not only ensures the accuracy of data transmission, but also retains the intrinsic connection between the data.

[0118] By analyzing the association between data points and mapping them to the edges of isolated trees, the information expression capability of isolated trees is enriched, and isolated trees can more comprehensively reflect the true face of data. By identifying and mapping the association between the first data point and the second data point (i.e., the first association), and the association between the two second data points (i.e., the second association), a comprehensive isolation forest model containing both data point information and the relationship between data points is constructed, and the model contains multiple isolated trees.

[0119] In the above embodiment, by mapping the content fields and association relationships of data points to isolated tree nodes and edges, not only the content of the data points is retained, but also the associations between data are integrated, thereby enhancing the data representation capabilities of the isolated tree, enabling the model to more comprehensively reflect the data structure and relationships, and improving the accuracy and efficiency of subsequent anomaly detection and risk assessment.

[0120] Furthermore, the path length of each isolated tree is calculated, and the isolation score of each isolated tree is generated based on the path length of each isolated tree, including:

[0121] For each isolated tree in the isolated tree set, traverse the entire isolated tree starting from the tree node of each isolated tree until the last leaf node;

[0122] Count the sum of the edges from the tree node to the last leaf node of each isolated tree to get the path length of each isolated tree;

[0123] The number of branches of each isolated tree is identified, and the isolation score of each isolated tree is calculated based on the path length of each isolated tree and the number of branches of each isolated tree.

[0124] In this embodiment, by calculating the path length and the number of branches of the isolated tree, an isolation scoring mechanism is introduced. The statistics of the path length not only reflect the depth distribution of the data point in the tree, but also reveal the complexity of the data structure. A shorter path length indicates that the data point is more likely to be isolated, while a longer path length indicates that the data structure is more complex and is more likely to be an abnormal data point. The isolation score is calculated by combining the path length and the number of branches. The isolation score can quantify the uniqueness and complexity of each isolated tree in representing the data. By identifying isolated trees with abnormal scores, potential data anomalies or complex patterns can be quickly located, enhancing the flexibility and accuracy of data processing.

[0125] The isolation score is a metric used to quantify the complexity of an isolation tree and the density of data point distribution. It is calculated based on the path length and number of branches of the isolation tree. A higher score usually indicates a more complex isolation tree, a more uneven distribution of data points in the tree, and may contain more anomalies or complex patterns. This scoring mechanism helps identify important data structure features in data analysis and anomaly detection.

[0126] In a specific embodiment of the present application, the isolation score can be calculated by the following formula:

[0127]

[0128] In the formula, G is the isolation score, T is the number of branches of the isolated tree, PL i is the path length of the i-th branch.

[0129] In the above embodiment, the isolation score is generated by calculating the path length and the number of branches of the isolated tree, which effectively quantifies the complexity and data distribution characteristics of the isolated tree. The scoring mechanism provides an accurate basis for anomaly detection, enhances the model's ability to recognize data complexity, and improves the accuracy and efficiency of risk assessment.

[0130] Furthermore, before determining the abnormal isolated tree based on the isolation score of each isolated tree and the preset abnormal score threshold, and obtaining the data point set corresponding to the abnormal isolated tree as the abnormal data point set, the method further includes:

[0131] Based on the isolation score of each isolated tree in the isolated tree set, construct an isolation score distribution;

[0132] Calculate the mean and standard deviation of the isolation score distribution;

[0133] Set anomaly score thresholds based on the mean and standard deviation of the isolation score distribution.

[0134] In this embodiment, before identifying abnormal isolated trees, a statistical analysis is introduced to set a reasonable abnormal score threshold. By constructing the distribution of isolation scores and calculating their mean (μ) and standard deviation (σ), the discrete degree and central tendency of isolated tree scores can be evaluated more scientifically, and the abnormal score threshold can be dynamically adjusted. For example, the abnormal score threshold can be set according to the following formula:

[0135] L=μ-k*σ

[0136] Where L is the anomaly score threshold, μ is the mean of the isolation score distribution, σ is the standard deviation of the isolation score distribution, and k is an adjustment parameter, which is usually a constant. The anomaly score threshold set based on these statistics can more accurately identify isolated trees that deviate from the normal distribution range as anomalies.

[0137] In the above embodiment, by statistically analyzing the isolation score, constructing the score distribution and calculating the mean and standard deviation, and dynamically setting the abnormal threshold, the accuracy of anomaly detection is significantly improved, the subjective judgment error is reduced, and the objectivity and accuracy of abnormal isolation tree identification are ensured, thereby effectively screening out abnormal data points.

[0138] Further, based on the isolation score of each isolated tree and a preset abnormal score threshold, an abnormal isolated tree is determined, including:

[0139] Determine in turn whether the isolation score of each isolated tree is greater than the abnormal score threshold;

[0140] If the isolation score of an isolated tree is greater than the abnormal score threshold, the isolated tree with the isolation score greater than the abnormal score threshold is determined as an abnormal isolated tree.

[0141] In this embodiment, accurate identification of abnormal isolated trees is achieved by comparing the isolation score of the isolated tree with the preset abnormal score threshold. When the isolation score of the isolated tree is greater than the abnormal score threshold, the isolated tree with the isolation score greater than the abnormal score threshold is determined as an abnormal isolated tree; when the isolation score of the isolated tree is less than or equal to the abnormal score threshold, the isolated tree with the isolation score less than or equal to the abnormal score threshold is determined as a normal isolated tree.

[0142] In the above embodiment, by comparing the isolation score with the threshold, abnormal isolation trees are accurately identified, normal data is effectively filtered, the pertinence and accuracy of anomaly detection are improved, and manual intervention is reduced through automated processes, improving processing efficiency and ensuring the timeliness and reliability of data quality monitoring.

[0143] In this embodiment, the data anomaly identification method is executed on the electronic device (eg Figure 1 The server shown) can be connected by wire or by wireless connection. It should be noted that the wireless connection method may include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultrawideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0144] It should be emphasized that in order to further ensure the privacy and security of the above data stream, the above data stream can also be stored in a blockchain node.

[0145] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0146] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0147] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0148] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a data anomaly identification device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0149] like Figure 3 As shown, the data anomaly identification device 300 described in this embodiment includes:

[0150] The data stream acquisition module 301 is used to receive an abnormality identification instruction and acquire a data stream to be processed from a preset multi-source database;

[0151] A data segmentation module 302 is used to segment the data stream to be processed to obtain a plurality of data point sets;

[0152] A data input module 303 is used to input each data point set into a preset risk assessment model in sequence, wherein the risk assessment model includes a plurality of isolated trees, and the number of isolated trees is equal to the number of data point sets;

[0153] A data point import module 304 is used to determine an isolated tree matching each data point set and import each data point set into the matching isolated tree;

[0154] An isolation score module 305, for calculating the path length of each isolated tree, and generating an isolation score of each isolated tree based on the path length of each isolated tree;

[0155] The anomaly identification module 306 is used to determine the abnormal isolated tree based on the isolation score of each isolated tree and a preset anomaly score threshold, and obtain the data point set corresponding to the abnormal isolated tree as the abnormal data point set.

[0156] Furthermore, the data segmentation module 302 is specifically used for:

[0157] The data stream to be processed is parsed to identify the timestamp and data type of each data point in the data stream to be processed; the data stream to be processed is segmented based on the preset time window and timestamp to obtain all data points of the data stream to be processed; all data points of the data stream to be processed are clustered based on the data type to obtain several data point sets.

[0158] Furthermore, the data anomaly identification device 300 is also used for:

[0159] The number of data point sets is counted to obtain the number of data point sets; an isolated tree construction algorithm is used to generate a number of isolated trees matching the number of data point sets by recursive partitioning to obtain an isolated tree set; a weight parameter of each isolated tree is calculated, and a risk assessment model is constructed based on the weight parameter of each isolated tree and the isolated tree set.

[0160] Furthermore, the data point import module 304 is specifically used for:

[0161] Identify the data point structure of each data point set; search for the isolated tree structure that matches the data structure of each data point set in the isolated tree set to obtain the matching isolated tree of each data point set; and map the data points in each data point set to the matching isolated tree according to the data point structure based on the tree structure of the matching isolated tree.

[0162] Furthermore, the data point import module 304 is also used for:

[0163] Identify the tree nodes, leaf nodes and edges in the matching isolated tree; identify the data points matching the tree nodes in the data point set, obtain the first data points, and map the content field of the first data points to the tree nodes; identify the data points matching the leaf nodes in the data point set, obtain the second data points, and map the content field of the second data points to the leaf nodes; identify the association relationship between the first data point and the second data point, obtain the first association relationship, and map the first association relationship to the edge between the tree node and the leaf node; identify the association relationship between the two second data points, obtain the second association relationship, and map the second association relationship to the edge between the two leaf nodes.

[0164] Furthermore, the isolation scoring module 305 is specifically used for:

[0165] For each isolated tree in the isolated tree set, traverse the entire isolated tree from the tree node of each isolated tree to the last leaf node; count the sum of the edges from the tree node of each isolated tree to the last leaf node to obtain the path length of each isolated tree; identify the number of branches of each isolated tree, and calculate the isolation score of each isolated tree based on the path length of each isolated tree and the number of branches of each isolated tree.

[0166] Furthermore, the data anomaly identification device 300 is also used for:

[0167] Based on the isolation score of each isolated tree in the isolated tree set, an isolation score distribution is constructed; the mean and standard deviation of the isolation score distribution are calculated; and an abnormal score threshold is set based on the mean and standard deviation of the isolation score distribution.

[0168] Furthermore, the abnormality identification module 306 is specifically used to:

[0169] Determine in turn whether the isolation score of each isolated tree is greater than the abnormal score threshold;

[0170] If the isolation score of an isolated tree is greater than the abnormal score threshold, the isolated tree with the isolation score greater than the abnormal score threshold is determined as an abnormal isolated tree.

[0171] In the above embodiment, the present application discloses a data anomaly identification device, which belongs to the field of big data technology. First, the data to be processed is extracted and decomposed into multiple data point sets through data segmentation technology. Subsequently, a risk assessment model with multiple isolated trees is used, which is composed of isolated trees with the same number as the data point set, and each tree is allocated to specifically process a data point set. By calculating the path length of the data points in each isolated tree, its isolation degree is evaluated and a corresponding score is generated. Finally, according to the preset abnormal score threshold, the isolated tree with abnormally high isolation degree is automatically identified, and its corresponding data point set is determined to be an abnormal data point set. The present application realizes efficient anomaly identification of complex data streams through precise data segmentation and matching isolated tree processing, and uses the risk assessment model with multiple isolated trees to effectively reduce the computational complexity and improve the accuracy of anomaly detection. By setting the abnormal score threshold, the abnormal data point set can be automatically screened out to ensure data quality and security.

[0172] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0173] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.

[0174] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0175] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the data anomaly identification method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0176] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the data anomaly identification method.

[0177] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0178] In the above embodiment, the present application discloses a computer device, which belongs to the field of big data technology. First, the data to be processed is extracted and decomposed into multiple data point sets through data segmentation technology. Subsequently, a risk assessment model with multiple isolated trees is used, which is composed of isolated trees with the same number as the data point set, and each tree is allocated to specifically process a data point set. By calculating the path length of the data points in each isolated tree, its isolation degree is evaluated and a corresponding score is generated. Finally, based on the preset abnormal score threshold, the isolated tree with abnormally high isolation degree is automatically identified, and its corresponding data point set is determined to be an abnormal data point set. The present application realizes efficient anomaly identification of complex data streams through precise data segmentation and matching isolated tree processing, and uses the risk assessment model with multiple isolated trees to effectively reduce the computational complexity and improve the accuracy of anomaly detection. By setting the abnormal score threshold, the abnormal data point set can be automatically screened out to ensure data quality and security.

[0179] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data anomaly identification method as described above.

[0180] In the above embodiment, the present application discloses a computer-readable storage medium, which belongs to the field of big data technology. First, the data to be processed is extracted and decomposed into multiple data point sets through data segmentation technology. Subsequently, a risk assessment model with multiple isolated trees is used, which is composed of isolated trees with the same number as the data point set, and each tree is allocated to specifically process a data point set. By calculating the path length of the data points in each isolated tree, its isolation degree is evaluated and a corresponding score is generated. Finally, according to the preset abnormal score threshold, the isolated tree with abnormally high isolation degree is automatically identified, and its corresponding data point set is determined to be an abnormal data point set. The present application realizes efficient anomaly identification of complex data streams through precise data segmentation and matching isolated tree processing, and uses the risk assessment model with multiple isolated trees to effectively reduce the computational complexity and improve the accuracy of anomaly detection. By setting the abnormal score threshold, the abnormal data point set can be automatically screened out to ensure data quality and security.

[0181] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0182] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0183] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for identifying data anomalies, characterized in that: include: Receive anomaly identification instructions and obtain the data stream to be processed from the preset multi-source database; Performing data segmentation on the data stream to be processed to obtain a plurality of data point sets; Inputting each of the data point sets into a preset risk assessment model in sequence, wherein the risk assessment model includes a plurality of isolated trees, and the number of the isolated trees is equal to the number of the data point sets; Determine an isolated tree matching each of the data point sets, and import each of the data point sets into the matching isolated tree; Calculating the path length of each of the isolated trees, and generating an isolation score of each of the isolated trees based on the path length of each of the isolated trees; Based on the isolation score of each of the isolated trees and a preset abnormal score threshold, an abnormal isolated tree is determined, and a data point set corresponding to the abnormal isolated tree is obtained as an abnormal data point set.

2. The data anomaly identification method according to claim 1, characterized in that: The data stream to be processed is segmented to obtain a plurality of data point sets, including: Parsing the data stream to be processed to identify the timestamp and data type of each data point in the data stream to be processed; Segment the data stream to be processed based on a preset time window and the timestamp to obtain all data points of the data stream to be processed; All data points of the to-be-processed data stream are clustered based on the data type to obtain a plurality of data point sets.

3. The data anomaly identification method according to claim 1, characterized in that: Before inputting each of the data point sets into the preset risk assessment model in sequence, the method further includes: Counting the number of the data point sets to obtain the number of data point sets; An isolated tree construction algorithm is used to generate a number of isolated trees matching the number of the data point set by recursive partitioning to obtain an isolated tree set; The weight parameter of each isolated tree is calculated, and the risk assessment model is constructed based on the weight parameter of each isolated tree and the isolated tree set.

4. The data anomaly identification method according to claim 3, characterized in that: The determining an isolated tree matching each of the data point sets and importing each of the data point sets into the matching isolated tree comprises: identifying a data point structure for each of the sets of data points; Searching for an isolated tree structure matching the data structure of each of the data point sets in the isolated tree set, respectively, to obtain a matching isolated tree for each of the data point sets; According to the tree structure of the matching isolated tree, the data points in each data point set are mapped to the matching isolated tree according to the data point structure.

5. The data anomaly identification method according to claim 4, characterized in that: The step of mapping the data points in each data point set to the matching isolation tree according to the tree structure of the matching isolation tree comprises: identifying tree nodes, leaf nodes, and edges in the matching isolation tree; Identify a data point matching the tree node in the data point set to obtain a first data point, and map a content field of the first data point to the tree node; Identify a data point matching the leaf node in the data point set to obtain a second data point, and map a content field of the second data point to the leaf node; Identify an association relationship between the first data point and the second data point, obtain a first association relationship, and map the first association relationship to an edge between the tree node and the leaf node; Identify the association relationship between the two second data points, obtain a second association relationship, and map the second association relationship to the edge between the two leaf nodes.

6. The data anomaly identification method according to claim 5, characterized in that: The step of calculating the path length of each of the isolated trees and generating an isolation score of each of the isolated trees based on the path length of each of the isolated trees comprises: For each isolated tree in the isolated tree set, traverse the entire isolated tree starting from the tree node of each isolated tree until the last leaf node; Counting the sum of the edges from the tree node to the last leaf node of each isolated tree to obtain the path length of each isolated tree; The number of branches of each of the isolated trees is identified, and the isolation score of each of the isolated trees is calculated based on the path length of each of the isolated trees and the number of branches of each of the isolated trees.

7. The data anomaly identification method according to claim 3, characterized in that: Before determining the abnormal isolated tree based on the isolation score of each of the isolated trees and the preset abnormal score threshold, and obtaining the data point set corresponding to the abnormal isolated tree as the abnormal data point set, the method further includes: Constructing an isolation score distribution based on the isolation score of each of the isolated trees in the isolated tree set; Calculating a mean and a standard deviation of the isolation score distribution; Setting the anomaly score threshold based on the mean and standard deviation of the isolation score distribution; The step of determining an abnormal isolated tree based on the isolation score of each of the isolated trees and a preset abnormal score threshold comprises: Determine in turn whether the isolation score of each of the isolated trees is greater than the abnormal score threshold; If the isolation score of the isolated tree is greater than the abnormal score threshold, the isolated tree with the isolation score greater than the abnormal score threshold is determined as an abnormal isolated tree.

8. A data anomaly identification device, characterized in that: include: A data stream acquisition module is used to receive anomaly identification instructions and obtain the data stream to be processed from a preset multi-source database; A data segmentation module, used to segment the data stream to be processed to obtain a plurality of data point sets; A data input module, used to input each of the data point sets into a preset risk assessment model in sequence, wherein the risk assessment model includes a plurality of isolated trees, and the number of the isolated trees is equal to the number of the data point sets; A data point importing module, used to determine an isolated tree matching each of the data point sets, and import each of the data point sets into the matching isolated tree; An isolation score module, used for calculating the path length of each of the isolated trees, and generating an isolation score of each of the isolated trees based on the path length of each of the isolated trees; The anomaly identification module is used to determine the abnormal isolated tree based on the isolation score of each of the isolated trees and a preset anomaly score threshold, and obtain the data point set corresponding to the abnormal isolated tree as the abnormal data point set.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data anomaly identification method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data anomaly identification method according to any one of claims 1 to 7 are implemented.

Images