Associated abnormal data set identification method and device based on big data and electronic equipment

Abstract

The invention discloses a big data-based associated abnormal data set identification method and device and electronic equipment, and relates to the field of data processing. According to the method, an original data set is constructed according to a target enterprise, an upstream enterprise and a downstream enterprise; the preprocessed structured data and the converted unstructured data are merged, and a target data set is obtained; based on the target data set, constructing a data association graph, nodes in the data association graph representing data elements in the target data set, and edges in the data association graph representing association relationships among the data elements; in the data association graph, identifying an association density abnormal region; marking data elements contained in the association density abnormal region as abnormal data; and determining a corresponding abnormal data set according to the position information of the abnormal data in the original data set. By implementing the technical scheme provided by the invention, the accuracy of abnormal data monitoring is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to a method, device and electronic device for identifying associated abnormal data sets based on big data. Background Art

[0002] With the rapid development of information technology, enterprises and organizations have accumulated a large amount of data resources. These data contain rich information, but also bring about problems of data management and security. How to effectively manage and use these data and ensure the security and accuracy of the data has become an important research topic. Existing data anomaly detection technology has made some progress, but it still faces many challenges in the face of large-scale, multi-source heterogeneous data.

[0003] At present, existing data anomaly detection technologies are often based on a single data type or a simple statistical model, which is difficult to adapt to the actual needs of diversified data types and complex relationships. This limitation leads to the inability to accurately identify real abnormal data in practical applications, especially when facing complex upstream and downstream relationships between enterprises, thus affecting the accuracy of abnormal data detection.

[0004] Therefore, there is an urgent need for methods, devices and electronic devices for identifying associated abnormal data sets based on big data. Summary of the invention

[0005] The present application provides a method, device and electronic device for identifying associated abnormal data sets based on big data, which improves the accuracy of abnormal data monitoring.

[0006] In a first aspect of the present application, a method for identifying an associated abnormal data set based on big data is provided, the method comprising: acquiring a target enterprise and upstream enterprises and downstream enterprises associated with the target enterprise; constructing an original data set according to the target enterprise, the upstream enterprise and the downstream enterprise, the original data set comprising structured data and unstructured data; preprocessing the structured data, performing triple conversion on the unstructured data, and merging the preprocessed structured data and the converted unstructured data to obtain a target data set; constructing a data association graph based on the target data set, the nodes in the data association graph represent the data elements in the target data set, the edges in the data association graph represent the association relationship between each of the data elements, and the association relationship is calculated by the similarity between the data elements; in the data association graph, identifying an abnormal association density area, the abnormal association density area is an area with abnormal association density, and the association density refers to the number of edges in a unit area; marking the data elements contained in the abnormal association density area as abnormal data; and determining the corresponding abnormal data set according to the position information of the abnormal data in the original data set.

[0007] By adopting the above technical solution, by acquiring the data of the target enterprise and its upstream and downstream enterprises, an original data set containing structured data and unstructured data is constructed, and the original data set is preprocessed and transformed to obtain the target data set. A data association graph is constructed based on the target data set, in which nodes represent data elements and edges represent the association relationship between data elements. In the data association graph, an abnormal association density area is identified, and the data elements in the abnormal association density area are marked as abnormal data, and the corresponding abnormal data set is determined according to the position information of the abnormal data in the original data set. The method makes full use of the structured and unstructured data of the target enterprise and its upstream and downstream enterprises, and uniformly represents heterogeneous data through data preprocessing and transformation, so as to construct a comprehensive and high-quality target data set. A graph model is used to represent the association relationship between data elements, and the association strength is quantified by similarity, so as to characterize the complex association between data elements. By identifying the abnormal association density area based on the graph model, abnormal data sets can be automatically and efficiently discovered. By mapping the abnormal data back to the original data set, the source and context of the abnormal data set can be determined, which is convenient for subsequent analysis and disposal. Therefore, the method can make full use of big data, automatically, accurately and efficiently identify the associated abnormal data set, and provide data support for applications such as enterprise risk prevention and public opinion monitoring.

[0008] Optionally, in the data association graph, identifying an abnormal association density area specifically includes: gridding the data association graph to divide the data association graph into multiple grid units of the same size, each of the grid units including at least one node; calculating the association density of each of the grid units, the association density being the actual number of edges in the grid unit divided by a preset number, the preset number being the maximum number of edges that may exist in the grid unit; calculating the domain association density of a first grid unit based on the association density, the domain association density being the average value of the association density of a second grid unit adjacent to the first grid unit, the first grid unit being any one of the multiple grid units; calculating the absolute value of the difference between the association density of the first grid unit and the domain association density of the first grid unit, and using the absolute value as the density difference of the first grid unit; determining a first grid unit whose density difference is greater than or equal to a preset difference threshold as an abnormal grid unit, and merging adjacent abnormal grid units into the abnormal association density area.

[0009] By adopting the above technical solution, the data association graph is gridded to divide the data association graph into grid units of the same size, each grid unit containing at least one node. The association density of each grid unit is calculated, that is, the actual number of edges divided by the maximum possible number of edges. The domain association density of each grid unit is calculated, that is, the average value of the association density of adjacent grid units. The absolute value of the difference between the association density of each grid unit and the domain association density is calculated as the density difference of the grid unit. The grid unit with a density difference greater than or equal to the preset difference threshold is marked as an abnormal unit, and the adjacent abnormal units are merged into an abnormal association density region. The method cleverly utilizes the gridding process to transform the irregular graph structure into a regular grid structure, which is convenient for calculating and comparing the association density. By calculating the association density and domain association density of the grid unit, the degree of association between the internal and surrounding areas of the unit can be measured. Using the density difference as an indicator, the abnormal degree of the grid unit can be quantified and the abnormal grid unit can be automatically identified. By adopting the method of regional merging, a complete and continuous abnormal region can be obtained. Therefore, the method can effectively identify abnormal association density regions and provide a reliable basis for the identification of abnormal data sets.

[0010] Optionally, constructing the original data set based on the target enterprise, the upstream enterprise and the downstream enterprise specifically includes: obtaining the structured data of the target enterprise, the upstream enterprise and the downstream enterprise, the structured data including basic enterprise information data, financial data and transaction data; obtaining the unstructured data of the target enterprise, the upstream enterprise and the downstream enterprise, the unstructured data including enterprise news data, judicial litigation data and patent data; and performing data cleaning and data integration on the structured data and the unstructured data to obtain the original data set.

[0011] By adopting the above technical solution, by obtaining the structured data and unstructured data of the target enterprise and its upstream and downstream enterprises, the two types of data are cleaned and integrated respectively to obtain the original data set. Structured data includes basic information of the enterprise, financial data, transaction data, etc., and unstructured data includes corporate news, judicial proceedings, patent data, etc. This method comprehensively collects multi-source heterogeneous data inside and outside the enterprise, covering multiple business areas and data types of the enterprise. Through data cleaning, low-quality data such as noise and errors are removed, and the accuracy and reliability of the data are improved. Through data integration, data from different sources and formats are unified and associated to form a complete and consistent original data set. This method lays a solid data foundation for the identification of associated abnormal data sets, which helps to improve the accuracy and comprehensiveness of identification.

[0012] Optionally, the step of converting the unstructured data into triples specifically includes: performing natural language processing on the unstructured data to extract key information; converting the key information into triples according to a preset template, the triples including a subject, a predicate and an object; and using the triples as data elements of the target data set.

[0013] By adopting the above technical solution, the key information is extracted by natural language processing of unstructured data, and the key information is converted into triples according to the preset template. The triples include three elements: subject, predicate and object. The triples are used as data elements of the target data set and participate in subsequent analysis together with structured data. This method cleverly uses natural language processing technology to extract structured semantic information from unstructured data and reveals the logical relationship within the text. The triple representation can cover most types of semantic relationships and has strong expression ability. Unstructured data is converted into triples so that it has the same representation as structured data, which facilitates the fusion and association analysis of the two types of data. Unstructured data represented by triples can enrich the semantic information of the target data set and improve the coverage and accuracy of the association relationship. Therefore, this method provides strong support for the fusion and analysis of unstructured data and expands the scope of application of associated abnormal data set identification.

[0014] Optionally, constructing a data association graph based on the target data set specifically includes: taking each data element in the target data set as a node of the data association graph; calculating the similarity between a first data element and a second data element, the first data element and the second data element being any two data elements among a plurality of the data elements; if it is determined that the similarity is greater than or equal to a preset similarity threshold, establishing an edge between the first data element and the second data element to obtain the data association graph.

[0015] By adopting the above technical solution, the data elements in the target data set are used as nodes of the graph, and the similarity between any two data elements is calculated. If the similarity is greater than or equal to the preset similarity threshold, an edge is established between the two data elements. This method intuitively and vividly depicts the association relationship between data elements, and objectively and accurately measures the strength of the association through the quantitative indicator of similarity. The setting of the preset similarity threshold can filter out weakly associated or unassociated data element pairs, making the association graph more concise and reliable. Using the data association graph as the basis for abnormal area identification can make full use of the association information between data elements, and depict the association pattern and abnormal pattern of the data set from both global and local perspectives. The quality and accuracy of the data association graph directly affect the effect of abnormal area identification. This method provides key data structure and algorithm support for abnormal area identification, which helps to improve the effect and efficiency of identification.

[0016] Optionally, calculating the similarity between the first data element and the second data element specifically includes: extracting a first eigenvector of the first data element, and extracting a second eigenvector of the second data element; calculating the cosine similarity between the first eigenvector and the second eigenvector, and using the cosine similarity as the similarity between the first data element and the second data element.

[0017] By adopting the above technical solution, by extracting the feature vectors of any two data elements, the cosine similarity of the two feature vectors is calculated as the similarity of the two data elements. This method cleverly utilizes the vector space model to map the data elements to the high-dimensional feature space and characterizes the characteristics of the data elements through the feature vectors. Using cosine similarity as a metric for the similarity of data elements can overcome the influence of differences in data types, numerical ranges, etc., and provide a unified and reliable similarity evaluation. The similarity calculation method based on feature vectors and cosine similarity provides a reasonable and effective similarity measurement for the construction of data association graphs, which helps to improve the accuracy and interpretability of association relationships.

[0018] Optionally, determining the corresponding abnormal data set according to the position information of the abnormal data in the original data set specifically includes: obtaining the position information of the abnormal data in the target data set, the position information including the row number and column number of the abnormal data in the target data set; obtaining the position information of the abnormal data in the structured data and the unstructured data according to the position information of the abnormal data in the target data set; determining the target structured data associated with the abnormal data according to the position information of the abnormal data in the structured data, and determining the target unstructured data associated with the abnormal data according to the position information of the abnormal data in the unstructured data; merging the target structured data and the target unstructured data to obtain a subset of the original data; and using the subset of the original data as the abnormal data set corresponding to the abnormal data.

[0019] By adopting the above technical solution, the position information of the abnormal data in the target data set, including the row number and the column number, is obtained, and then the position of the abnormal data in the structured data and the unstructured data is obtained according to the position information. According to the position of the abnormal data in the structured data and the unstructured data, the target structured data and the target unstructured data associated with the abnormal data are determined, and they are merged to obtain the original data subset as the abnormal data set corresponding to the abnormal data. This method makes full use of the position correspondence relationship of the abnormal data in different data sets. Through position mapping, the abnormal data is restored from abstract data elements to specific structured records and unstructured texts. By extracting the associated data of the abnormal data, an abnormal data set is formed. By merging structured data and unstructured data, the abnormal situation can be analyzed from multiple dimensions and multiple levels, and the associated characteristics and evolution laws of the abnormality can be mined. The method of locating and extracting the abnormal data set based on the original data set provides rich and reliable data support for the interpretation and disposal of the abnormality, which is helpful to analyze the cause of the abnormality, evaluate the impact of the abnormality, and formulate response strategies.

[0020] In a second aspect of the present application, a device for identifying associated abnormal data sets based on big data is provided, characterized in that the device includes an acquisition module and a processing module, wherein: the acquisition module is used to acquire a target enterprise and upstream enterprises and downstream enterprises associated with the target enterprise; the processing module is used to construct an original data set based on the target enterprise, the upstream enterprise and the downstream enterprise, and the original data set includes structured data and unstructured data; the processing module is also used to preprocess the structured data, perform triple conversion on the unstructured data, and merge the preprocessed structured data and the converted unstructured data to obtain the target data set; the processing module is also used to preprocess the structured data based on the target enterprise, the upstream enterprise and the downstream enterprise, and the original data set includes structured data and unstructured data. The target data set is used to construct a data association graph, wherein the nodes in the data association graph represent the data elements in the target data set, and the edges in the data association graph represent the association relationship between the data elements, and the association relationship is obtained by calculating the similarity between the data elements; the processing module is also used to identify an abnormal association density area in the data association graph, wherein the abnormal association density area is an area with abnormal association density, and the association density refers to the number of edges in a unit area; the processing module is also used to mark the data elements contained in the abnormal association density area as abnormal data; the processing module is also used to determine the corresponding abnormal data set according to the position information of the abnormal data in the original data set.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By acquiring the data of the target enterprise and its upstream and downstream enterprises, an original data set containing structured data and unstructured data is constructed, and the original data set is preprocessed and transformed to obtain the target data set. A data association graph is constructed based on the target data set, where nodes represent data elements and edges represent the association relationship between data elements. In the data association graph, the abnormal association density area is identified, and the data elements in the abnormal association density area are marked as abnormal data. According to the location information of the abnormal data in the original data set, the corresponding abnormal data set is determined. This method makes full use of the structured and unstructured data of the target enterprise and its upstream and downstream enterprises, and through data preprocessing and transformation, the heterogeneous data is uniformly represented to construct a comprehensive and high-quality target data set. The graph model is used to represent the association relationship between data elements, and the association strength is quantified by similarity, so as to characterize the complex association between data elements. Based on the graph model, the abnormal association density area can be identified, and the abnormal data set can be automatically and efficiently discovered. By mapping the abnormal data back to the original data set, the source and context of the abnormal data set can be determined, which is convenient for subsequent analysis and disposal. Therefore, this method can make full use of big data to automatically, accurately and efficiently identify the associated abnormal data set, and provide data support for applications such as enterprise risk prevention and public opinion monitoring.

[0024] 2. By gridding the data association graph, the data association graph is divided into grid units of the same size, each grid unit contains at least one node. The association density of each grid unit is calculated, that is, the actual number of edges divided by the maximum possible number of edges. The domain association density of each grid unit is calculated, that is, the average association density of adjacent grid units. The absolute value of the difference between the association density of each grid unit and the domain association density is calculated as the density difference of the grid unit. The grid units with a density difference greater than or equal to the preset difference threshold are marked as abnormal units, and the adjacent abnormal units are merged into abnormal association density regions. This method cleverly utilizes gridding to transform irregular graph structures into regular grid structures, which is convenient for calculating and comparing association density. By calculating the association density and domain association density of grid units, the degree of association between the internal and surrounding areas of the unit can be measured. Using the density difference indicator, the degree of abnormality of the grid unit can be quantified and the abnormal grid unit can be automatically identified. By using the region merging method, a complete and continuous abnormal region can be obtained. Therefore, this method can effectively identify abnormal association density regions and provide a reliable basis for the identification of abnormal data sets.

[0025] 3. By obtaining the structured data and unstructured data of the target enterprise and its upstream and downstream enterprises, the two types of data are cleaned and integrated respectively to obtain the original data set. Structured data includes basic information of the enterprise, financial data, transaction data, etc., and unstructured data includes corporate news, judicial proceedings, patent data, etc. This method comprehensively collects multi-source heterogeneous data inside and outside the enterprise, covering multiple business areas and data types of the enterprise. Through data cleaning, low-quality data such as noise and errors are removed, and the accuracy and reliability of the data are improved. Through data integration, data from different sources and formats are unified and associated to form a complete and consistent original data set. This method lays a solid data foundation for the identification of associated abnormal data sets, which helps to improve the accuracy and comprehensiveness of identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of a method for identifying an associated abnormal data set based on big data disclosed in an embodiment of the present application; Figure 2 It is a module schematic diagram of a device for identifying associated abnormal data sets based on big data disclosed in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0027] Explanation of the reference numerals: 201, acquisition module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0028] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0029] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0030] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0031] This application provides a method for identifying abnormal data sets based on big data. Figure 1 , Figure 1 1 is a flow chart of a method for identifying an associated abnormal data set based on big data provided by an embodiment of the present application. The method is applied to a server, which is a server that executes an associated abnormal data set identification program based on big data. The server can be a single server, or a server cluster composed of multiple servers, or a cloud computing service center. The method includes steps S101 to S107, and the steps are as follows: Step S101: Acquire a target enterprise and upstream enterprises and downstream enterprises associated with the target enterprise.

[0032] In step S101, the server can connect to the database of each enterprise and directly extract the information of the target enterprise, upstream enterprises and downstream enterprises from the database. The database of the enterprise usually contains the basic information of the enterprise, supplier information, customer information, etc. The server can determine the target enterprise and its related enterprises (upstream enterprises and downstream enterprises) based on this information.

[0033] Step S102: construct an original data set according to the target enterprise, upstream enterprises and downstream enterprises, where the original data set includes structured data and unstructured data.

[0034] In step S102, an original data set is constructed according to the target enterprise, upstream enterprises and downstream enterprises, specifically including: obtaining structured data of the target enterprise, upstream enterprises and downstream enterprises, the structured data including basic information data, financial data and transaction data of the enterprise; obtaining unstructured data of the target enterprise, upstream enterprises and downstream enterprises, the unstructured data including enterprise news data, judicial litigation data and patent data; cleaning and integrating the structured data and unstructured data to obtain the original data set.

[0035] Specifically, the server constructs an original data set according to the target enterprise, the upstream enterprise and the downstream enterprise. The original data set includes structured data and unstructured data.

[0036] For structured data, the server obtains basic information data, financial data and transaction data of the target enterprise, upstream enterprises and downstream enterprises. Basic information data of enterprises include enterprise name, registered address, registered capital, establishment time, business scope, etc. Financial data includes the balance sheet, income statement, cash flow statement, etc. of the enterprise; transaction data includes the purchase order, sales order, logistics documents, etc. of the enterprise. The server obtains these structured data through multiple channels such as connecting to the internal database of the enterprise, third-party data service providers, and public databases of government departments.

[0037] For unstructured data, the server obtains corporate news data, judicial litigation data, and patent data of target companies, upstream companies, and downstream companies. Corporate news data includes news reports and announcements related to the company; judicial litigation data includes legal litigation records in which the company participates as a plaintiff, defendant, or third party; and patent data includes patent information applied for or owned by the company. The server can obtain these unstructured data by crawling news websites, judicial case databases, patent databases, and other channels on the Internet.

[0038] After acquiring structured and unstructured data, the server cleans and integrates the data. Data cleaning refers to identifying and correcting errors, inconsistencies, and duplications in the data to improve data quality. Data integration refers to integrating data from multiple sources into a unified, coherent data view. For example, the server obtains the financial data of a company from different databases, which may have different data formats and field names. The server maps this data to a unified data model. For another example, the server crawls a large amount of corporate news from a news website and needs to associate it with existing structured data, such as matching the company name mentioned in the news with the company name in the company's basic information data, and adding the news information to the corresponding company record. Through data cleaning and integration, the server finally obtains an original data set that combines structured and unstructured data.

[0039] Step S103: preprocess the structured data, perform triple conversion on the unstructured data, and merge the preprocessed structured data and the converted unstructured data to obtain a target data set.

[0040] In step S103, the step of converting the unstructured data into triples specifically includes: performing natural language processing on the unstructured data to extract key information; converting the key information into triples according to a preset template, the triples including a subject, a predicate and an object; and using the triples as data elements of the target data set.

[0041] Specifically, for structured data, the server performs preprocessing operations such as data cleaning and format conversion. Specifically: the server first performs a data quality check on the structured data, identifies and handles data quality issues such as missing values, outliers, and duplicate values. For example, for missing values, the server can use mean filling, median filling and other methods to fill in according to the field type and distribution characteristics; for outliers, the server uses truncation, replacement, and deletion methods to process according to the field value range and distribution characteristics; for duplicate values, the server uses deduplication, merging, and retention methods to process according to the uniqueness constraint of the field. Then, the server converts the format of the structured data, converting data from different sources and formats into a unified format. For example, the server converts CSV format data into JSON format, converts date strings into standard date and time formats, and converts string type values ​​into numeric types.

[0042] The server performs triple conversion on unstructured data, converting the unstructured data into a structured triple form for subsequent data analysis and processing. First, the server performs natural language processing on the unstructured data to extract key information. Unstructured data usually exists in the form of natural language text, such as news reports, legal documents, patent specifications, etc. The server uses natural language processing technology to analyze and understand these texts. Specifically, the server first performs word segmentation and part-of-speech tagging on the text, splitting the text into individual words and tagging the part of speech of each word, such as nouns, verbs, adjectives, etc. Then, the server performs named entity recognition to identify named entities such as person names, place names, and organization names that appear in the text. Next, the server performs keyword extraction, extracting keywords in the text according to a predefined keyword dictionary or using algorithms such as TF-IDF, such as "company", "investment", "lawsuit", etc. Finally, the server performs syntactic analysis to identify the subject, predicate, and object sentence components in the text.

[0043] After extracting the key information, the server converts the key information into triples according to a preset template. The preset template defines the structure and format of the triples, usually including three parts: subject, predicate, and object. The subject represents the doer of the action, the predicate represents the action or relationship, and the object represents the recipient or object of the action. The server maps the extracted key information into the preset template to generate the corresponding triples. For example, from the news text "Company A acquired 80% of the equity of Company B", the server extracts the key information "Company A", "acquired", "Company B", "80%", "equity". According to the preset template "[Company 1] acquires [Company 2] [equity percentage] equity", the triples "<Company A, acquired, Company B>" and "<Company A, acquired equity percentage, 80%>" are generated. After generating the triples, the server uses the triples as data elements of the target data set. Each triple represents a fact or relationship in the unstructured data, and multiple triples together constitute a structured representation of the unstructured data.

[0044] After completing the preprocessing of the structured data and the triple conversion of the unstructured data, the server associates and merges the two parts of data according to the primary key or foreign key to obtain a complete and structurally unified target data set. Each record in the target data set consists of several fields or triples.

[0045] Step S104: Based on the target data set, construct a data association graph. The nodes in the data association graph represent the data elements in the target data set, and the edges in the data association graph represent the association relationships between the various data elements. The association relationships are obtained through similarity calculations between the data elements.

[0046] In step S104, based on the target data set, a data association graph is constructed, specifically including: using each data element in the target data set as a node of the data association graph; calculating the similarity between a first data element and a second data element, where the first data element and the second data element are any two data elements among multiple data elements; if it is determined that the similarity is greater than or equal to a preset similarity threshold, then an edge is established between the first data element and the second data element to obtain the data association graph.

[0047] Specifically, the server constructs a data association graph based on the target data set. The data association graph is a graph model used to represent the association relationships between data elements. In the data association graph, nodes represent data elements and edges represent the association relationships between data elements. First, the server uses each data element in the target data set as a node of the data association graph. The target data set includes structured data and unstructured data. The data elements can be field values in structured data, such as enterprise names, transaction amounts, etc.; or they can be triples obtained by converting unstructured data, such as "<Company A, investment, Company B>". The server creates a node for each data element, and the attributes of the node include the type, value, etc. of the data element. Then, the server calculates the similarity between data elements to determine whether there is an association relationship between them.

[0048] The server calculates the similarity for each pair of data elements in the target data set. If the similarity is greater than or equal to the preset similarity threshold, it is considered that there is an association relationship between these two data elements, and an edge is established between them. The setting of the preset similarity threshold needs to be determined according to specific business scenarios and data characteristics, and the optimal value can be found through experiments and tuning. This application does not limit this. Through the above steps, the server finally obtains a complete data association graph.

[0049] In a possible implementation manner, calculating the similarity between the first data element and the second data element specifically includes: extracting the first feature vector of the first data element and extracting the second feature vector of the second data element; calculating the cosine similarity between the first feature vector and the second feature vector, and using the cosine similarity as the similarity between the first data element and the second data element.

[0050] Specifically, the server extracts the feature vectors of the first data element and the second data element. A feature vector is a mathematical representation used to describe the characteristics and attributes of a data element. The method of extracting feature vectors depends on the type and structure of the data element: for text-type data elements, such as company names, news headlines, etc., the server uses a bag-of-words model or a TF-IDF method to convert the text into a numerical vector of a fixed length. For example, for a news text, the server first segments it into words to obtain a list of words; then, it counts the frequency of each word in the text to form a word frequency vector; finally, the word frequency vector is normalized to obtain the feature vector of the news text.

[0051] For data elements of numerical type, such as transaction amount, stock price, etc., the server directly uses its value as a component of the feature vector. For example, for a transaction record, the server extracts its transaction amount, transaction time, transaction object and other numerical fields to form a multi-dimensional numerical vector as the feature vector of the transaction.

[0052] After extracting the first feature vector of the first data element and the second feature vector of the second data element, the server calculates the cosine similarity between the two feature vectors. Cosine similarity can measure the cosine value of the angle between two vectors and can reflect the directional similarity of the vectors. Finally, the server uses the calculated cosine similarity as the similarity between the first data element and the second data element. The server can set a preset similarity threshold. When the similarity between two data elements is greater than or equal to the threshold, it is considered that there is an association relationship between them and an edge is established between them.

[0053] Step S105: in the data association graph, identifying an abnormal association density region, where the abnormal association density region is an area with abnormal association density, and the association density refers to the number of edges in a unit area.

[0054] In step S105, in the data association graph, an abnormal association density area is identified, specifically including: gridding the data association graph to divide the data association graph into multiple grid units of the same size, each grid unit including at least one node; calculating the association density of each grid unit, the association density is the actual number of edges in the grid unit divided by a preset number, and the preset number is the maximum number of edges that may exist in the grid unit; according to the association density, calculating the domain association density of the first grid unit, the domain association density is the average value of the association density of the second grid unit adjacent to the first grid unit, and the first grid unit is any one of the multiple grid units; calculating the absolute value of the difference between the association density of the first grid unit and the domain association density of the first grid unit, and taking the absolute value as the density difference of the first grid unit; determining the first grid unit whose density difference is greater than or equal to the preset difference threshold as an abnormal grid unit, and merging the adjacent abnormal grid units into an abnormal association density area.

[0055] Specifically, the server identifies abnormal association density areas in the data association graph. Abnormal association density areas refer to areas in the data association graph where the association density (i.e., the number of edges in a unit area) is significantly higher or lower than that of the surrounding areas. These areas usually indicate that the association relationship between data elements is abnormally close or sparse, which may reflect some abnormal events or special patterns.

[0056] First, the server performs gridding processing on the data association graph and divides it into multiple grid units of the same size. In the embodiment of the present application, the gridding processing can be understood as a spatial indexing method that can convert an irregular graph structure into a regular grid structure. The server sets an appropriate grid size according to the size and density of the data association graph to balance the computing efficiency and accuracy. Each grid unit contains at least one node in the data association graph and the edges connected to these nodes. Then, the server calculates the association density of each grid unit. The calculation formula of the association density is: association density = actual number of edges in the grid unit / preset number. Among them, the preset number is the maximum number of edges that may exist in the grid unit, which can be calculated based on the number of combinations of the number of nodes in the grid unit. For example, for a grid unit containing n nodes, the preset number is n(n-1) / 2. By dividing by the preset number, the association density can be normalized to the interval [0, 1], which is convenient for comparison between different grid units.

[0057] Next, the server calculates the domain association density of each grid cell. The domain association density reflects the average association density level of the area surrounding a grid cell. For a first grid cell, its domain association density is the average value of the association density of the second grid cell adjacent to it. Adjacent grid cells are grid cells that are adjacent to the first grid cell in the up, down, left, right, and diagonal directions. By calculating the domain association density, the server can determine the local association density environment of each grid cell.

[0058] After calculating the association density and domain association density, the server calculates the density difference of each grid unit. The calculation formula of density difference is: density difference = | association density of the first grid unit - domain association density of the first grid unit |. Density difference measures the difference between the association density of a grid unit and the association density of its surrounding areas. The greater the difference, the more likely the grid unit is an abnormal area.

[0059] Finally, the server identifies the abnormal grid cells according to the preset difference threshold, and merges the adjacent abnormal grid cells into associated density abnormal areas. The server marks the grid cells whose density difference is greater than or equal to the preset difference threshold as abnormal grid cells. The setting of the preset difference threshold is determined according to the distribution characteristics of the data and business needs. It is necessary to be able to detect the real abnormal area and avoid false alarms as much as possible. This application does not limit the specific value of the preset difference threshold. The server adopts a regional growing algorithm, using abnormal grid cells as seeds, and continuously absorbs the surrounding abnormal grid cells until no new abnormal grid cells are added to form a complete abnormal area.

[0060] For example, the server performs abnormal area detection on the data association graph of an enterprise. The data association graph contains 1,000 nodes and 5,000 edges, representing the association relationships between 1,000 enterprises. The server divides the data association graph into 100 grid units of the same size, each unit containing 10 nodes. Then, the server calculates the association density and domain association density of each unit, and finds that the density difference of 5 of the units exceeds the preset threshold of 0.5. Then, the server performs regional merger on these 5 abnormal units, and finally obtains 2 association density abnormal areas. Step S106: Mark the data elements contained in the association density abnormal area as abnormal data.

[0061] Step S106: Mark the data elements included in the abnormal correlation density region as abnormal data.

[0062] In step S106, the server marks the data elements included in the abnormal correlation density region as abnormal data. The abnormal data refers to the data elements corresponding to the nodes in the abnormal correlation density region.

[0063] Specifically, the server first identifies the nodes contained in each abnormal correlation density region. Since the abnormal correlation density region is formed by merging multiple adjacent abnormal grid cells, the nodes in the abnormal region are the set of all nodes in these abnormal grid cells. The server traverses the grid cells in the abnormal region, obtains the node list therein, and forms an abnormal node set.

[0064] Then, the server marks the data element corresponding to each node in the abnormal node set as abnormal data. The server searches for the data element corresponding to the node in the original data set based on the node's attribute information, such as node ID, node type, etc., and marks it as abnormal. The marking method can be to attach a special label or field to the data element to indicate that it belongs to the abnormal area; or the abnormal data element can be stored separately in a special abnormal data set and managed separately from normal data.

[0065] Step S107: Determine the corresponding abnormal data set according to the position information of the abnormal data in the original data set.

[0066] In step S107, the corresponding abnormal data set is determined according to the position information of the abnormal data in the original data set, specifically including: obtaining the position information of the abnormal data in the target data set, the position information including the row number and column number of the abnormal data in the target data set; obtaining the position information of the abnormal data in the structured data and the unstructured data according to the position information of the abnormal data in the target data set; determining the target structured data associated with the abnormal data according to the position information of the abnormal data in the structured data, and determining the target unstructured data associated with the abnormal data according to the position information of the abnormal data in the unstructured data; merging the target structured data and the target unstructured data to obtain a subset of the original data; and using the subset of the original data as the abnormal data set corresponding to the abnormal data.

[0067] Specifically, the server determines the abnormal data set associated with the abnormal data according to the position information of the abnormal data in the original data set. The abnormal data set refers to the data set in the same triplet as the abnormal data. Specifically, the server first obtains the position information of the abnormal data in the target data set. The target data set is a combination of structured data and unstructured data, and the abnormal data has a clear row number and column number in it. The row number indicates the number of the record or document where the abnormal data is located, and the column number indicates the specific position or field of the abnormal data in the record or document. The server searches for the position information of the abnormal data in the index structure of the target data set through the unique identifier of the abnormal data, such as the node ID, the data element ID, etc. Then, the server reversely searches for the position information of the abnormal data in the original structured data and the unstructured data according to the position information of the abnormal data in the target data set. Since the target data set is obtained by preprocessing and transforming the structured data and the unstructured data, there is a certain mapping relationship between the position information of the abnormal data in the target data set and its position information in the original data. Through this mapping relationship, the server can accurately locate the abnormal data in the structured data and the unstructured data.

[0068] For structured data, the server searches for the corresponding records and fields in the structured data table or database according to the row number and column number of the abnormal data, extracts other field data in the same record as the abnormal data, and forms the target structured data. For unstructured data, the server searches for the corresponding documents or text fragments in the unstructured data document or corpus according to the row number of the abnormal data, extracts other text data in the same document as the abnormal data, and forms the target unstructured data.

[0069] Finally, the server merges the extracted target structured data and target unstructured data to obtain a subset of the original data as the abnormal data set corresponding to the abnormal data. The abnormal data set inherits the attribute structure and content characteristics of the original data, but only contains records and documents associated with the abnormal data, which is smaller in scale and more targeted.

[0070] Reference Figure 2The present application also provides an associated abnormal data set identification device based on big data, which is a server, and the server includes an acquisition module 201 and a processing module 202; the acquisition module 201 is used to acquire the target enterprise and the upstream enterprises and downstream enterprises associated with the target enterprise; the processing module 202 is used to construct an original data set according to the target enterprise, the upstream enterprises and the downstream enterprises, and the original data set includes structured data and unstructured data; the processing module 202 is also used to preprocess the structured data, perform triple conversion on the unstructured data, and merge the preprocessed structured data and the converted unstructured data to obtain the target data set; the processing module 202 , and is also used to construct a data association graph based on the target data set, where the nodes in the data association graph represent the data elements in the target data set, and the edges in the data association graph represent the association relationship between the data elements, and the association relationship is obtained by calculating the similarity between the data elements; the processing module 202 is also used to identify the abnormal association density area in the data association graph, where the abnormal association density area is the area with abnormal association density, and the association density refers to the number of edges in a unit area; the processing module 202 is also used to mark the data elements contained in the abnormal association density area as abnormal data; the processing module 202 is also used to determine the corresponding abnormal data set according to the position information of the abnormal data in the original data set.

[0071] In a possible implementation, the processing module 202 identifies an abnormal association density region in a data association graph, specifically including: the processing module 202 performs gridding processing on the data association graph, dividing the data association graph into a plurality of grid units of the same size, each grid unit including at least one node; the processing module 202 calculates the association density of each grid unit, where the association density is the actual number of edges in the grid unit divided by a preset number, where the preset number is the maximum number of edges that may exist in the grid unit; the processing module 202 calculates the domain association density of a first grid unit based on the association density, where the domain association density is the average value of the association density of a second grid unit adjacent to the first grid unit, where the first grid unit is any one of the plurality of grid units; the processing module 202 calculates the absolute value of the difference between the association density of the first grid unit and the domain association density of the first grid unit, and uses the absolute value as the density difference of the first grid unit; the processing module 202 determines that a first grid unit whose density difference is greater than or equal to a preset difference threshold is an abnormal grid unit, and merges adjacent abnormal grid units into an abnormal association density region.

[0072] In a possible implementation, the processing module 202 constructs an original data set based on the target enterprise, upstream enterprises and downstream enterprises, specifically including: the acquisition module 201 acquires the structured data of the target enterprise, upstream enterprises and downstream enterprises, the structured data includes basic information data of the enterprise, financial data and transaction data; the acquisition module 201 acquires the unstructured data of the target enterprise, upstream enterprises and downstream enterprises, the unstructured data includes corporate news data, judicial litigation data and patent data; the processing module 202 cleans and integrates the structured data and the unstructured data to obtain the original data set.

[0073] In a possible implementation, the step of converting unstructured data into triples by the processing module 202 specifically includes: the processing module 202 performs natural language processing on the unstructured data to extract key information; the processing module 202 converts the key information into triples according to a preset template, and the triples include a subject, a predicate, and an object; the processing module 202 uses the triples as data elements of the target data set.

[0074] In a possible implementation, the processing module 202 constructs a data association graph based on the target data set, specifically including: the processing module 202 uses each data element in the target data set as a node of the data association graph; the processing module 202 calculates the similarity between the first data element and the second data element, where the first data element and the second data element are any two data elements among multiple data elements; if the processing module 202 determines that the similarity is greater than or equal to a preset similarity threshold, an edge is established between the first data element and the second data element to obtain a data association graph.

[0075] In a possible implementation, the processing module 202 calculates the similarity between the first data element and the second data element, specifically including: the processing module 202 extracts the first eigenvector of the first data element, and extracts the second eigenvector of the second data element; the processing module 202 calculates the cosine similarity between the first eigenvector and the second eigenvector, and uses the cosine similarity as the similarity between the first data element and the second data element.

[0076] In a possible implementation, the processing module 202 determines the corresponding abnormal data set according to the position information of the abnormal data in the original data set, specifically including: the acquisition module 201 acquires the position information of the abnormal data in the target data set, the position information includes the row number and column number of the abnormal data in the target data set; the processing module 202 acquires the position information of the abnormal data in the structured data and the unstructured data according to the position information of the abnormal data in the target data set; the processing module 202 determines the target structured data associated with the abnormal data according to the position information of the abnormal data in the structured data, and determines the target unstructured data associated with the abnormal data according to the position information of the abnormal data in the unstructured data; the processing module 202 merges the target structured data and the target unstructured data to obtain a subset of the original data; and uses the subset of the original data as the abnormal data set corresponding to the abnormal data.

[0077] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0078] The present application also provides an electronic device. Figure 3 , Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0079] The communication bus 302 is used to realize the connection and communication between these components.

[0080] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0081] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0082] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0083] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally also be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3 , the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of a method for identifying an associated abnormal data set based on big data.

[0084] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program of the associated abnormal data set identification method based on big data stored in the memory 305, and when executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0085] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments.

[0086] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0090] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0091] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0092] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for identifying associated abnormal data sets based on big data, characterized in that: The method comprises: Obtain the target enterprise and the upstream and downstream enterprises associated with the target enterprise; Constructing an original data set according to the target enterprise, the upstream enterprise, and the downstream enterprise, wherein the original data set includes structured data and unstructured data; Preprocessing the structured data, performing triple conversion on the unstructured data, and merging the preprocessed structured data and the converted unstructured data to obtain a target data set; Based on the target data set, a data association graph is constructed, wherein nodes in the data association graph represent data elements in the target data set, edges in the data association graph represent association relationships between the data elements, and the association relationships are obtained by calculating similarities between the data elements; In the data association graph, identifying an abnormal association density region, wherein the abnormal association density region is an area with abnormal association density, wherein the association density refers to the number of edges within a unit area; Marking the data elements contained in the abnormal correlation density region as abnormal data; According to the position information of the abnormal data in the original data set, a corresponding abnormal data set is determined.

2. The method according to claim 1, characterized in that In the data association graph, identifying an abnormal association density region specifically includes: Performing gridding processing on the data association graph, dividing the data association graph into a plurality of grid units of the same size, each of the grid units including at least one node; Calculating the association density of each of the grid cells, where the association density is the actual number of edges in the grid cell divided by a preset number, where the preset number is the maximum number of edges that may exist in the grid cell; According to the association density, calculating the domain association density of the first grid unit, the domain association density is the average value of the association densities of the second grid units adjacent to the first grid unit, the first grid unit is any one of the plurality of grid units; Calculating an absolute value of a difference between an association density of the first grid unit and a domain association density of the first grid unit, and using the absolute value as a density difference of the first grid unit; The first grid unit whose density difference is greater than or equal to a preset difference threshold is determined as an abnormal grid unit, and adjacent abnormal grid units are merged into the associated density abnormal area.

3. The method according to claim 1, characterized in that The constructing of the original data set according to the target enterprise, the upstream enterprise and the downstream enterprise specifically includes: Acquire structured data of the target enterprise, the upstream enterprise, and the downstream enterprise, wherein the structured data includes basic enterprise information data, financial data, and transaction data; Acquire unstructured data of the target enterprise, the upstream enterprise, and the downstream enterprise, wherein the unstructured data includes enterprise news data, judicial litigation data, and patent data; The structured data and the unstructured data are cleaned and integrated to obtain the original data set.

4. The method according to claim 1, characterized in that: The step of converting the unstructured data into triples specifically includes: Performing natural language processing on the unstructured data to extract key information; According to a preset template, the key information is converted into a triple, wherein the triple includes a subject, a predicate, and an object; The triplet is used as a data element of the target data set.

5. The method according to claim 1, characterized in that The constructing a data association graph based on the target data set specifically includes: Using each data element in the target data set as a node of the data association graph; Calculating a similarity between a first data element and a second data element, where the first data element and the second data element are any two data elements among the plurality of data elements; If it is determined that the similarity is greater than or equal to a preset similarity threshold, an edge is established between the first data element and the second data element to obtain the data association graph.

6. The method according to claim 5, characterized in that The calculating the similarity between the first data element and the second data element specifically includes: extracting a first feature vector of the first data element, and extracting a second feature vector of the second data element; The cosine similarity between the first feature vector and the second feature vector is calculated, and the cosine similarity is used as the similarity between the first data element and the second data element.

7. The method according to claim 1, characterized in that The determining, according to the location information of the abnormal data in the original data set, the corresponding abnormal data set specifically includes: Acquire location information of the abnormal data in the target data set, wherein the location information includes a row number and a column number of the abnormal data in the target data set; According to the location information of the abnormal data in the target data set, acquiring the location information of the abnormal data in the structured data and the unstructured data; Determine target structured data associated with the abnormal data according to location information of the abnormal data in the structured data, and determine target unstructured data associated with the abnormal data according to location information of the abnormal data in the unstructured data; Merging the target structured data and the target unstructured data to obtain an original data subset; The original data subset is used as an abnormal data set corresponding to the abnormal data.

8. A device for identifying associated abnormal data sets based on big data, characterized in that: The device comprises an acquisition module (201) and a processing module (202), wherein: The acquisition module (201) is used to acquire a target enterprise and upstream enterprises and downstream enterprises associated with the target enterprise; The processing module (202) is used to construct an original data set according to the target enterprise, the upstream enterprise and the downstream enterprise, wherein the original data set includes structured data and unstructured data; The processing module (202) is further used to preprocess the structured data, perform triple conversion on the unstructured data, and merge the preprocessed structured data and the converted unstructured data to obtain a target data set; The processing module (202) is further configured to construct a data association graph based on the target data set, wherein the nodes in the data association graph represent data elements in the target data set, and the edges in the data association graph represent association relationships between the data elements, wherein the association relationships are obtained by calculating the similarity between the data elements; The processing module (202) is further used to identify an abnormal association density region in the data association graph, wherein the abnormal association density region is an area with abnormal association density, and the association density refers to the number of edges in a unit area; The processing module (202) is further configured to mark the data elements contained in the abnormal correlation density region as abnormal data; The processing module (202) is further configured to determine a corresponding abnormal data set according to location information of the abnormal data in the original data set.

9. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images