Data checking method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of data processing, and relates to a data checking method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining initial heterogeneous data, and constructing a graph database instance according to the initial heterogeneous data and a pre-constructed graph database model; extracting an initial data set from the graph database instance, and performing privacy encryption processing on the initial data set to obtain a privacy data set; carrying out decryption analysis on the privacy data set to obtain a data association graph; acquiring a preset data auditing rule, and performing data inspection on the data association graph according to the data auditing rule to obtain a data inspection result; and converting the data inspection result into a data inspection report, and sending the data inspection report to a display interface for display. According to the method and the device, the heterogeneous data can be safely and effectively audited and checked, so that the corresponding data check report is obtained and displayed.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, specifically to the field of digital medicine, and in particular to a data checking method, device, computer equipment and storage medium. Background Art

[0002] In the field of digital healthcare or finance, as business expands and becomes more complex, data governance becomes an indispensable part of enterprise management. Data auditing plays a decisive and key role in data governance. Therefore, how to conduct data auditing quickly and effectively becomes a key focus of data governance.

[0003] Traditional data auditing methods face many challenges, among which the complexity of multi-source heterogeneous data is the primary challenge. In the field of digital medicine, data comes from multiple sources such as electronic medical records, medical images, and gene sequencing reports, while in the field of financial technology, data covers various aspects such as transaction records, market forecasts, and risk assessments. The formats and structures of these data vary significantly, and the data quality is also uneven. When dealing with such large-scale data, traditional auditing methods find it difficult to quickly discover problematic data and handle them in a timely manner, which greatly reduces the quality and efficiency of data auditing. In addition, these data are highly private. During the data audit process, they need to be checked by multiple parties, and it is difficult to effectively ensure the security and privacy of data during transmission, which greatly reduces the security of data inspection and is not conducive to the effective development of the business. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a data checking method, apparatus, computer equipment and storage medium to solve the problem of being unable to safely and effectively audit heterogeneous data.

[0005] In order to solve the above technical problems, the present application provides a data checking method, which adopts the following technical solution:

[0006] Acquire initial heterogeneous data, and build a graph database instance according to the initial heterogeneous data and a pre-built graph database model;

[0007] Extracting an initial data set from the graph database instance, and performing privacy encryption processing on the initial data set to obtain a privacy-protected data set;

[0008] Decrypting and analyzing the privacy-preserving data set to obtain a data association graph;

[0009] Obtaining preset data audit rules, and performing data inspection on the data association graph according to the data audit rules to obtain data inspection results;

[0010] The data inspection result is converted into a data inspection report, and the data inspection report is sent to a display interface for display.

[0011] Furthermore, the step of obtaining initial heterogeneous data and constructing a graph database instance according to the initial heterogeneous data and a pre-constructed graph database model specifically includes:

[0012] Acquire a heterogeneous data extraction identifier, and extract the initial heterogeneous data from a database according to the heterogeneous data extraction identifier;

[0013] Performing attribute feature analysis on the initial heterogeneous data according to a data mining algorithm to obtain key attributes of the data;

[0014] Perform association relationship mining on the key attributes of the data according to an association rule mining algorithm to construct a data attribute association graph;

[0015] Performing semantic association feature learning on the data attribute association graph according to a graph neural network algorithm to generate model optimization parameters;

[0016] The graph database model is optimized according to the model optimization parameters, and the initial heterogeneous data is graphically represented according to the optimized graph database model to construct the graph database instance.

[0017] Furthermore, the step of extracting an initial data set from the graph database instance and performing privacy encryption processing on the initial data set to obtain a privacy-protected data set specifically includes:

[0018] Acquire data query information, and extract the initial data set from the graph database instance according to the data query information;

[0019] Performing homomorphic encryption processing on the initial data set to obtain an encrypted data set;

[0020] Performing a privacy level assessment on the encrypted data set to obtain a privacy assessment result;

[0021] Determining whether the privacy assessment result is greater than or equal to a preset privacy threshold;

[0022] If the privacy assessment result is greater than or equal to the privacy threshold, desensitizing the encrypted data set to obtain a desensitized data set;

[0023] If the privacy assessment result is less than the privacy threshold, marking the encrypted data set as a non-massified data set;

[0024] The desensitized data set and the non-desensitized data set are merged into the privacy-enhanced data set.

[0025] Furthermore, the step of performing decryption analysis on the privacy-preserving data set to obtain a data association graph specifically includes:

[0026] Performing homomorphic decryption on the privacy-preserving data set to obtain a decrypted data set;

[0027] Constructing a data graph model according to the decrypted data set;

[0028] Calculate the path distance of the nodes of the data graph model based on the shortest path algorithm to obtain the node path distance, and use the node path distance as the node association strength;

[0029] Calculate the importance score of each node in the data graph model based on a preset ranking algorithm;

[0030] The data association graph is constructed according to the node association strength and the importance score.

[0031] Furthermore, the step of obtaining a preset data audit rule, performing a data check on the data association graph according to the data audit rule, and obtaining a data check result specifically includes:

[0032] Obtaining an audit rule extraction identifier, and extracting the data audit rule from a database according to the audit rule extraction identifier;

[0033] Parsing the data audit rules to obtain data integrity rules, consistency rules, and validity rules;

[0034] The data association graph is subjected to data inspection according to the data integrity rule, the consistency rule, and the validity rule to obtain the data inspection result.

[0035] Furthermore, the step of converting the data inspection result into a data inspection report specifically includes:

[0036] Extract features from the data inspection results according to a pre-trained feature recognition model to obtain abnormal data features and key information features;

[0037] Based on natural language processing technology, semantic understanding and information extraction are performed on the abnormal data features and the key information features to obtain data compliance indicators, data integrity indicators, and data consistency indicators;

[0038] The data compliance indicator, the data integrity indicator, and the data consistency indicator are input into a pre-trained text generation model to obtain the data inspection report.

[0039] Furthermore, before the step of extracting features from the data inspection results according to the pre-trained feature recognition model to obtain abnormal data features and key information features, the following steps are also included:

[0040] Acquire a sample inspection result data set, and parse the sample inspection result data set to obtain a data record attribute value;

[0041] Performing data cleaning and feature selection on the sample inspection result data set according to the data record attribute values ​​to obtain a standard result data set;

[0042] Performing feature conversion on the standard result data set to obtain a result feature data set;

[0043] The preset decision tree model is trained according to the result feature data set to obtain the feature recognition model.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides a data checking device, which adopts the following technical solution:

[0045] A data construction module, used to obtain initial heterogeneous data and construct a graph database instance according to the initial heterogeneous data and a pre-constructed graph database model;

[0046] A data processing module, used to extract an initial data set from the graph database instance, and perform privacy encryption processing on the initial data set to obtain a privacy-protected data set;

[0047] A data analysis module, used to perform decryption analysis on the privacy-preserving data set to obtain a data association graph;

[0048] A data checking module is used to obtain preset data auditing rules, perform data checking on the data association graph according to the data auditing rules, and obtain data checking results;

[0049] The report display module is used to convert the data inspection result into a data inspection report, and send the data inspection report to the display interface for display.

[0050] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0051] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of any of the above data checking methods when executing the computer-readable instructions.

[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0053] A computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data checking method described in any one of the above items are implemented.

[0054] Compared with the prior art, the embodiment of the present application has the following beneficial effects: the embodiment obtains initial heterogeneous data, builds a graph database instance according to the initial heterogeneous data and a pre-built graph database model; extracts an initial data set from the graph database instance, and performs privacy encryption processing on the initial data set to obtain a privacy-protected data set; performs decryption analysis on the privacy-protected data set to obtain a data association graph; obtains preset data audit rules, performs data inspection on the data association graph according to the data audit rules to obtain data inspection results; converts the data inspection results into a data inspection report, and sends the data inspection report to a display interface for display. This effectively realizes safe and efficient auditing of heterogeneous data to obtain corresponding data inspection reports for display. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0057] Figure 2 A flowchart of an embodiment of a data inspection method according to the present application;

[0058] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S10;

[0059] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S20;

[0060] Figure 5 yes Figure 2 A flowchart of a specific implementation of step S30;

[0061] Figure 6 yes Figure 2 A flowchart of a specific implementation of step S40;

[0062] Figure 7yes Figure 2 A flowchart of a specific implementation of step S50;

[0063] Figure 8 is a structural schematic diagram of an embodiment of a data inspection device according to the present application;

[0064] Fig. 9 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0066] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor does it refer to non-related or alternative embodiments that are mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0067] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0068] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0069] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0070] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0071] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0072] It should be noted that the data checking method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the data checking device is generally arranged in the server / terminal device.

[0073] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0074] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for data checking according to the present application. The data checking method comprises the following steps:

[0075] Step S10, obtaining initial heterogeneous data, and constructing a graph database instance according to the initial heterogeneous data and a pre-constructed graph database model;

[0076] In this embodiment, the initial heterogeneous data is different types of data (such as structured data, semi-structured data, and unstructured data) obtained from multiple sources. The initial heterogeneous data can be relevant data information in the medical field, wherein the structured data includes: basic patient information, diagnosis information, examination results, treatment records, etc., the semi-structured data includes: electronic medical records, medical imaging reports, laboratory result reports, etc., and the unstructured data includes: medical images, doctor's notes, patient self-report information, etc. The pre-built graph database model refers to the graph data structure designed according to business requirements and data characteristics before building the graph database. The model defines the nodes (representing entities or concepts), edges (representing the relationship or behavior between entities) and their attributes and labels in the graph. In the graph data model, nodes and edges are basic components, and these elements are distinguished by describing attributes and labels. The graph database instance refers to a specific data set created and stored in an actual graph database system according to the pre-established graph data model. The graph database instance contains the actual data of nodes, edges, and their attributes and labels defined according to the model. This part of data can be heterogeneous data from multiple sources. After cleaning, preprocessing and transformation, it is imported into the graph database to support complex queries, analysis and applications.

[0077] Step S20, extracting an initial data set from the graph database instance, and performing privacy encryption processing on the initial data set to obtain a privacy-protected data set;

[0078] In this embodiment, the privacy encryption process includes homomorphic encryption and privacy assessment, wherein homomorphic encryption is a special encryption algorithm whose core idea is to allow specific computing operations to be performed directly on the encrypted data without decrypting the data first. The privacy assessment is a privacy level assessment performed on the initial data set after homomorphic encryption, and the corresponding privacy assessment result is obtained according to the privacy level assessment, so as to facilitate the subsequent further processing of the encrypted initial data set according to the privacy assessment result to obtain a privacy-protected data set.

[0079] Step S30, performing decryption analysis on the privacy-preserving data set to obtain a data association graph;

[0080] In this embodiment, decryption analysis includes homomorphic decryption, data graph model construction, and model node analysis, wherein homomorphic decryption is a decryption process corresponding to homomorphic encryption operations, and homomorphic decryption refers to decrypting a privacy-enhanced dataset using the same private key as homomorphic encryption to restore the original dataset information. Data graph model construction is a graph model construction based on the encrypted privacy-enhanced dataset. The data graph model includes nodes (representing entities or concepts) and edges (representing the relationship between entities), and attributes and labels are assigned to nodes and edges to describe their characteristics and relationships. Model node analysis is to analyze and calculate the nodes in the data graph model to obtain the corresponding node key information, which includes model key nodes, node association strength, and importance scores.

[0081] Step S40, obtaining a preset data audit rule, and performing a data check on the data association graph according to the data audit rule to obtain a data check result;

[0082] In this embodiment, the preset data audit rules include data integrity rules, consistency rules, and validity rules. Among them, the data integrity rules are used to ensure the comprehensiveness of the information in the data set, avoid the lack of key data causing the analysis results to deviate from the actual situation, and confirm whether all necessary data are included by traversing each node and edge in the data association map, and compare the data association map with the original privacy data set to check whether any data is lost during the conversion or processing process. The consistency rules are used to ensure the consistency of data between different systems, databases or tables, avoid data inconsistency causing business problems, and use data verification tools or algorithms to perform cross-source data consistency verification, and check whether the nodes and edges in the data association map meet the consistency constraints. The validity rules are used to ensure the degree to which the data reflects the real situation, avoid invalid data leading to erroneous conclusions, and verify the accuracy of the data in the data association map by comparing the authoritative source or original records to confirm the accuracy of the data to check whether the nodes and edges in the data association map conform to the business logic or common sense rules.

[0083] Step S50: convert the data inspection result into a data inspection report, and send the data inspection report to a display interface for display.

[0084] In this embodiment, the data inspection report is report data generated by textual representation of the data inspection results, wherein the data inspection report corresponds to an inspection report framework and key content parameters, and the key content parameters correspond to data compliance indicators, data integrity indicators, and data consistency indicators. The data compliance indicators, data integrity indicators, and data consistency indicators are further extracted from abnormal data features and key information features extracted from the data inspection results.

[0085] In this embodiment, the above method can be applied to a medical service system, in which the medical service system performs data verification on the stored medical-related data, thereby effectively ensuring the real-time validity of the medical-related data, so as to facilitate subsequent data sorting. Specifically, in this embodiment, the medical service system can be one or more of a medical insurance system and a disease insurance system, the initial heterogeneous data is multi-source medical heterogeneous data collected through multiple channels, the initial heterogeneous data is stored in the medical insurance system and the disease insurance system and is obtained from the database of the above system, the data inspection report is generated by the above system through processing by the method of this embodiment, and is displayed in the display interface of the above system.

[0086] This embodiment obtains initial heterogeneous data, builds a graph database instance based on the initial heterogeneous data and a pre-built graph database model; extracts an initial data set from the graph database instance, and performs privacy encryption processing on the initial data set to obtain a privacy-protected data set; performs decryption analysis on the privacy-protected data set to obtain a data association graph; obtains preset data audit rules, performs data inspection on the data association graph according to the data audit rules to obtain data inspection results; converts the data inspection results into a data inspection report, and sends the data inspection report to a display interface for display. This effectively implements safe and efficient auditing of heterogeneous data to obtain corresponding data inspection reports for display.

[0087] refer to Figure 3 In some optional implementations of this embodiment, step S10 includes the following steps:

[0088] Step S101, obtaining a heterogeneous data extraction identifier, and extracting the initial heterogeneous data from the database according to the heterogeneous data extraction identifier;

[0089] In this embodiment, the heterogeneous data extraction identifier is a unique information identifier corresponding to the initial heterogeneous data, and traversal matching is performed in the database through the heterogeneous data extraction identifier to extract the corresponding initial heterogeneous data.

[0090] Step S102, performing attribute feature analysis on the initial heterogeneous data according to a data mining algorithm to obtain key attributes of the data;

[0091] In this embodiment, the data mining algorithm may adopt a decision tree algorithm to classify based on the characteristic attributes of the initial heterogeneous data, thereby intuitively showing which characteristic attributes play a key role in the classification process, so as to effectively identify the key attributes of the data.

[0092] Step S103, mining association relationships of the key attributes of the data according to an association rule mining algorithm, and constructing a data attribute association graph;

[0093] In this embodiment, the association rule mining algorithm can adopt Apriori, by setting necessary parameters for the association rule mining algorithm, such as support, confidence and lift, and applying the association rule mining algorithm after setting parameters to the key attributes of the data to generate corresponding association rules, which are used to describe the relationship between the key attributes, and according to the characteristics and quantity of the association rules, a suitable graphical representation method is selected to graphically represent the association rules and the key attributes of the data, so as to effectively form a data attribute association diagram. In this implementation, the graphical representation method can be represented by an undirected graph.

[0094] Step S104, performing semantic association feature learning on the data attribute association graph according to a graph neural network algorithm to generate model optimization parameters;

[0095] In this embodiment, the graph neural network algorithm can select a graph convolutional network, and by constructing the architecture of the graph convolutional network model and training the convolutional network model, effective semantic association feature learning can be performed on the data attribute association graph. Among them, the architecture of the graph convolutional network model includes an input layer, a hidden layer, and an output layer. The input layer receives the feature matrix of the node and the structural information of the graph. The hidden layer learns the embedded representation of the node through graph convolution or other mechanisms, and the output layer predicts according to task requirements (such as node classification, link prediction, etc.). When the convolutional network model training is completed, the convolutional network model training will output model optimization parameters including optimized nodes, edge embeddings, and model internal parameters.

[0096] Step S105, optimizing the graph database model according to the model optimization parameters, and graphically representing the initial heterogeneous data according to the optimized graph database model to construct the graph database instance.

[0097] In this embodiment, the model optimization parameters learned by the convolutional network model are applied to the graph database model to update the model structure or parameters. Specifically, the structure of the graph database model can be adjusted according to the model optimization parameters, such as adding or deleting nodes, edges, or modifying the properties of nodes and edges. Then, each entity in the initial heterogeneous data is mapped to the corresponding node or edge in the graph database model, and characteristic values ​​are assigned to the nodes and edges in the graph database according to the embedded representation in the model optimization parameters, thereby effectively realizing the construction of a graph database instance based on the optimized graph database model and the graphical representation of the initial heterogeneous data.

[0098] This embodiment obtains a heterogeneous data extraction identifier, extracts the initial heterogeneous data from the database according to the heterogeneous data extraction identifier; performs attribute feature analysis on the initial heterogeneous data according to a data mining algorithm to obtain data key attributes; performs association relationship mining on the data key attributes according to an association rule mining algorithm to construct a data attribute association graph; performs semantic association feature learning on the data attribute association graph according to a graph neural network algorithm to generate model optimization parameters; optimizes the graph database model according to the model optimization parameters, and graphically represents the initial heterogeneous data according to the optimized graph database model to construct the graph database instance. In this way, a graph database instance that effectively graphically represents the initial heterogeneous data is generated according to the data key attributes and the graph data model, so as to facilitate subsequent privacy encryption processing.

[0099] refer to Figure 4 In some optional implementations of this embodiment, step S20 includes the following steps:

[0100] Step S201, obtaining data query information, and extracting the initial data set in the graph database instance according to the data query information;

[0101] In this embodiment, the data query information is an information identifier corresponding to the initial data set, which is used to indicate the data type, attributes, and specific filtering conditions or association relationships of the query data. The data query information is traversed and matched in the graph database instance to extract the corresponding initial data set.

[0102] Step S202, performing homomorphic encryption processing on the initial data set to obtain an encrypted data set;

[0103] In this embodiment, a preset homomorphic encryption scheme (including encryption tools, libraries or frameworks) is used to encrypt each data item in the initial data set, a key pair (public key and private key) is generated by a key generation algorithm corresponding to homomorphic encryption, and then the public key is used to encrypt the data, thereby obtaining an encrypted data set. After obtaining the encrypted data set, the encrypted data set can be saved in a privacy database, and further privacy level assessment processing can be performed in the privacy database.

[0104] Step S203, performing a privacy level assessment on the encrypted data set to obtain a privacy assessment result;

[0105] In this embodiment, the privacy level of the encrypted data set can be evaluated through the secure multi-party computing protocol, where secure multi-party computing is a privacy-preserving distributed computing technology in the field of cryptography, which allows multiple participants to collaboratively calculate an agreed function without mutual trust and without a trusted third party, while ensuring that, except for the calculation results, each participant cannot infer the original data of other participants through the interactive data during the calculation process. By setting the preset evaluation target (such as the privacy leakage risk of the evaluation data set, determining the privacy protection level of the data, etc.) as the agreed function in the secure multi-party computing, the privacy evaluation result corresponding to the encrypted data set can be effectively evaluated.

[0106] Step S204, determining whether the privacy assessment result is greater than or equal to a preset privacy threshold;

[0107] In this embodiment, the privacy level corresponding to the privacy assessment result is quantified to obtain the corresponding data privacy value. In this embodiment, the privacy level includes primary, intermediate, and advanced, corresponding to the primary privacy value of 10, the intermediate privacy value of 20, and the advanced privacy value of 30. The data privacy is compared with the preset privacy threshold to determine whether the privacy assessment result is greater than or equal to the preset privacy threshold. In this embodiment, the privacy threshold can be set to 15, and the privacy threshold can be adjusted accordingly according to the actual situation.

[0108] Step S205: if the privacy assessment result is greater than or equal to the privacy threshold, desensitizing the encrypted data set to obtain a desensitized data set;

[0109] In this embodiment, when the data privacy value corresponding to the privacy assessment result is greater than or equal to the privacy threshold, the sensitive data is desensitized by using at least one of hashing, masking, and data replacement according to the privacy level corresponding to the encrypted data set to obtain a desensitized data set. For example, when the privacy level is medium, hashing can be used for desensitization, and when the privacy level is high, a combination of hashing and masking or hashing and data replacement can be used for desensitization.

[0110] Step S206: if the privacy assessment result is less than the privacy threshold, marking the encrypted data set as a non-massified data set;

[0111] In this embodiment, when the data privacy value corresponding to the privacy assessment result of the encrypted data set is less than the privacy threshold, the encrypted data set is marked as a non-desensitized data set through preset marking information.

[0112] Step S207: merge the desensitized data set and the non-desensitized data set into the privacy-enhanced data set.

[0113] In this embodiment, the desensitized data set and the non-desensitized data set are integrated to obtain a privacy-enhanced data set including desensitized private data and non-desensitized non-private data.

[0114] This embodiment obtains data query information, extracts the initial data set in the graph database instance according to the data query information; performs homomorphic encryption processing on the initial data set to obtain an encrypted data set; performs privacy level assessment on the encrypted data set to obtain a privacy assessment result; determines whether the privacy assessment result is greater than or equal to a preset privacy threshold; if the privacy assessment result is greater than or equal to the privacy threshold, desensitizes the encrypted data set to obtain a desensitized data set; if the privacy assessment result is less than the privacy threshold, marks the encrypted data set as a non-desensitized data set; and merges the desensitized data set and the non-desensitized data set into the privacy-enhanced data set. Thus, a privacy-enhanced data set that performs effective privacy encryption processing on the data in the graph database instance is obtained, so as to facilitate the subsequent construction of a data association graph.

[0115] refer to Figure 5 In some optional implementations of this embodiment, step S30 includes the following steps:

[0116] Step S301, performing homomorphic decryption on the privacy-preserving data set to obtain a decrypted data set;

[0117] In this embodiment, the private data set is decrypted using the private key generated during the homomorphic encryption to obtain a decrypted data set. After decryption, the correctness of the decryption result can be verified by comparing the partially decrypted data set with the initial data set (which can be obtained through a backup).

[0118] Step S302, constructing a data graph model according to the decrypted data set;

[0119] In this embodiment, the data graph model is a data structure used to represent entities (nodes) and their relationships (edges). By identifying entities that can be used as nodes (such as people, objects, events, etc.) and relationships that can be used as edges (such as relationships between people, relationships between objects and events, etc.) in the decrypted data set, and creating a node for each entity, and assigning a unique identifier and related attributes (such as name, type, value, etc.) to the node, and then creating edges connecting the nodes based on the relationships between the entities, the data graph model is effectively constructed. Among them, the edges can also have attributes, such as the type of relationship, weight, etc.

[0120] Step S303, calculating the path distance of the nodes of the data graph model based on the shortest path algorithm to obtain the node path distance, and using the node path distance as the node association strength;

[0121] In this embodiment, the shortest path algorithm can select the F l oyd-Warshall algorithm, and each node of the data graph model is selected as the source node by the F l oyd-Warshall algorithm for analysis and calculation, so as to obtain the shortest path distance corresponding to each node to other nodes, and the shortest path distance is the node path distance, and the node path distance is used to reflect the association strength between the two model nodes. The shorter the path, the stronger the association between the nodes; the longer the path, the weaker the association, and the shortest path length can be standardized, such as all path lengths can be divided by the maximum path length in the graph, so that the value of the association strength varies between 0 and 1, so as to facilitate subsequent intuitive analysis.

[0122] Step S304, calculating the importance score of each node in the data graph model based on a preset ranking algorithm;

[0123] In this embodiment, the preset ranking algorithm can adopt the PageRank algorithm, and the importance score of each node in the data graph model is calculated by the PageRank algorithm. Calculating the importance score of the node according to the PageRank algorithm includes the following steps: Initialization: assigning an initial PageRank value to each node, which can be equal or based on certain prior knowledge or assumptions. Iterative calculation: According to the in-link and out-link situation of each node, the PageRank value of each node is continuously updated according to the PageRank iterative formula. Convergence judgment: Determine whether the PageRank value converges, that is, whether it reaches a stable state. This state judgment can be achieved by comparing whether the difference between the PageRank value of the current iteration and the previous iteration is less than a preset threshold. If converged, stop the iteration; otherwise, continue the iteration. When the iteration is completed, the importance score corresponding to the node is output.

[0124] Step S305: construct the data association graph according to the node association strength and the importance score.

[0125] In this embodiment, the data graph model is optimized based on the node association strength and the corresponding importance score of the node. For example, based on the threshold of the node association strength, the edges with weak associations are filtered out to simplify the graph and highlight important relationships. The nodes are clustered according to the importance score and association strength of the nodes to identify key nodes and community structures in the graph, thereby effectively constructing a data association graph.

[0126] This embodiment obtains a decrypted data set by homomorphically decrypting the privacy-preserving data set; constructs a data graph model based on the decrypted data set; calculates the path distance of the nodes of the data graph model based on the shortest path algorithm to obtain the node path distance, and uses the node path distance as the node association strength; calculates the importance score of each node in the data graph model based on a preset ranking algorithm; and constructs the data association graph based on the node association strength and the importance score. In this way, a data association graph constructed according to the importance and association of the nodes in the data graph model is effectively obtained to facilitate subsequent data inspection and processing.

[0127] Continue to refer Figure 6 In some optional implementations of this embodiment, step S40 includes the following steps:

[0128] Step S401, obtaining an audit rule extraction identifier, and extracting the data audit rule from the database according to the audit rule extraction identifier;

[0129] In this embodiment, the audit rule extraction identifier is a unique information identifier corresponding to the data audit rule, and the audit rule extraction identifier is used to perform traversal matching in the database to extract the corresponding data audit rule.

[0130] Step S402, parsing the data audit rules to obtain data integrity rules, consistency rules, and validity rules;

[0131] In this embodiment, the data audit rules include data integrity rules, consistency rules, and validity rules. The data integrity rules, consistency rules, and validity rules are obtained by effectively parsing the data audit rules.

[0132] Step S403: Perform data check on the data association graph according to the data integrity rule, the consistency rule, and the validity rule to obtain the data check result.

[0133] In this embodiment, data information in the data association graph is checked by data integrity rules, consistency rules, and validity rules to obtain corresponding integrity check results, consistency check results, and validity check results, which are then integrated to obtain data check results.

[0134] This embodiment obtains the audit rule extraction identifier, extracts the data audit rule in the database according to the audit rule extraction identifier; parses the data audit rule to obtain data integrity rules, consistency rules, and validity rules; performs data inspection on the data association graph according to the data integrity rules, the consistency rules, and the validity rules, thereby effectively obtaining data inspection results of accurate data inspection according to the data audit rules, so as to facilitate the subsequent conversion of data inspection reports.

[0135] Continue to refer Figure 7 In some optional implementations of this embodiment, step S50 includes the following steps:

[0136] Step S501, extracting features from the data inspection results according to a pre-trained feature recognition model to obtain abnormal data features and key information features;

[0137] In this embodiment, the pre-trained feature recognition model adopts a decision tree model, and the decision tree model is trained by a sample inspection result data set to obtain a feature recognition model that can effectively extract features from the data inspection results. Abnormal data features include abnormal values, missing values, inconsistencies, etc. of the data, and key information features include specific attributes of the data, classification labels, timestamps, etc.

[0138] Step S502, performing semantic understanding and information extraction on the abnormal data features and the key information features based on natural language processing technology to obtain data compliance indicators, data integrity indicators, and data consistency indicators;

[0139] In this embodiment, natural language processing technology can use semantic analysis to understand and parse the semantic content of abnormal data features and key information features by performing semantic analysis on abnormal data features and key information features, so as to effectively extract corresponding data compliance indicators, data integrity indicators, and data consistency indicators. Among them, data compliance indicators include data privacy protection, data permissions, data format standards, etc., data integrity indicators include data integrity checks (such as the number of records, whether field values ​​are complete), data quality assessments, etc., and data consistency indicators include data consistency checks (such as whether field values ​​are consistent), data synchronization status, etc.

[0140] Step S503: input the data compliance indicator, the data integrity indicator, and the data consistency indicator into a pre-trained text generation model to obtain the data inspection report.

[0141] In this embodiment, the pre-trained text generation model can adopt a transformer-based text generation model, by converting data compliance indicators, data integrity indicators, and data consistency indicators into the input format corresponding to the text generation model, and then adding corresponding descriptive information, so as to effectively process the above indicators through the text generation model to generate a corresponding data inspection report.

[0142] This embodiment obtains abnormal data features and key information features by performing feature extraction on the data inspection results according to a pre-trained feature recognition model; performs semantic understanding and information extraction on the abnormal data features and the key information features based on natural language processing technology to obtain data compliance indicators, data integrity indicators, and data consistency indicators; and inputs the data compliance indicators, the data integrity indicators, and the data consistency indicators into a pre-trained text generation model, thereby effectively obtaining a data inspection report generated according to each data indicator in the data inspection results, so as to facilitate subsequent report presentation.

[0143] In some optional implementations of this embodiment, before step S501, the following steps are further included:

[0144] Acquire a sample inspection result data set, and parse the sample inspection result data set to obtain a data record attribute value;

[0145] In this embodiment, the sample inspection result data set is a data set composed of historical data inspection results. Each data in the sample inspection result data set corresponds to multiple data record attribute values, and the data record attribute values ​​include field name, data type, value, etc.

[0146] Performing data cleaning and feature selection on the sample inspection result data set according to the data record attribute values ​​to obtain a standard result data set;

[0147] In this embodiment, data cleaning includes missing value processing, outlier processing, data type conversion, data deduplication, etc. Feature selection is to select the most valuable features for model training based on the results of correlation analysis and feature importance evaluation, and remove redundant or low-value features.

[0148] Performing feature conversion on the standard result data set to obtain a result feature data set;

[0149] In this embodiment, feature conversion includes standardization / normalization (standardizing numerical features (such as using Z scores) or normalization (such as scaling values ​​to a range of 0-1) to eliminate the impact of different feature dimensions), encoding conversion (encoding categorical features, such as using One-Hot Encoding or Label Encoding), feature construction, and feature scaling. By performing the above feature conversion on the standard result data set, a result feature data set that can effectively train a decision tree model is obtained.

[0150] The preset decision tree model is trained according to the result feature data set to obtain the feature recognition model.

[0151] In this embodiment, the step of training a preset decision tree model according to the result feature data set includes: dividing the standard result data set into a training set and a test set, usually in a ratio of 70%-80% (training set) and 20%-30% (test set); using the training set to train the preset decision tree model, during the training process, the depth, pruning strategy, splitting standard and other parameters of the decision tree can be adjusted to optimize the model performance; using the test set to evaluate the trained decision tree model, and calculating the model's accuracy, recall rate, F1 score and other indicators; after training and evaluation, using the decision tree model as a feature recognition model for subsequent feature recognition tasks.

[0152] This embodiment obtains a sample inspection result data set and parses the sample inspection result data set to obtain data record attribute values; performs data cleaning and feature selection on the sample inspection result data set according to the data record attribute values ​​to obtain a standard result data set; performs feature conversion on the standard result data set to obtain a result feature data set; and trains a preset decision tree model according to the result feature data set, thereby effectively obtaining a feature recognition model that can accurately identify features in the data inspection results, so as to facilitate subsequent feature extraction processing of the data inspection results.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0154] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0155] Further references Figure 8 , as a response to the above Figure 1 In order to realize the method shown in the figure, the present application provides an embodiment of a data inspection device, and the device embodiment is Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0156] like Figure 8 As shown, the data inspection device 600 described in this embodiment includes: a data construction module 601, a data processing module 602, a data analysis module 603, a data inspection module 604, and a report display module 605. Among them:

[0157] The data construction module 601 is used to obtain initial heterogeneous data and construct a graph database instance according to the initial heterogeneous data and a pre-constructed graph database model;

[0158] The data processing module 602 is used to extract an initial data set from the graph database instance, and perform privacy encryption processing on the initial data set to obtain a privacy-protected data set;

[0159] The data analysis module 603 is used to perform decryption analysis on the privacy-preserving data set to obtain a data association graph;

[0160] The data checking module 604 is used to obtain a preset data auditing rule, and perform a data check on the data association graph according to the data auditing rule to obtain a data check result;

[0161] The report display module 605 is used to convert the data inspection result into a data inspection report, and send the data inspection report to the display interface for display.

[0162] By adopting the above-mentioned data inspection device, this embodiment can obtain initial heterogeneous data, build a graph database instance according to the initial heterogeneous data and the pre-built graph database model; extract the initial data set from the graph database instance, and perform privacy encryption processing on the initial data set to obtain a privacy-protected data set; perform decryption analysis on the privacy-protected data set to obtain a data association graph; obtain preset data audit rules, perform data inspection on the data association graph according to the data audit rules, and obtain data inspection results; convert the data inspection results into a data inspection report, and send the data inspection report to the display interface for display. This effectively realizes the safe and effective audit of heterogeneous data to obtain the corresponding data inspection report for display.

[0163] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0164] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 7 with components 71-73, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0165] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0166] The memory 71 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 71 can be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard disk equipped on the computer device 7, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 71 can also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as computer-readable instructions of the data inspection method, etc. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or are to be output.

[0167] The processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to run computer-readable instructions stored in the memory 71 or process data, such as computer-readable instructions for running the data checking method.

[0168] The network interface 73 may include a wireless network interface or a wired network interface. The network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0169] By adopting the above-mentioned computer device, this embodiment can obtain initial heterogeneous data, build a graph database instance according to the initial heterogeneous data and the pre-built graph database model; extract the initial data set from the graph database instance, and perform privacy encryption processing on the initial data set to obtain a privacy-protected data set; perform decryption analysis on the privacy-protected data set to obtain a data association graph; obtain preset data audit rules, perform data inspection on the data association graph according to the data audit rules, and obtain data inspection results; convert the data inspection results into a data inspection report, and send the data inspection report to the display interface for display. This effectively realizes the safe and effective audit of heterogeneous data to obtain the corresponding data inspection report for display.

[0170] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data checking method as described above.

[0171] By adopting the above-mentioned computer-readable storage medium, this embodiment can obtain initial heterogeneous data, build a graph database instance according to the initial heterogeneous data and the pre-built graph database model; extract the initial data set from the graph database instance, and perform privacy encryption processing on the initial data set to obtain a privacy-protected data set; perform decryption analysis on the privacy-protected data set to obtain a data association graph; obtain preset data audit rules, perform data inspection on the data association graph according to the data audit rules, and obtain data inspection results; convert the data inspection results into a data inspection report, and send the data inspection report to the display interface for display. This effectively realizes the safe and effective audit of heterogeneous data to obtain the corresponding data inspection report for display.

[0172] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0173] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0174] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A data checking method, characterized in that: The steps include: Acquire initial heterogeneous data, and build a graph database instance according to the initial heterogeneous data and a pre-built graph database model; Extracting an initial data set from the graph database instance, and performing privacy encryption processing on the initial data set to obtain a privacy-protected data set; Decrypting and analyzing the privacy-preserving data set to obtain a data association graph; Obtaining preset data audit rules, and performing data inspection on the data association graph according to the data audit rules to obtain data inspection results; The data inspection result is converted into a data inspection report, and the data inspection report is sent to a display interface for display.

2. The data checking method according to claim 1, characterized in that: The step of obtaining initial heterogeneous data and building a graph database instance according to the initial heterogeneous data and a pre-built graph database model specifically includes: Acquire a heterogeneous data extraction identifier, and extract the initial heterogeneous data from a database according to the heterogeneous data extraction identifier; Performing attribute feature analysis on the initial heterogeneous data according to a data mining algorithm to obtain key attributes of the data; Perform association relationship mining on the key attributes of the data according to an association rule mining algorithm to construct a data attribute association graph; Performing semantic association feature learning on the data attribute association graph according to a graph neural network algorithm to generate model optimization parameters; The graph database model is optimized according to the model optimization parameters, and the initial heterogeneous data is graphically represented according to the optimized graph database model to construct the graph database instance.

3. The data checking method according to claim 1, characterized in that: The step of extracting an initial data set from the graph database instance and performing privacy encryption processing on the initial data set to obtain a privacy-protected data set specifically includes: Acquire data query information, and extract the initial data set from the graph database instance according to the data query information; Performing homomorphic encryption processing on the initial data set to obtain an encrypted data set; Performing a privacy level assessment on the encrypted data set to obtain a privacy assessment result; Determining whether the privacy assessment result is greater than or equal to a preset privacy threshold; If the privacy assessment result is greater than or equal to the privacy threshold, desensitizing the encrypted data set to obtain a desensitized data set; If the privacy assessment result is less than the privacy threshold, marking the encrypted data set as a non-massified data set; The desensitized data set and the non-desensitized data set are merged into the privacy-enhanced data set.

4. The data checking method according to claim 1, characterized in that: The step of performing decryption analysis on the privacy-preserving data set to obtain a data association graph specifically includes: Performing homomorphic decryption on the privacy-preserving data set to obtain a decrypted data set; Constructing a data graph model according to the decrypted data set; Calculate the path distance of the nodes of the data graph model based on the shortest path algorithm to obtain the node path distance, and use the node path distance as the node association strength; Calculate the importance score of each node in the data graph model based on a preset ranking algorithm; The data association graph is constructed according to the node association strength and the importance score.

5. The data checking method according to claim 1, characterized in that: The step of obtaining a preset data audit rule, performing a data check on the data association graph according to the data audit rule, and obtaining a data check result specifically includes: Obtaining an audit rule extraction identifier, and extracting the data audit rule from a database according to the audit rule extraction identifier; Parsing the data audit rules to obtain data integrity rules, consistency rules, and validity rules; The data association graph is subjected to data inspection according to the data integrity rule, the consistency rule, and the validity rule to obtain the data inspection result.

6. The data checking method according to claim 1, characterized in that: The step of converting the data inspection result into a data inspection report specifically includes: Extract features from the data inspection results according to a pre-trained feature recognition model to obtain abnormal data features and key information features; Based on natural language processing technology, semantic understanding and information extraction are performed on the abnormal data features and the key information features to obtain data compliance indicators, data integrity indicators, and data consistency indicators; The data compliance indicator, the data integrity indicator, and the data consistency indicator are input into a pre-trained text generation model to obtain the data inspection report.

7. The data checking method according to claim 6, characterized in that: Before the step of extracting features from the data inspection results according to the pre-trained feature recognition model to obtain abnormal data features and key information features, the following steps are also included: Acquire a sample inspection result data set, and parse the sample inspection result data set to obtain a data record attribute value; Performing data cleaning and feature selection on the sample inspection result data set according to the data record attribute values ​​to obtain a standard result data set; Performing feature conversion on the standard result data set to obtain a result feature data set; The preset decision tree model is trained according to the result feature data set to obtain the feature recognition model.

8. A data checking device, characterized in that: include: A data construction module, used to obtain initial heterogeneous data and construct a graph database instance according to the initial heterogeneous data and a pre-constructed graph database model; A data processing module, used to extract an initial data set from the graph database instance, and perform privacy encryption processing on the initial data set to obtain a privacy-protected data set; A data analysis module, used to perform decryption analysis on the privacy-preserving data set to obtain a data association graph; A data checking module is used to obtain preset data auditing rules, perform data checking on the data association graph according to the data auditing rules, and obtain data checking results; The report display module is used to convert the data inspection result into a data inspection report, and send the data inspection report to the display interface for display.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the data checking method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the data checking method according to any one of claims 1 to 7 are implemented.

Images