Bayesian network-based medical structured data desensitization method and equipment thereof

Abstract

The invention relates to the technical field of big data, in particular to a medical structured data desensitization method and device based on a Bayesian network. The method comprises the following steps of: firstly, converting medical structural data into discretized data conforming to regulations; and carrying out structure learning and parameter learning on the original data set through an approximate solution method of score search and a maximum likelihood algorithm to obtain a Bayesian network containing an association relationship between indexes and conditional probability distribution corresponding to the association relationship, and finally generating a desensitized data set by using the Bayesian network. In order to verify the similarity between the generated desensitized data and the original data set, the desensitized data set is continuously utilized to construct the Bayesian networks, and the consistency of the two Bayesian networks in structure and parameter is compared. According to the method, the scheme is integrated and deployed in the database by utilizing PL / R extension of PostgreSQL, so that the desensitization process can be completed without data export, and the problem that the existing scheme is difficult to consider multi-dimensional requirements of data desensitization, associated information fidelity, security and the like is solved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a desensitization method for medical structured data based on a Bayesian network, and a corresponding computer program product, storage medium, and computer device. Background Art

[0002] With the advancement of medical informatization, medical databases have accumulated vast amounts of data from clinical diagnosis and treatment, health monitoring, and biomedical research. These data provide a crucial foundation for the development of disease prediction, precision medicine, and medical artificial intelligence. However, in the process of mining and sharing the value of medical data, the sensitivity of medical data has become increasingly prominent. Medical data contains a large amount of sensitive information (such as medical history and genetic data). Once leaked, it not only violates patient privacy but also may raise ethical and legal issues. To protect patient privacy and comply with medical data security regulations, medical data desensitization is necessary. The main goal of data desensitization is to reduce data sensitivity through substitution and transformation according to desensitization rules, minimize the exposure of sensitive data, and maintain the usability of the desensitized data as much as possible, ensuring that the desensitized data has commonality with the original data in the probability distribution of global attributes. This ensures that the desensitized data still meets the data needs of association analysis, machine learning, and real-time query.

[0003] While traditional data desensitization methods, such as anonymization, randomization, and data replacement, can reduce the risk of patient privacy breaches to a certain extent, the degree of desensitization is directly correlated with the degree of data distortion, often leading to reduced data usability. This is particularly true for medical structured data with high-dimensional attributes. While desensitizing these methods, they inevitably lead to the loss of attribute associations. Therefore, finding data desensitization methods that effectively protect patient data privacy while preserving the data's inherent attribute associations has become an urgent task. Summary of the Invention

[0004] In order to solve the problem that it is difficult to preserve the attribute association information while desensitizing the existing medical structured data with high-dimensional attributes, the present invention provides a desensitization method for medical structured data based on a Bayesian network, and its corresponding computer program product, storage medium and computer device.

[0005] The technical solutions provided by the present invention are as follows:

[0006] A desensitization method for medical structured data based on a Bayesian network includes the following steps:

[0007] A large amount of medical structured data is collected, and each piece of medical structured data is preprocessed to generate feature variables containing candidate values of various indicator items, thereby obtaining an original data set consisting of multiple feature variables.

[0008] The approximate solution method based on scoring search performs structured learning on each feature variable in the original data set, and then iteratively optimizes a directed acyclic graph with each indicator item as a node and the relationship between the indicator items as an edge.

[0009] Then, the maximum likelihood algorithm is used to learn the parameters of each feature variable in the original data set, and the sample data is used to estimate the conditional probability distribution of each node in the directed acyclic graph, thereby obtaining the first Bayesian network containing the joint probability distribution between the indicator items corresponding to all nodes.

[0010] Use the forward sampling algorithm to generate a masked data set containing multiple masked data based on the first Bayesian network; then use the approximate solution method of the scoring search and the maximum likelihood algorithm to generate a second Bayesian network that represents the joint probability distribution of the masked data; determine whether the structures and parameters of the first and second Bayesian networks are consistent:

[0011] (1) If yes, it means the generated desensitized dataset is qualified.

[0012] (2) If not, the generated desensitized data set is judged to be unqualified, and the medical structured data is re-processed, structured learning, parameter learning and probability distribution verification are performed until a qualified desensitized data set is obtained.

[0013] As a further improvement of the present invention, the preprocessing method of medical structured data includes:

[0014] i. When the candidate value of any indicator item is discrete data and the number of candidate values is not higher than the upper limit specified by the Bayesian network node attribute, the original candidate value of the indicator item is retained to constitute the corresponding feature variable.

[0015] ⅱ. When the candidate value of any indicator item is continuous data, or the candidate value of the indicator item is discrete data but the number of candidate values is higher than the upper limit specified by the Bayesian network node attribute, the original candidate value of the indicator item is mapped to a discrete interval that meets the requirements through the chiMerge algorithm, and then a feature variable composed of the new candidate value is generated.

[0016] As a further improvement of the present invention, the process of generating desensitized data according to the first Bayesian network using the forward sampling algorithm includes:

[0017] Starting from the root node of the first Bayesian network, the node states are sampled one by one in the order of topological sorting; a state is randomly selected as the state of the child node based on the states and conditional probability distribution of all parent nodes, and this process is repeated until the states of all nodes are determined; the characteristic variables of the corresponding indicator items are determined according to the state of each node, and then a piece of desensitized data is obtained.

[0018] As a further improvement of the present invention, a hill climbing algorithm is used to iteratively optimize a directed acyclic graph representing the correlation relationship between indicator items based on the original data set; the algorithm execution process includes:

[0019] 1. Initialization

[0020] Each indicator item is regarded as a node, and an edgeless blank graph is initialized as the initial network. The candidate network data and the maximum iteration rounds are preset, and a BIC scoring function is designed to evaluate the fit of the generated directed acyclic graph.

[0021] 2. Iterative Search

[0022] Through multiple rounds of iterative search, directed edges are gradually added between nodes in the initial network to achieve network structure optimization:

[0023] In each iteration, the initial network is first processed using a local graph search operator, updating the edges between nodes to generate multiple candidate networks. The BIC scoring function is then used to generate a BIC score for each candidate network, and the following decision is made: if any candidate network has a higher BIC score than the original initial network, it is used as the initial network for the next round; otherwise, the original initial network is retained as the initial network for the next round.

[0024] The operation strategies of the search operator include adding directed edges, deleting directed edges, and reversing directed edges.

[0025] 3. Optimize output

[0026] When the preset maximum number of iterations is reached or the BIC scoring function converges, the search process ends and the network with the highest BIC score in the candidate network set is returned as the required directed acyclic graph.

[0027] As a further improvement of the present invention, the BIC scoring function F is used to evaluate the fitting degree of the generated directed acyclic graph. BIC (G|D) is as follows:

[0028]

[0029] In the above formula, n represents the number of nodes in the candidate network; m ij Represents the i-th variable X in the dataset D i The number of samples when the parent node takes the jth value; m ijk Represents the i-th variable X in the dataset D i The parent node takes the jth value and X i The number of samples when taking the kth value; q i Indicated as X i The number of parent nodes; r i Indicated as X im represents the number of samples in the dataset D.

[0030] As a further improvement of the present invention, in the parameter learning process, the maximum likelihood estimation method is used to select the parameter value θ that maximizes the probability of the original data appearing. * ; comprising the following steps:

[0031] S1: For the data set D, the degree of fit between a certain value θ0 of the parameter θ and the data set D is measured by the conditional probability P(D|θ=θ0).

[0032] S2: Given θ, the conditional probability P(D|θ) of a dataset D is called the likelihood of θ. For a given dataset D, θ can take different values on its domain, and thus the likelihood function L(θ|D) of θ is constructed:

[0033] L(θ|D)=P(D|θ),

[0034] S3: Find the value of θ that can make the likelihood function L(θ|D) reach its maximum value, that is, θ * :

[0035] θ * =argmax θ L(θ|D).

[0036] As a further improvement of the present invention, the first Bayesian network is obtained by iterative optimization under the condition of a specified seed.

[0037] Accordingly, the method for determining whether the desensitized data set generated by the forward sampling method is qualified includes the following steps:

[0038] S01: Specifies the first seed in the seed set.

[0039] S02: Use the approximate solution method based on scoring search to perform structured learning on the desensitized dataset, and then iteratively optimize the directed acyclic graph in the second Bayesian network.

[0040] S03: Determine whether the structure of the second Bayesian network is consistent with that of the first Bayesian network based on the edges in the directed acyclic graph:

[0041] (1) The parameters of the desensitized data set are learned through the maximum likelihood algorithm, and the joint probability distribution between the indicators in the second Bayesian network is obtained.

[0042] (2) Otherwise, the desensitized dataset is unqualified.

[0043] S04: Determine whether the parameters of the second Bayesian network are consistent with those of the first Bayesian network based on the joint probability distribution:

[0044] (1) If yes, it means the current desensitized data set is qualified and the corresponding result is returned.

[0045] (2) Otherwise, the desensitized dataset is unqualified.

[0046] S05: When the generated desensitized data set fails, check whether the currently used seed is the last seed in the seed set:

[0047] (1) If yes, it means the generation of the desensitized dataset failed.

[0048] (2) Otherwise, select the next seed in the seed set and return to step S2 to regenerate the directed acyclic graph of the second Bayesian network based on the desensitized data set.

[0049] The present invention also includes a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the aforementioned Bayesian network-based medical structured data desensitization method, and then generates a corresponding desensitized data set based on the original medical structured data.

[0050] The present invention also includes a storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the aforementioned Bayesian network-based medical structured data desensitization method, and then generates a corresponding desensitized data set based on the original medical structured data.

[0051] The present invention also includes a computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When executed by the processor, the computer program implements the steps of the aforementioned Bayesian network-based medical structured data desensitization method, thereby generating a corresponding desensitized dataset based on the original medical structured data.

[0052] The present invention has the following beneficial effects:

[0053] The present invention provides a desensitization method for medical structured data based on a Bayesian network, and provides a way to deploy and implement the scheme in a database. The scheme generates the required desensitized data set by building a Bayesian network model of the original data set, and ensures the similarity between the desensitized data and the original data by analyzing the consistency of the Bayesian network model of the two data sets in structure and parameters. On this basis, the present invention also utilizes the PL / R extension of PostgreSQL to realize integrated deployment in the database, so that the entire desensitization process can be completed without data export, to further improve the security of the desensitized data generation process.

[0054] The technical solution provided by the present invention can fully protect data privacy in two dimensions: data desensitization processing and data security operation, while retaining the correlation between the original data features to the greatest extent possible. The desensitized data set generated by this solution shows a high degree of similarity with the original data set in terms of feature correlation, feature distribution, significance test, and predictive performance of the machine learning model. The desensitized data set generated using the solution of the present invention has the performance equivalent to the original data set and has the potential to replace the original data set for data mining and analysis. It is expected to provide a new solution for medical data auditing and the in-depth application of data-driven medical artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flowchart of the steps of the Bayesian network-based medical structured data desensitization method provided in Example 1 of the present invention.

[0056] Figure 2 A diagram of a typical Bayesian network generated based on medical structured data.

[0057] Figure 3 This is a flowchart of iteratively generating a directed acyclic graph in a Bayesian network using a hill climbing algorithm in Example 1 of the present invention.

[0058] Figure 4 This is a flowchart for determining whether the desensitized data set generated by the forward sampling method is qualified in Example 1 of the present invention.

[0059] Figure 5 This is the overall architecture of the DB4AI system used in Example 2 of the present invention.

[0060] Figure 6 The structure of the Bayesian network generated based on the original dataset in the test experiment.

[0061] Figure 7 This is the structure of the Bayesian network generated based on the desensitized dataset when the random seed is 1 in the test experiment.

[0062] Figure 8 The structure of the Bayesian network generated based on the desensitized dataset when the random seed is 10 in the test experiment.

[0063] Figure 9 The structure of the Bayesian network generated based on the desensitized dataset when the random seed is 17 in the test experiment.

[0064] Figure 10 This is the feature correlation heat map of the original dataset in the test experiment.

[0065] Figure 11 This is a heat map of feature correlation of the desensitization dataset in the test experiment.

[0066] Figure 12 This is the histogram distribution of Clucose features in the original dataset and desensitized data in the test experiment.

[0067] Figure 13 The histogram of BMI features in the original data set and desensitized data in the test experiment. DETAILED DESCRIPTION

[0068] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0070] Example 1

[0071] The present embodiment proposes a desensitization method for medical structured data based on Bayesian networks (Bayesian Networks, BN), and the method first converts medical structured data into discretized data that meets the node attribute regulations in the Bayesian network.Then the statistics and association attributes of the original data set are learned by the approximate solution method of score search and the maximum likelihood algorithm, and a Bayesian network comprising the correlation relationship between indicators and the conditional probability distribution corresponding thereto is obtained.Finally, the Bayesian network is utilized to generate a desensitized data set with feature probability similarity to the original data set.It should be emphasized that, in order to ensure that the feature probability similarity between the desensitized data set generated and the original data set meets the requirements, the present embodiment also uses the same scheme to generate the Bayesian network corresponding to the desensitized data set, and performs structure and parameter judgment.If the Bayesian network structure and parameters corresponding to the two data sets are consistent, then the desensitized data set generated is described to meet the requirements, otherwise the desensitized data set generated is described to be non-compliant.

[0072] Specifically, if Figure 1 As shown, the desensitization method for medical structured data based on Bayesian networks provided in this embodiment includes the following four stages:

[0073] 1. Data Preprocessing

[0074] A large amount of medical structured data is collected, and each piece of medical structured data is preprocessed to generate feature variables containing candidate values of various indicator items, thereby obtaining an original data set consisting of multiple feature variables.

[0075] In actual applications, the types of medical structured data collected from users are diverse, including natural discrete data such as age and gender, as well as continuous features such as blood pressure, height, weight, and blood sugar content. In order to simplify the construction of the Bayesian network, it is necessary to discretize the continuous features in the data preprocessing stage, and then complete the Bayesian network construction and the generation of the desensitized data set. In this embodiment, the preprocessing method of medical structured data includes the following:

[0076] (1) When the candidate value of any indicator item is discrete data and the number of candidate values is not higher than the upper limit specified by the Bayesian network node attributes, the original candidate value of the indicator item is retained to form the corresponding feature variable.

[0077] Indicators that do not require discretization are those whose candidate values are naturally discrete and whose domain of definition is small. Typical examples of such indicators include the user's age and the number of physical examinations the user has had in the past year. For example, the indicator item age has only two candidate values: male or female. In this embodiment, these two states can be represented by eigenvalues 1 and 2. Therefore, these indicators can be left unprocessed.

[0078] (2) When the candidate value of any indicator item is continuous data, or the candidate value of the indicator item is discrete data but the number of candidate values is higher than the upper limit specified by the Bayesian network node attribute, the original candidate value of the indicator item is mapped to a discrete interval that meets the requirements through the chiMerge algorithm, and then a feature variable composed of the new candidate value is generated.

[0079] There are two types of indicators that require discretization. One type refers to indicators whose candidate values are continuous. For example, for a user's blood pressure and blood sugar levels, regardless of detection accuracy, the diastolic and systolic blood pressures can actually take any value between their respective upper and lower limits. The other type refers to indicators whose candidate values are discrete but whose domains are relatively large. For example, for the indicator item age, although its candidate values are discrete and integer, its domain can range from 0 to over 100, protecting more than 100 candidate values. For these two indicators, it is generally impossible to construct the required Bayesian network based on their original values. Therefore, this embodiment discretizes them, mapping the candidate values of each indicator to a narrower discrete interval. For example, for age, the user's actual age value can be mapped to a corresponding age interval according to multiple intervals such as 0-10, 10-20, 20-30, etc.

[0080] In actual application, when preprocessing medical structured data, this embodiment needs to screen whether the features need to be discretized. By factoring the continuous and discrete features in the original data set, if the factorization level exceeds a certain threshold value, discretization processing will be performed. The threshold value can be reasonably set according to the actual situation.

[0081] Data can be discretized using the chi-square binning (chiMerge) method to obtain the discretized features of the original dataset. The chiMerge method merges adjacent intervals based on the similarity of feature class distributions, reducing feature value states and data complexity. During processing, the chi-squared value is used to measure interval similarity, preserving the inherent characteristics and relationships of the data and minimizing information loss. The binning process also has clear stopping conditions, preventing over-binning or under-binning, thereby improving the performance and stability of subsequent Bayesian network modeling.

[0082] 2. Building a Bayesian Network

[0083] In combination with the previous content, it can be seen that the present invention intends to perform feature learning on the original data set, generate a Bayesian network that reflects the joint probability distribution between the indicator items in the original data set, and then use this Bayesian network to assist in generating a desensitized data set with a high degree of similarity in feature probability with the original data set. The desensitized data set is then applied to certain machine learning tasks, thereby avoiding direct application of user privacy information while meeting the quality of the data set.

[0084] The Bayesian network is essentially a probabilistic graph model, which establishes the dependency relationship between attribute variables through prior knowledge and data learning, and then obtains the conditional probability. Figure 2 As shown in Figure 1, a Bayesian network is a directed acyclic graph (DAG) consisting of nodes (representing attribute variables) and directed edges connecting the nodes (representing dependencies). It graphically and quantitatively describes the complex relationships between attribute variables. This probabilistic graphical model can provide a logically clearer and more understandable framework for high-dimensional data analysis than conventional methods.

[0085] A Bayesian network can be formalized as a binary pair<G,Θ> . Where G represents a DAG network structure, G =<X,A> . X represents the set of nodes in the network, X i Represents a node in the node set, corresponding to the attribute variable in the medical structured data, X i ∈X; A is the set of directed edges in the network, a ij Represents node X i and node X j A directed connection between them, and is used to represent the direct dependency relationship between them, a ij∈A. Θ is a network parameter, such as the probability value of the node, θ i Represents the node X i The associated conditional probability distribution function, θ i ∈Θ.

[0086] Bayesian networks imply the conditional independence assumption, which states that given a node's set of parent nodes, the node is independent of all its non-descendant nodes. Therefore, the joint probability distribution of all nodes represented by the Bayesian network can be expressed as the product of the conditional probabilities of each node, as shown below:

[0087]

[0088] In the above formula, n represents the number of nodes in the Bayesian network; π(X i ) represents X i The parent node set of .

[0089] Therefore, a Bayesian network can be viewed as a graphical structure that represents the joint probability distribution between attribute variables. To obtain this structure and its parameter distribution, it is necessary to complete structural learning and parameter learning of the Bayesian network based on the original data. Structural learning can obtain the dependencies between nodes in the Bayesian network based on the original data set, thereby obtaining a corresponding G. Parameter learning can further obtain the joint probability distribution of each node in G, that is, the corresponding Θ. The following details the process of structural learning and parameter learning:

[0090] 2.1 Structure Learning

[0091] The approximate solution method based on scoring search performs structured learning on each feature variable in the original data set, and then iteratively optimizes a directed acyclic graph with each indicator item as a node and the relationship between the indicator items as an edge.

[0092] The goal of structural learning is to find a DAG network structure that best fits a given dataset. However, strictly extracting a Bayesian network from data has been proven to be an NP (Non-deterministic Polynomial) problem. Therefore, this embodiment uses an approximate solution method based on scoring search to obtain a highly fitting DAG network. This approximate solution method based on scoring search is an iterative optimization algorithm. It first determines a scoring function to evaluate the degree of fit between the DAG structure and the data sample. Then, based on the value of the scoring function, it employs a specific search strategy to continuously reconstruct the DAG structure across all possible solution spaces, ultimately selecting the network structure with the best score as the final optimization result.

[0093] In order to apply the relevant algorithms to optimize the DAG network, it is first necessary to design a corresponding scoring function. In this embodiment, the scoring function design process is as follows:

[0094] For a known data set D, assuming that the prior probability of the BN structure is P(G), then its posterior probability P(G|D) is:

[0095]

[0096] Among them, the marginal probability P(D) of the data is a normalized constant independent of G; P(D|G) is the likelihood probability of the data set D. When P(G)P(D|G) reaches the maximum value, the network structure is the optimal solution.

[0097] Taking the logarithm of P(G)P(D|G) yields the scoring function logP(G,D) based on Bayesian statistics:

[0098] logP(G,D)=logP(G)P(D|G)=logP(G)+logP(D|G),

[0099] The main idea of Bayesian scoring is to select the network structure G with the maximum a posteriori probability (MAP) given the prior knowledge and original data. * ,Right now:

[0100]

[0101] Therefore, the application of the Bayesian scoring function requires first determining the prior distribution of the original data, which often presents certain difficulties in the actual data desensitization process. If a specific distribution is assumed, such as a uniform distribution, the result of statistical smoothing will distort the Bayesian score value of the complex structure. For this purpose, the likelihood function is introduced, and the calculation is based on the conditional probability of the original data. The value of the likelihood function represents the degree of fit between the candidate network model and the statistical distribution of the original data. In order to prevent data overfitting, a penalty term is added to the likelihood function calculation, thereby forming the Bayesian Information Criterion (BIC) scoring function F shown in the following formula: BIC (G|D):

[0102] F BIC (G|D)=S LL (D|G)+Δ(G|D)

[0103] Among them, S LL (D|G) is the log-likelihood function used to describe the degree of fit of graph G to dataset D; Δ(G|D) is a penalty term that describes the complexity of the model, which satisfies:

[0104]

[0105] Where m represents the number of samples in the dataset D; d is the number of independent parameters; it satisfies:

[0106]

[0107] In the above formula, q i Indicated as X i The number of parent nodes; r i Indicated as X i The number of values ​​of .

[0108] In summary, the BIC scoring function F is obtained to evaluate the degree of fit of the generated directed acyclic graph. BIC (G|D) is as follows:

[0109]

[0110] In the above formula, m ij Represents the i-th variable X in the dataset D i The number of samples when the parent node takes the jth value; m ijk Represents the i-th variable X in the dataset D i The parent node takes the jth value and X i The number of samples when taking the kth value.

[0111] In this embodiment, the BIC scoring function F designed above can be combined BIC (G|D) uses the Hill-Climbing algorithm (HC) to iteratively generate the directed acyclic graph G in the Bayesian network. Specifically, Figure 3 As shown, the algorithm execution process includes:

[0112] (1) Initialization

[0113] With each indicator item as a node, initialize an edgeless blank graph G0 as the initial network. Preset the number of candidate networks and the maximum iteration round N max A BIC scoring function is selected for evaluating the degree of fit of the generated directed acyclic graph. The scoring function in this embodiment adopts the aforementioned Bayesian Information Criterion scoring function.

[0114] (2) Iterative Search

[0115] Through the following multiple rounds of iterative search, directed edges are gradually added between nodes in the initial network to achieve network structure optimization:

[0116] (a) In each iteration, the initial network is first processed by the local graph search operator to update the edges between nodes, thereby generating multiple candidate networks.

[0117] Among them, the search operators adopted by the hill climbing algorithm of this embodiment include three categories, including adding directed edges, deleting directed edges and reversing directed edges. Among the operation strategies of the above three search operators, the "add directed edge" operator refers to adding a directed edge between any two nodes in the original network that do not have a directed edge. The "delete directed edge" operator refers to deleting an existing directed edge between any two nodes in the original network. The "reverse directed edge" operator refers to reversing the direction of the directed edge between any two nodes in the original network. Among them, in the early stage of the algorithm execution, the operator for deleting directed edges is usually not executed, but in the later stage of the algorithm execution, the three operators need to be flexibly applied so that the network structure can be optimized during the iteration process.

[0118] (b) After each round of update, the BIC score function is used to generate the BIC score of each candidate network and make the following decisions:

[0119] If the BIC score of any candidate network is higher than that of the original initial network, it will be used as the initialization network for the next round;

[0120] Otherwise, the original initial network is retained as the initial network for the next round.

[0121] Based on the above decisions, in each round of iteration, each candidate network with a more optimized network structure can always be saved, and then gradually approach the required Bayesian network structure.

[0122] It's also important to note that the initial network is a single, blank graph with no edges. Therefore, in the first round of the algorithm, to ensure that the number of candidate networks in subsequent rounds meets the preset parameters, all candidate networks generated from the blank graph using the search operator are retained. In subsequent iterations, only candidate networks with relatively high BIC scores before and after the update are retained.

[0123] (3) Optimize output

[0124] When the preset maximum number of iterations is reached or the BIC scoring function converges, the search process ends and the network with the highest BIC score in the candidate network set is returned as the required directed acyclic graph.

[0125] The above content describes the process of iteratively searching for the structure of the Bayesian network using the hill climbing algorithm in this embodiment. However, the hill climbing algorithm is only one example of an approximate solution method based on scoring search that can be used by technicians. In actual application, technicians can also use other iterative optimization algorithms to generate the required Bayesian network structure.

[0126] 2.2 Parameter Learning

[0127] After determining the structure of the Bayesian network (i.e. the optimized DAG obtained in the previous text), parameter learning is performed on this basis, and the conditional probability distribution of each node in the network structure is estimated from the original data. Specifically, the parameter learning of this embodiment adopts the maximum likelihood estimation method, selects the parameter value that makes the original data have the largest probability, and then obtains the joint probability distribution of all nodes, and generates a complete Bayesian network. Wherein, in order to distinguish it from the Bayesian network established by the desensitized data set in order to realize verification in the following text, the Bayesian network established by the original data set in this stage in this embodiment is called the first Bayesian network, and accordingly, the Bayesian network generated according to the desensitized data set is called the second Bayesian network.

[0128] Specifically, this embodiment uses the maximum likelihood algorithm to perform parameter learning on each feature variable in the original data set, uses sample data to estimate the conditional probability distribution of each node in the directed acyclic graph, and then obtains a first Bayesian network containing the joint probability distribution between the indicator items corresponding to all nodes. The construction process of the first Bayesian network includes the following steps:

[0129] For the data set D, the conditional probability P(D|θ=θ0) is used to measure the degree of fit between a certain value θ0 of the parameter θ and the data set D.

[0130] Given θ, the conditional probability P(D|θ) of the dataset D is called the likelihood of θ. For a given dataset D, θ can take different values on its domain, and thus the likelihood function L(θ|D) of θ is constructed:

[0131] L(θ|D)=P(D|θ),

[0132] Solve the value of θ that can make the likelihood function L(θ|D) reach its maximum value, that is, θ * :

[0133] θ * =argmax θ L(θ|D).

[0134] 3. Desensitized Data Generation

[0135] Through the processing of the above steps, this embodiment has generated a first Bayesian network based on the preprocessed medical structured data, which can reflect the joint probability distribution of each indicator item in the original data. On this basis, this embodiment uses forward sampling (FS) to generate multiple desensitized data based on the first Bayesian network, thereby forming the required desensitized data set; since the desensitized data is estimated using the first Bayesian network rather than derived from real data, the desensitized data can maintain statistical characteristics and association attributes similar to the original data set, thereby improving the availability of the data while "hiding" the privacy information in the original medical structured data.

[0136] Among them, the forward sampling method is a sampling method used in probabilistic graphical models, mainly used for approximate inference. Its basic idea is to sample from the target distribution and then use the sampled data to estimate the expected value of the target function. The key to the forward sampling method is how to effectively generate samples from the posterior distribution. Specifically, in this embodiment, the process of generating desensitized data by the forward sampling method is shown in Table 1, including the following process:

[0137] Starting from the root node of the first Bayesian network, the node states are sampled one by one in the order of topological sorting; a state is randomly selected as the state of the child node based on the states and conditional probability distribution of all parent nodes, and this process is repeated until the states of all nodes are determined; the characteristic variables of the corresponding indicator items are determined according to the state of each node, and then a piece of desensitized data is obtained.

[0138] Table 1: Forward sampling algorithm flow in Bayesian network

[0139]

[0140]

[0141] 4. Attribute Correlation Verification

[0142] In this embodiment, technicians found that the network structure generated by the desensitized data obtained by the forward sampling algorithm is not always consistent with the Bayesian network structure of the original data set. Given the key role of network structure in characterizing the underlying patterns of data, in order to ensure that the generated desensitized data has higher usability, the Bayesian network structure of the sampled desensitized data is re-analyzed to further improve the data quality. The verification process is as follows:

[0143] A second Bayesian network representing the joint probability distribution of the desensitized data is generated again using the approximate solution method of the scoring search and the maximum likelihood algorithm; and the structures and parameters of the first and second Bayesian networks are determined to be consistent:

[0144] (1) If yes, it means the generated desensitized dataset is qualified.

[0145] (2) If not, the generated desensitized data set is judged to be unqualified, and the medical structured data is re-processed, structured learning, parameter learning and probability distribution verification are performed until a qualified desensitized data set is obtained.

[0146] It should also be noted that during the construction of the first Bayesian network or the second Bayesian network, the structural optimization algorithm needs to set a random seed in the initialization phase to guide the iterative process. The first Bayesian network is obtained by iterative optimization under the specified seed. Combined with the forward sampling algorithm, it can be seen that the sampled data is affected by random initialization. Different random number seeds lead to different sampling initialization processes and result in differences in the desensitized data sets generated by different sampling batches. In order to obtain desensitized data that maximizes the retention of the original data feature relationships, it is necessary to search for the optimal random number seed.

[0147] Therefore, if Figure 4 As shown, the method provided in this embodiment for determining whether the desensitized data set generated by the forward sampling method is qualified includes the following steps:

[0148] S01: Set a seed set containing all seeds and specify the first seed in the seed set.

[0149] S02: Use the approximate solution method based on scoring search to perform structured learning on the desensitized dataset, and then iteratively optimize the directed acyclic graph in the second Bayesian network.

[0150] S03: Determine whether the structure of the second Bayesian network is consistent with that of the first Bayesian network based on the edges in the directed acyclic graph:

[0151] (1) The parameters of the desensitized data set are learned through the maximum likelihood algorithm, and the joint probability distribution between the indicators in the second Bayesian network is obtained.

[0152] (2) Otherwise, the desensitized dataset is unqualified.

[0153] In this embodiment, the nodes in the first and second Bayesian networks correspond to indicators in the medical structured data, so the nodes contained in the two networks are consistent. Based on this, determining whether the network structures of the two networks are consistent essentially determines whether the directed edges contained in the two networks are completely consistent. After structured learning, this embodiment can obtain a list of directed edges in the second Bayesian network, compare it with the list of directed edges in the first Bayesian network, and determine whether the structures of the two networks are consistent.

[0154] S04: Determine whether the parameters of the second Bayesian network are consistent with those of the first Bayesian network based on the joint probability distribution:

[0155] (1) If yes, it means the current desensitized data set is qualified and the corresponding result is returned.

[0156] (2) Otherwise, the desensitized dataset is unqualified.

[0157] When any two Bayesian networks have the same structure, we compare the joint probability distributions between their nodes, determined after parameter learning. If these parameters are also identical, then the two Bayesian networks are identical. This also indicates that the datasets used to generate the two Bayesian networks are highly similar.

[0158] S05: When the generated desensitized data set fails, check whether the currently used seed is the last seed in the seed set:

[0159] (1) If yes, it means the generation of the desensitized dataset failed.

[0160] (2) Otherwise, select the next seed in the seed set and return to step S2 to regenerate the directed acyclic graph of the second Bayesian network based on the desensitized data set.

[0161] After traversing all the seeds in the seed set, it was found that all the second Bayesian networks generated based on the desensitized dataset could not be guaranteed to be consistent with the first Bayesian network in both structure and parameters, which means that the datasets used to generate the two Bayesian networks are also dissimilar.

[0162] Example 2

[0163] Example 1 provides a desensitization method for medical structured data based on a Bayesian network. On this basis, this embodiment is further based on the theory of in-database machine learning (IDML) and implements this method in the database using the PL / R extension of PostgreSQL to ensure that data operations and data desensitization processing are under strict supervision of the database.

[0164] Specifically, the scheme in Example 1 accurately depicts the intrinsic structure of multidimensional data by learning the statistical and association attributes of the original data and establishing the probabilistic dependency relationship between the attribute variables, and obtains the Bayesian network structure set of the desensitized data and the original data by comparing the forward sampling, and screens out the desensitized data set with the same structure and characteristic probability similarity with the original data. The scheme of this embodiment further utilizes the efficient computing and storage capabilities of the database management system, and realizes the construction of the Bayesian network and the generation of desensitized data in the database through the PL / R extension, which significantly improves the desensitization efficiency and provides reliable technical support for the privacy protection and sharing of medical structured data.

[0165] 1. DB4AI Architecture

[0166] In the 1960s, with the advancement of computer technology and the growing demand for data processing, early file systems gradually began to exhibit problems such as data redundancy and inconsistency, leading to the development of database management systems. These systems not only ensure data consistency and security but also significantly improve the efficiency of data management and retrieval. In the field of medical data analysis, databases provide a centralized and organized storage environment for medical big data. AI can uncover hidden patterns and trends within complex medical data, providing powerful support for early disease detection and the planning and implementation of treatment plans. Data analysis is inseparable from databases and AI technologies, which work together and complement each other, driving the continuous development and innovation of medical data analysis technology.

[0167] DB4AI (Database for AI) refers to a research field that closely integrates database technology with AI, aiming to leverage the database's efficient data management and computing capabilities to support the training, reasoning, and deployment of AI models. The DB4AI system architecture enables efficient and secure machine learning and data analysis within the database through collaborative work at multiple levels. Figure 5 DB4AI first provides a standard SQL interface through the user interface layer, allowing users to interact with the system using the SQL query language. Furthermore, the system is compatible with programming languages and machine learning frameworks such as Python, TensorFlow, and R, providing users with a diverse selection of tools for data analysis and model building.

[0168] At the core AI framework layer, the system integrates multiple AI frameworks and interacts with them through Python / R interfaces, supporting the execution of machine learning algorithms and data processing. Users can flexibly select frameworks and invoke corresponding algorithms based on task requirements. The user structure layer focuses on the management and interaction of user data, organizing data and providing access control mechanisms to ensure data security. To achieve efficient data processing, DB4AI leverages built-in RDBMS functions and the underlying database optimization algorithm layer to quickly execute queries and computations, improving processing efficiency. The C++ interface layer bridges the gap between the high-level AI framework and the underlying database system, implementing interaction between the two in C++ to ensure efficient and stable system operation. The data flow layer manages data transmission and processing within the system, ensuring accurate data transfer from the user interface to the underlying database layer across all layers, ensuring data accuracy and consistency. The underlying abstraction layer abstracts underlying database operations, providing basic functions such as matrix operations. It also connects to the RDBMS through C++, facilitating database access and operation by upper-level applications. Finally, the query process and data integrity and security check layer of the RDBMS are the key to system reliability and security. They are responsible for processing user query requests and performing data integrity and security checks to prevent data corruption and unauthorized access.

[0169] Traditional machine learning tasks often involve multiple processes, including data collection, data cleaning, data calculation and analysis, and data storage. The algorithm training process also includes data preprocessing, training, model storage, and management. The model training process also includes hyperparameter tuning. Through the DB4AI system architecture, the entire lifecycle of machine learning models, such as these, can be largely integrated within the database. Model training, management, and optimization processes are performed closest to the data storage side, providing convenient, ready-to-use AI full-declaration lifecycle management capabilities in the form of SQL statements on the database side. This not only improves data processing efficiency, but also effectively protects the privacy and security of medical data, and has far-reaching significance for medical data analysis.

[0170] In-database machine learning refers to the training, evaluation, and execution of machine learning models directly within the database management system, without exporting data to an external machine learning platform [5]. Compared with traditional machine learning, IDML has the advantage of being able to fully utilize the high-performance storage and computing capabilities of the database system, as well as its rich data management functions. At the same time, IDML can significantly reduce the cost and complexity of data movement and improve the efficiency and security of data processing. In-database machine learning provides the foundation for DB4AI. It integrates machine learning algorithms into the database system, giving the database native intelligent computing capabilities, covering key links such as data storage and model building, thereby enabling DB4AI to perform more efficient data processing and analysis.

[0171] 2. PL / R Extensions

[0172] The PostgreSQL database is renowned for its rich plugin ecosystem and highly extensible nature. These plugins add diverse functionality to the database, enabling it to meet the needs of diverse applications. These include the PostGIS plugin for geographic information systems (GIS), which provides geographic object support; the pg_stat_statements plugin for performance monitoring and optimization; and the pgAudit plugin for security auditing. Users can select appropriate plugins based on their business needs and, by installing and configuring them, enhance the database's functionality and performance.

[0173] As a powerful and highly scalable relational database management system, PostgreSQL has won wide recognition for its excellent performance, stability and rich feature set. However, when faced with complex data analysis tasks, traditional SQL queries may seem inadequate. PostgreSQL has built-in simple statistical functions, but no good machine learning library. PL / R and PL / Python are both extension languages in the DB4AI architecture of the PostgreSQL database, used to write stored procedures and functions. PostgreSQL uses the plpython3u / plr plug-in to write user-defined functions (UDF), that is, a SQL statement framework plus standard Python / R functions, to call the machine learning library to implement machine learning processing of data in the database. This organically links machine learning with PostgresSQL data storage and retrieval, and realizes complex calculations within the database. Compared with PL / Python, PL / R has the following advantages:

[0174] (1) Relatively reliable extension mechanism: Although PL / R also has similar security issues, due to the professionalism of the R language and its widespread application in data analysis, its related functions and libraries are generally of high quality and security. In addition, the PostgreSQL community is continuously strengthening support and security reviews for PL / R to ensure its stable and secure operation in the database environment.

[0175] (2) Independent Data Processing Environment: PL / R can create a relatively independent R language data processing environment within PostgreSQL, isolating the execution of R code from the core functionality of PostgreSQL. This isolation mechanism can prevent errors or security issues in the R language from propagating to other parts of the database, thereby improving the stability and security of the entire system.

[0176] (3) Security differences in data processing and privacy protection: Python provides a wealth of data processing and analysis tools, but when processing sensitive data, developers need to implement encryption, access control, and other measures to ensure data security and privacy. The R language has a deep foundation in statistical analysis and data processing, and many of its built-in functions and methods have been rigorously tested and verified. At the same time, R also provides some data encryption and privacy protection functions, but these also require developers to use them correctly.

[0177] PL / R, a programming language extension designed specifically for PostgreSQL, embeds the powerful statistical analysis and data mining capabilities of the R language directly into the database system, allowing users to execute R scripts within the database for complex data analysis and processing. PL / R makes it possible to directly call R functions and libraries from within the PostgreSQL environment. This integration greatly simplifies the process of performing complex statistical analysis and machine learning tasks within the PostgreSQL database, while also improving processing efficiency and security. When calling PL / R functions, the argument values provided to these functions are automatically converted into a format that R can understand, ensuring smooth data transfer and processing between the two environments, as shown in Table 2.

[0178] Table 2: PostgreSQL and R language variable correspondence table

[0179]

[0180] The PL / R execution process is as follows: First, PL / R parses and compiles R code to generate executable bytecode or machine code. Then, based on the requirements in the R code, PL / R retrieves the required data from the database. This typically involves executing SQL statements and transferring data. After retrieving the data, PL / R executes the R code within the R interpreter to process and analyze the data. Finally, PL / R returns the R code execution results to the database for subsequent storage or query operations.

[0181] Specifically, the specific usage of PL / R is: creating UDF, storing UDF, and calling UDF. Its function architecture pseudo code is shown in Figure 3.

[0182] Table 3: PL / R function architecture pseudocode

[0183]

[0184] By executing AI operations directly in the database, PL / R significantly reduces data transmission and processing overhead, thereby improving overall performance and streamlining the development process. Developers no longer need to maintain separate AI services; all operations can be completed within the database, reducing the complexity of development and deployment. Furthermore, because data never needs to leave the database, PL / R effectively reduces the risk of data leakage and enhances data security, which is particularly important for processing sensitive data (such as in healthcare, finance, and other fields). PL / R also supports real-time AI processing, enabling developers to perform AI analysis as data is being stored, supporting real-time decision-making.

[0185] Example 3

[0186] In order to better apply the Bayesian network-based medical structured data desensitization method provided in Example 1, this embodiment further provides a computer program product, a storage medium, and a corresponding computer device capable of implementing the above method. Specifically, this embodiment provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned Bayesian network-based medical structured data desensitization method, and then generates a corresponding desensitized data set based on the original medical structured data.

[0187] This embodiment also includes a storage medium storing a computer program. When the computer program is executed by a processor, the steps of the aforementioned Bayesian network-based medical structured data desensitization method are implemented, and a corresponding desensitized data set is generated based on the original medical structured data.

[0188] This embodiment also includes a computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When executed by the processor, the computer program implements the steps of the aforementioned Bayesian network-based medical structured data desensitization method, thereby generating a corresponding desensitized dataset based on the original medical structured data.

[0189] In actual applications, the computer device can be an embedded device and deployed in various terminal devices to support data processing and interaction. It can also be used as an independent computer device to support data processing needs in certain scenarios. This non-embedded computer device can be a laptop, tablet computer, desktop computer, or a medium or large computer device such as a rack server, blade server, tower server, or cabinet server (including a standalone server or a server cluster consisting of multiple servers) that can execute computer programs.

[0190] Specifically, the computer device of this embodiment includes at least, but is not limited to, a memory and a processor that can be interconnected via a system bus. In this embodiment, the memory (i.e., a readable storage medium) includes flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory can be an internal storage unit of the computer device, such as the hard disk or internal memory of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk equipped with the computer device, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, etc. Of course, the memory can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is generally used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or are about to be output.

[0191] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is generally used to control the overall operation of a computer device.

[0192] Performance Testing

[0193] In order to verify the performance of the solution of the present invention, the technical staff developed an experimental plan and conducted simulation tests on the relevant solutions. The experimental process is as follows:

[0194] 1. Experimental environment and experimental data

[0195] This experiment was conducted on a Microsoft Windows 11 platform using the database management tool Navicat Premium (connected to PostgreSQL). The specific experimental environment configuration is shown in Table 4.

[0196] Table 4: Environment configuration in simulation experiment

[0197]

[0198] To verify the feasibility and effectiveness of the proposed method, we used the Pima Indian Diabetes Dataset from the UCI Machine Learning Database. This publicly available electronic health record dataset, designed to support cross-sectional clinical research, contains attribute features of various data types, dimensions, and parameter distributions, totaling 768 samples. As shown in the table below, each sample contains eight attribute features and one categorical feature.

[0199] Table 5: Datasets corresponding to medical structured data collected from diabetic patients

[0200]

[0201] To facilitate visualization of the Bayesian network structure, some features of the dataset are abbreviated. For example, the feature "Pregnancies" in the table is abbreviated as "Pr", the feature "BloodPressure" in the table is abbreviated as "BP", the feature "SkinThickness" in the table is abbreviated as "ST", and the feature "DiabetesPedigreeFunction" in the table is abbreviated as "DPF".

[0202] 2. Desensitized data generation

[0203] This experiment uses PL / R to implement a user-defined function called "desensitization." This function accepts the original dataset as a parameter and outputs a desensitized dataset. After executing the function, two output forms are generated: (1) the desensitized dataset is stored as a view in the database for easy subsequent query and call; (2) a physical file copy of the desensitized data is saved on the local disk to support further verification and analysis. The corresponding desensitized dataset can be obtained by running SQL statements.

[0204] 3. Analysis of experimental results

[0205] This experiment compares the Bayesian networks constructed based on the original dataset and the desensitized dataset, and tests the feature correlation, feature distribution, Kolmogorov-Smirnov test, and consistency of machine learning prediction results between the original and desensitized datasets.

[0206] (3.1) Structure of Bayesian Network

[0207] In this experiment, the structure learning results of the original dataset and the desensitized dataset are as follows: Figure 6-Figure 9 As shown. Among them, Figure 6 is the Bayesian network structure of the original data set, Figure 7-Figure 9These are the Bayesian network structures of the desensitized datasets obtained when the random number seeds are 1, 10, and 17. According to the above method, the desensitized dataset with the same Bayesian network structure as the original dataset is finally selected.

[0208] (3.2) Feature correlation heatmap

[0209] According to the experimental results, the feature correlation heat maps of the original dataset and the desensitized dataset were further generated. The experimental results were as follows: Figure 10 and Figure 11 shown.

[0210] According to the correlation heatmap analysis, the numerical difference in feature correlation between the desensitized dataset and the original dataset is mostly between 0.01 and 0.1, maintaining a high degree of similarity. In particular, between features with direct dependencies in the Bayesian network structure, the correlation heatmap of the original and desensitized data shows that the correlation coefficients between the features DPF and Pr are 0.53 and 0.56, respectively; the correlation coefficients between the features Outcome and Glucose are 0.49 and 0.46, respectively; and the correlation coefficients between the features ST and BP are 0.50 and 0.52, respectively, indicating a strong correlation between the features of the two datasets. This also shows that, while ensuring the consistency of the Bayesian network structures of the original and desensitized data, the desensitized dataset well preserves the correlation between the features of the original data.

[0211] (3.3) Feature distribution

[0212] According to the experimental results, the feature distribution histograms of some features (Clucose and BMI) in the original dataset and the desensitized dataset were further generated. The experimental results obtained are as follows: Figure 12-13 As shown, the left side shows the corresponding feature distribution histograms in the original dataset, and the right side shows the corresponding feature distribution histograms in the desensitized dataset.

[0213] Analyzing the feature distribution histograms of original data and desensitized data can intuitively show the distribution of each feature in the dataset. Figure 12-13 The results show that the distribution of the same features in the desensitized dataset is highly similar to that in the original dataset. For example, for the features glucose and BMI, both the classification of the features and the frequency of occurrence of each category in the dataset are consistent between the desensitized and original data. This demonstrates that the desensitized data performs well in simulating the distribution characteristics of the original data, with each feature accurately fitting the distribution patterns of the original data.

[0214] (3.3) Kolmogorov-Smirnov test

[0215] According to the experimental results, the Kolmogorov-Smirnov test significance of some features of the diabetes dataset is further tested. The test results are shown in the following table:

[0216] Table 6: Test significance test results

[0217]

[0218] The Kolmogorov-Smirnov test is a non-parametric test method used to compare the differences between a sample distribution and a theoretical distribution or between two sample distributions. The D and p-values in the Kolmogorov-Smirnov test results are statistics used to measure the differences between the original data and the theoretical distribution or between two groups of samples. The D value, that is, the Kolmogorov-Smirnov statistic, represents the maximum absolute deviation between the empirical cumulative distribution function and the theoretical cumulative distribution function. The p-value represents the probability of observing the current D value or a larger difference under the condition that the null hypothesis (that the two data sets come from the same distribution) is true. The results of the Kolmogorov-Smirnov significance test show that the p-values are all greater than 0.05, indicating that the null hypothesis cannot be rejected, that is, there is insufficient evidence to show that the two data sets come from different distributions. Therefore, the original data set and the desensitized data set are significantly consistent and statistically significant.

[0219] Finally, based on the above experimental data, we can find that: an in-depth comparison between the original dataset and the desensitized dataset in terms of feature correlation, feature distribution, and consistency of machine learning prediction results shows that the desensitized dataset shows extremely similar characteristics to the original dataset in these key indicators.

[0220] 4. Performance comparison

[0221] This experiment further uses the original data set and the desensitized data set for machine learning training. In the experiment, the Bayesian network model scheme of the present invention is used as the experimental group, and the performance is compared with the relevant schemes of the random forest model and the logistic regression model as the control group. The above three models can model the data change process well and are applied to the classification prediction tasks of unbalanced data sets. In the experiment, the original data is randomly divided into a ratio of 3:7, of which 30% is the training set, and the test set includes two parts: 70% of the original data and the generated desensitized data set. Accuracy, Precision, Recall, and F1_score are selected as evaluation indicators in the experiment. In the performance comparison experiment, the performance of the Bayesian network model, random forest module and logistic regression module are shown in Tables 7 to 9.

[0222] Table 7: Performance indicators of Bayesian network models

[0223]

[0224] Table 8: Performance metrics of random forest models

[0225]

[0226]

[0227] Table 9: Performance metrics of the logistic regression model

[0228]

[0229] Analysis of the data in the above tables reveals that the Bayesian network achieved highly consistent classification prediction metrics (such as accuracy and F1 score) on both test sets (relative deviation <5%), demonstrating its strong adaptability to the distribution characteristics of the desensitized data. The cross-dataset metric differences for random forest and logistic regression were slightly higher than those for the Bayesian network, but still met statistical equivalence (p>0.05), indicating that the desensitization process did not significantly disrupt the discriminative consistency of the data features.

[0230] The above comparison results verify that the Bayesian network data desensitization method used in this study can well preserve the key structure of the original data distribution characteristics, and the desensitized data has good equivalence in statistical analysis and machine learning effects.

[0231] The above-described embodiment merely represents one embodiment of the present invention. While the description is relatively specific and detailed, it should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, and these modifications and improvements fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A desensitization method for medical structured data based on Bayesian networks, characterized in that: It includes: Collect a large amount of medical structured data, pre-process each piece of medical structured data to generate feature variables containing candidate values of various indicator items, and then obtain an original data set consisting of multiple feature variables; An approximate solution method based on scoring search performs structured learning on each feature variable in the original data set, and then iteratively optimizes a directed acyclic graph with each indicator item as a node and the relationship between the indicator items as an edge; Performing parameter learning on each feature variable in the original data set using a maximum likelihood algorithm, estimating the conditional probability distribution of each node in the directed acyclic graph using sample data, and thereby obtaining a first Bayesian network containing the joint probability distribution between indicator items corresponding to all nodes; A forward sampling algorithm is used to generate a desensitized data set containing multiple desensitized data based on the first Bayesian network; a second Bayesian network representing the joint probability distribution of the desensitized data is again generated using the approximate solution method of the scoring search and the maximum likelihood algorithm; and a judgment is made as to whether the structures and parameters of the first Bayesian network and the second Bayesian network are consistent: if so, it indicates that the generated desensitized data set is qualified; otherwise, the desensitized data set is judged to be unqualified, and structured learning and parameter learning are performed again until a qualified desensitized data set is obtained.

2. The method for desensitizing medical structured data based on a Bayesian network according to claim 1, characterized in that: Preprocessing methods for medical structured data include: (1) When the candidate value of any indicator item is discrete data and the number of candidate values is not higher than the upper limit specified by the Bayesian network node attribute, the original candidate value of the indicator item is retained to form the corresponding feature variable; (2) When the candidate value of any indicator item is continuous data, or the candidate value of the indicator item is discrete data but the number of candidate values is higher than the upper limit specified by the Bayesian network node attribute, the original candidate value of the indicator item is mapped to a discrete interval that meets the requirements through the chiMerge algorithm, and then a feature variable composed of the new candidate value is generated.

3. The desensitization method for medical structured data based on Bayesian network according to claim 1, characterized in that: The process of generating masked data based on the first Bayesian network using the forward sampling algorithm includes: Starting from the root node of the first Bayesian network, the node states are sampled one by one in the order of topological sorting; a state is randomly selected as the state of the child node based on the states and conditional probability distribution of all parent nodes, and this process is repeated until the states of all nodes are determined; the characteristic variables of the corresponding indicator items are determined according to the state of each node, and then a piece of desensitized data is obtained.

4. The desensitization method for medical structured data based on Bayesian network according to claim 1, characterized in that: The hill climbing algorithm is used to iteratively optimize the directed acyclic graph representing the correlation between indicator items based on the original data set; The algorithm execution process includes:

1. Initialization With each indicator item as a node, a blank graph without edges is initialized as the initial network; the candidate network data and the maximum iteration round are preset, and a BIC scoring function is designed to evaluate the fit of the generated directed acyclic graph; 2. Iterative Search Through multiple rounds of iterative search, directed edges are gradually added between nodes in the initial network to achieve network structure optimization: In each iteration, the initial network is first processed by the local graph search operator, and the edges between nodes are updated to generate multiple candidate networks. Then, the BIC scoring function is used to generate the BIC score of each candidate network: if the BIC score of any candidate network is higher than that of the original initial network, it is used as the initial network for the next round; otherwise, the original initial network is retained as the initial network for the next round. The operation strategies of the search operator include adding directed edges, deleting directed edges and reversing directed edges; 3. Optimize output When the preset maximum number of iterations is reached or the BIC scoring function converges, the search process ends and the network with the highest BIC score in the candidate network set is returned as the required directed acyclic graph.

5. The method for desensitizing medical structured data based on a Bayesian network according to claim 4, characterized in that: The BIC score function F is used to evaluate the fit of the generated directed acyclic graph. BIC (G|D) is as follows: In the above formula, n represents the number of nodes in the candidate network; m ij Represents the i-th variable X in the dataset D i The number of samples when the parent node takes the jth value; m ijk Represents the i-th variable X in the dataset D i The parent node takes the jth value and X i The number of samples when taking the kth value; q i Indicated as X i The number of parent nodes; r i Indicated as X i m represents the number of samples in the dataset D.

6. The method for desensitizing medical structured data based on Bayesian networks according to claim 1, characterized in that: In the parameter learning process, the maximum likelihood estimation method is used to select the parameter value θ that maximizes the probability of the original data appearing. * ; comprising the following steps: S1: For data set D, the degree of fit between a certain value θ0 of parameter θ and data set D is measured by conditional probability P(D|θ=θ0); S2: Given θ, the conditional probability P(D|θ) of a dataset D is called the likelihood of θ. For a given dataset D, θ can take different values on its domain, and thus the likelihood function L(θ|D) of θ is constructed: L(θ|D)=P(D|θ), S3: Find the value of θ that can make the likelihood function L(θ|D) reach its maximum value, that is, θ * : i * =argmax θ L(θ|D).

7. The method for desensitizing medical structured data based on Bayesian networks according to claim 1, characterized in that: The first Bayesian network is obtained by iterative optimization under the condition of a specified seed; The method for determining whether the desensitized data set generated by the forward sampling method is qualified includes the following steps: S01: specifies the first seed in the seed set, S02: performing structured learning on the desensitized dataset using an approximate solution method based on scoring search, and then iteratively optimizing a directed acyclic graph in a second Bayesian network; S03: Determine whether the structure of the second Bayesian network is consistent with that of the first Bayesian network based on the edges in the directed acyclic graph: (1) Parameter learning is performed on the desensitized data set using a maximum likelihood algorithm, thereby obtaining a joint probability distribution between the indicator items in the second Bayesian network; (2) Otherwise, the desensitized dataset is unqualified; S04: Determine whether the parameters of the second Bayesian network are consistent with those of the first Bayesian network based on the joint probability distribution: (1) If yes, it means the current desensitized data set is qualified and the corresponding result is returned; (2) Otherwise, the desensitized dataset is unqualified; S05: When the desensitized dataset fails to meet the requirements, check whether the currently used seed is the last seed in the seed set: (1) If yes, it means the generation of the desensitized dataset failed; (2) Otherwise, select the next seed in the seed set and return to step S2 to regenerate the directed acyclic graph of the second Bayesian network based on the desensitized data set.

8. A computer program product comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the desensitization method for medical structured data based on a Bayesian network as described in any one of claims 1 to 7 are implemented, and a corresponding desensitized data set is generated based on the original medical structured data.

9. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the desensitization method for medical structured data based on a Bayesian network as described in any one of claims 1 to 7 are implemented, and a corresponding desensitized data set is generated based on the original medical structured data.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the computer program is executed by a processor, the steps of the desensitization method for medical structured data based on a Bayesian network as described in any one of claims 1 to 7 are implemented, and a corresponding desensitized data set is generated based on the original medical structured data.

Images