A trusted medical data secure circulation system and method based on privacy computing

Abstract

The present invention discloses a trusted medical data security circulation system and method based on privacy computing, and relates to the field of data security technology. The data provider trains a local model with the original medical data and performs weight clustering to generate cluster centers and weight mappings, and then encrypts the cluster centers and generates fingerprint arrays of weight mappings; the privacy computing platform reconstructs the weight mappings and performs secure aggregation based on the encrypted cluster centers and fingerprint arrays, generates a global model and distributes it to the data user; the data user receives and uses the global model for analysis and decision-making in a trusted execution environment. The present invention predicts and completes the data of the missing modalities through the data of the existing modalities to ensure the integrity and effectiveness of the joint model; the encryption technology and the privacy computing platform realize the high security of medical data in the circulation process, and different medical institutions can realize efficient sharing and utilization of data under the premise of ensuring data security.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a trusted medical data security circulation system and method based on privacy computing. Background Art

[0002] With the continuous advancement of medical technology and the acceleration of digital transformation, precision medicine has become an important direction for the development of modern medicine. Precision medicine relies on the analysis and processing of large amounts of medical data to provide personalized treatment plans and health management. However, the rapid growth and digitization of medical data have brought severe challenges in data privacy and security. Protecting the privacy and security of medical data has become a basic requirement for achieving precision medicine.

[0003] In the current process of medical data sharing and circulation, the main challenges are data leakage risks, insufficient privacy protection, and improper data use. Although traditional data security technologies, such as access control and data encryption, can protect data security to a certain extent, it is difficult to fully solve the above problems in the complex scenarios of data sharing and circulation. Privacy computing technology provides a guarantee for the "available but invisible" data in the circulation process. Privacy computing is a collection of technologies that implement data analysis and calculation under the premise of protecting the data itself from external leakage, aiming to achieve the goal of "available but invisible" data. It includes technologies such as secure multi-party computing (MPC) based on cryptography, trusted execution environment (TEE) based on hardware, and federated learning (FL) based on artificial intelligence.

[0004] Different data owners may be missing some modal data during the process of federated learning (also known as joint learning), and there is still a risk of data privacy leakage. How to use privacy computing technology to achieve the transformation and release of data value while protecting data and privacy security has become a problem that needs to be solved. Summary of the invention

[0005] In response to the risk of data leakage and missing modal data during the sharing and circulation of medical data, the present invention provides a trusted medical data security circulation system and method based on privacy computing to ensure the efficient circulation of medical data, which will help promote the development of precision medicine, improve the quality and efficiency of medical services, and ensure that patient privacy is fully protected.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0007] A trusted medical data security circulation system based on privacy computing, the system comprising:

[0008] An initialization module, used to initialize the privacy computing platform and all participants, and generate encryption keys and private keys. The participants include data providers and data users;

[0009] A data provider is used to provide and process raw medical data, train a local model and perform weight clustering using the raw medical data, generate cluster centers and weight maps, and a fingerprint array for encrypting cluster centers and generating weight maps;

[0010] Data circulation module, used for data transmission and communication between modules, and manages access rights and usage records of each module;

[0011] The privacy computing platform is used for data encryption, decryption and secure computing. Based on the encrypted cluster centers and fingerprint arrays, the weight map is reconstructed and securely aggregated to generate a global model and distribute it to data users.

[0012] Data consumers receive and use the global model for analysis and decision making in a trusted execution environment.

[0013] As a preferred solution of the present invention, the initialization module includes:

[0014] The platform unit is used to perform the initialization process on the privacy computing platform and generate public parameters that can be accessed by all participants;

[0015] The participant unit is used to perform the initialization process for all participants. All participants use the unique identifier and public parameters provided by the privacy computing platform to independently create their own key pairs, including private keys and public keys, use the public key as the encryption key, and generate a functional decryption key.

[0016] As a preferred solution of the present invention, the data provider includes a local model training unit, a clustering unit and an encryption unit;

[0017] The local model training unit is used to train the local model on the original medical data to obtain an optimized model weight;

[0018] The clustering unit is used to cluster the optimized model weights using a clustering algorithm to generate cluster centers and corresponding weight mappings;

[0019] The encryption unit is used to encrypt the cluster center using an encryption key to generate an encrypted cluster center, insert the weight map into the Bloom filter, and generate a fingerprint array through a hash operation;

[0020] The formula for the hash operation is:

[0021]

[0022] In the formula, H n Represents the fingerprint array of the nth data provider; U n is the set of keys that the nth data provider needs to perform hash operations on; φ(a) represents the position of the calculated key a in the fingerprint array H. The fingerprint array H n The construction of depends on the unique seed s of the nth data provider n ;

[0023] The encrypted cluster center and the fingerprint array are uploaded to the privacy computing platform through the data circulation module.

[0024] As a preferred solution of the present invention, the clustering algorithm is used to cluster the optimized model weights, specifically using the k-means clustering algorithm to assign the model weights to different cluster centers, and the weight clustering loss function L wc It is expressed as:

[0025]

[0026] In the formula, The optimized model weights of the local model of the nth data provider, is the model weight The i-th weight in Z n is the cluster center of the nth data provider, Z n,j is the jth cluster center of the nth data provider; K is the number of cluster centers, d is the dimension of the model weight; u ij is the indicator function, the i-th weight When u belongs to the jth cluster center ij =1, otherwise u ij =0.

[0027] As a preferred solution of the present invention, the training of the local model on the original medical data also includes applying a multimodal joint learning method to process and fuse multiple data modalities of the original medical data and perform joint learning, specifically including:

[0028] For raw medical data with missing modalities, the missing image data is generated from the text description through the pre-trained image generation model, and the missing text description is generated from the image through the pre-trained text generation model;

[0029] Use pre-trained image encoder and text encoder to extract image features and text features respectively, and use cross-modal attention mechanism to fuse the image features and text features to generate a joint representation. Specifically, given an image-text pair {x I ,x T} as input, using a non-overlapping patch embedding layer and a pre-trained text encoder To obtain the image feature sequence I com and text feature sequence T com T com , the image feature sequence I com and text feature sequence T com All belong to the common dimension d com After the position embedding operation, the image feature sequence and the text feature sequence are input into the cross-modal fusion layer, which includes a visual-to-language attention module and a language-to-visual attention module. Both attention modules are based on cross-modal attention and can effectively fuse the representations between the two input modal embeddings to form an image-to-text embedding X. I→T To demonstrate cross-modal attention:

[0030]

[0031] In the formula, CM I→T W represents the cross-modal attention mechanism from image to text, which is used to fuse the image feature sequence and the text feature sequence to generate a joint representation; Q Represents the query weight matrix, and the image feature sequence I com Convert to query vector; W K Represents the key-value weight matrix, and converts the text feature sequence T com Convert to key vector; T represents transpose operation; W V Represents the value weight matrix, which converts the text feature sequence T com Converted to a value vector; softmax() is the activation function used to normalize weights;

[0032] In the same way, we get the embedding X from text to image T→I , the obtained X I→T and X T→I connected together and through a shared projection head f shared (·) and the self-attention layer projected into the latent space as the final joint embedding X joint :

[0033]

[0034] Where SelfAttention() represents the self-attention layer.

[0035] As a preferred solution of the present invention, the privacy computing platform uses the fingerprint array H n And the only seeds n Reconstruct the Bloom filter of each data provider to estimate the weight map of each data provider. The weight map P of the nth data provider n The formula is:

[0036] P n ={j|Member(i,j)=true};

[0037] Where, the Member() function represents the execution of a member query to determine whether weight i belongs to cluster center j;

[0038] Replace the corresponding position in the weight map with the encrypted cluster center to generate the encrypted model weight, expressed as:

[0039]

[0040] Where W n The encrypted model weight for the nth data provider, The jth cluster center after encryption for the nth data provider;

[0041] Combined with the received functional decryption key, secure aggregation is performed to obtain the global model parameters to generate the global model, which is expressed as:

[0042] θ r+1 ={Dec({W rn} n∈N ,dk r )};

[0043] In the formula, θ r+1 is the global model parameter of the rth round, Dec() represents the decryption operation; W rn is the encrypted model weight of the nth data provider in the rth round; N is the set of data providers; dk r The decryption key for the function in round r, where dk r =dKeyComb({f(sk n ,|D n ∣)} n∈N ), sk n represents the key of the nth data provider, |D n | is the number of data samples of data provider n, f() is the functional decryption key function, and dKeyComb() means combining the functional decryption keys of all data providers to generate a complete functional decryption key function.

[0044] A method for secure circulation of trusted medical data based on privacy computing, the method comprising:

[0045] Initialize the privacy computing platform and generate public parameters that are accessible to all participants.

[0046] Perform an initialization process for all participants. All participants use the unique identifier and public parameters provided by the privacy computing platform to independently create their own key pairs, including private keys and public keys, use the public key as the encryption key, and generate a functional decryption key;

[0047] The data provider trains the local model on the original medical data to obtain the optimized model weights, clusters the optimized model weights using a clustering algorithm, generates cluster centers and corresponding weight mappings, encrypts the cluster centers using an encryption key, generates encrypted cluster centers, and inserts the weight mapping into a Bloom filter, generating a fingerprint array through a hash operation;

[0048] Uploading the encrypted cluster center and the fingerprint array to the privacy computing platform through a data circulation module;

[0049] The privacy computing platform uses the fingerprint array H n And the only seeds n Reconstruct the Bloom filter of each data provider to estimate the weight map of each data provider. The weight map P of the nth data provider n The formula is:

[0050] P n ={j|Member(i,j)=true};

[0051] Where, the Member() function represents the execution of a member query to determine whether weight i belongs to cluster center j;

[0052] Replace the corresponding position in the weight map with the encrypted cluster center to generate the encrypted model weight, expressed as:

[0053]

[0054] Where W n The encrypted model weight for the nth data provider, The jth cluster center after encryption for the nth data provider;

[0055] Combined with the received functional decryption key, secure aggregation is performed to obtain the global model parameters to generate the global model, which is expressed as:

[0056] θ r+1 ={Dec({W rn} n∈N ,dk r )};

[0057] In the formula, θ r+1 is the global model parameter of the rth round, Dec() represents the decryption operation; W rnis the encrypted model weight of the nth data provider in the rth round; N is the set of data providers; dk r The decryption key for the function in round r, where dk r =dKeyComb({f(sk n ,|D n ∣)} n∈N ), sk n represents the key of the nth data provider, |D n | is the number of data samples of data provider n, f() is the function decryption key function, and dKeyComb() represents combining the function decryption keys of all data providers to generate a complete function decryption key function;

[0058] Data consumers receive and use the global model in a trusted execution environment for analysis and decision making.

[0059] As a preferred solution of the present invention, the clustering algorithm is used to cluster the optimized model weights, specifically using the k-means clustering algorithm to assign the model weights to different cluster centers, and the weight clustering loss function L wc It is expressed as:

[0060]

[0061] In the formula, The optimized model weights of the local model of the nth data provider, is the model weight The i-th weight in Z n is the cluster center of the nth data provider, Z n,j is the jth cluster center of the nth data provider; K is the number of cluster centers, d is the dimension of the model weight; u ij is the indicator function, the i-th weight When u belongs to the jth cluster center ij =1, otherwise u ij =0.

[0062] As a preferred solution of the present invention, the training of the local model on the original medical data also includes applying a multimodal joint learning method to process and fuse multiple data modalities of the original medical data and perform joint learning, specifically including:

[0063] For raw medical data with missing modalities, the missing image data is generated from the text description through the pre-trained image generation model, and the missing text description is generated from the image through the pre-trained text generation model;

[0064] Use pre-trained image encoder and text encoder to extract image features and text features respectively, and use cross-modal attention mechanism to fuse the image features and text features to generate a joint representation. Specifically, given an image-text pair {x I ,x T} as input, using a non-overlapping patch embedding layer and a pre-trained text encoder To obtain the image feature sequence I com and text feature sequence T com T com , the image feature sequence I com and text feature sequence T com All belong to the common dimension d com After the position embedding operation, the image feature sequence and the text feature sequence are input into the cross-modal fusion layer, which includes a visual-to-language attention module and a language-to-visual attention module. Both attention modules are based on cross-modal attention and can effectively fuse the representations between the two input modal embeddings to form an image-to-text embedding X. I→T To demonstrate cross-modal attention:

[0065]

[0066] In the formula, CM I→T W represents the cross-modal attention mechanism from image to text, which is used to fuse the image feature sequence and the text feature sequence to generate a joint representation; Q Represents the query weight matrix, and the image feature sequence I com Convert to query vector; W K Represents the key-value weight matrix, and converts the text feature sequence T com Convert to key vector; T represents transpose operation; W V Represents the value weight matrix, which converts the text feature sequence T com Converted to a value vector; softmax() is the activation function used to normalize weights;

[0067] In the same way, we get the embedding X from text to image T→I , the obtained X I→T and X T→I connected together and through a shared projection head f shared (·) and the self-attention layer projected into the latent space as the final joint embedding X joint :

[0068]

[0069] Where SelfAttention() represents the self-attention layer.

[0070] The beneficial effects of the present invention are: for data in missing modalities, prediction and completion are performed through data in existing modalities to ensure the integrity and effectiveness of the joint model; encryption technology and privacy computing platform are used to achieve high security of medical data in the circulation process and protect patient privacy; clustering algorithms and joint learning technology are used to improve the processing efficiency of large-scale medical data and reduce the consumption of computing resources; by establishing a trusted privacy computing platform, different medical institutions can achieve efficient sharing and utilization of data while ensuring data security. The encryption method in the present invention has good scalability and can adapt to data sets of different sizes and complexities to achieve efficient joint learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0072] in:

[0073] Figure 1 is a system structure diagram of the present invention;

[0074] Figure 2 4 is a flow chart of a method in an embodiment of the present invention. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0076] like Figure 1 As shown, an embodiment of the present invention provides a trusted medical data security circulation system based on privacy computing, which mainly includes an initialization module, a data provider, a poem user, a data circulation module and a privacy computing platform.

[0077] The initialization module is used to initialize the privacy computing platform and all participants, generate encryption keys and private keys, and the participants include data providers and data users;

[0078] In one embodiment, the initialization module includes:

[0079] The platform unit is used to perform the initialization process on the privacy computing platform and generate public parameters that can be accessed by all participants;

[0080] The participant unit is used to perform the initialization process for all participants. All participants use the unique identifier and public parameters provided by the privacy computing platform to independently create their own key pairs, including private keys and public keys, use the public key as the encryption key, and generate a functional decryption key.

[0081] The data provider is used to provide and process the original medical data, train the local model and perform weight clustering through the original medical data, generate cluster centers and weight mapping, and a fingerprint array for encrypting the cluster centers and generating weight mapping;

[0082] Further, the data provider includes a local model training unit, a clustering unit, and an encryption unit;

[0083] The local model training unit is used to train the local model on the original medical data to obtain the optimized model weights;

[0084] The clustering unit is used to cluster the optimized model weights using a clustering algorithm to generate cluster centers and corresponding weight mappings; preferably, the optimized model weights are clustered using a clustering algorithm, specifically using a k-means clustering algorithm to assign the model weights to different cluster centers, and the weight clustering loss function L wc It is expressed as:

[0085]

[0086] In the formula, The optimized model weights of the local model of the nth data provider, is the model weight The i-th weight in Z n is the cluster center of the nth data provider, Z n,j is the jth cluster center of the nth data provider; K is the number of cluster centers, d is the dimension of the model weight; u ij is the indicator function, the i-th weight When u belongs to the jth cluster center ij =1, otherwise u ij =0;

[0087] The encryption unit is used to encrypt the cluster center using the encryption key, generate the encrypted cluster center, insert the weight map into the Bloom filter, and generate the fingerprint array through the hash operation;

[0088] The formula for the hash operation is:

[0089]

[0090] In the formula, H n Represents the fingerprint array of the nth data provider; U n is the set of keys that the nth data provider needs to perform hash operations on. The key mainly refers to the local model weight or its corresponding weight mapping relationship; φ(a) represents the position of the calculated key a in the fingerprint array H. The fingerprint array H n The construction of depends on the unique seed s of the nth data provider n ;

[0091] The encrypted cluster centers and fingerprint arrays are uploaded to the privacy computing platform through the data circulation module.

[0092] In another preferred embodiment, training the local model on the original medical data also includes applying a multimodal joint learning method to process and fuse multiple data modalities of the original medical data and perform joint learning, specifically including:

[0093] For raw medical data with missing modalities, the missing image data is generated from text descriptions through a pre-trained image generation model (such as DALLE2), and the missing text descriptions are generated from images through a pre-trained text generation model (such as BLIP2);

[0094] To avoid semantic ambiguity caused by synonyms and polysemy, we use coarse-to-fine prompts to enhance generation. The prompt template is: "A photo of {fine-grained label}, a kind of {category label}, {text description}", which helps the pre-trained model better understand the characteristics of the generated target and improve the semantic association between text prompts and generated images. For images with missing text data, we use the visual question answering (VQA) task and image description function to generate text descriptions. A series of prompt questions are used to obtain a detailed description of the image, and the final synthetic text is generated by combining the image description results.

[0095] Use pre-trained image encoder and text encoder to extract image features and text features respectively, and use cross-modal attention mechanism to fuse image features and text features to generate joint representation. Specifically: given an image-text pair {x I ,x T} as input, using a non-overlapping patch embedding layer and a pre-trained text encoder f T E (·) to obtain the image feature sequence I com and text feature sequence T com T com , image feature sequence I com and text feature sequence T com All belong to the common dimension d comAfter the position embedding operation, the image feature sequence and the text feature sequence are input into the cross-modal fusion layer. The cross-modal fusion layer includes a visual-to-language attention module and a language-to-visual attention module. Both attention modules are based on cross-modal attention and can effectively fuse the representations between the two input modal embeddings to embed the image into text. I→T To demonstrate cross-modal attention:

[0096]

[0097] In the formula, CM I→T W represents the cross-modal attention mechanism from image to text, which is used to fuse the image feature sequence and the text feature sequence to generate a joint representation; Q Represents the query weight matrix, and the image feature sequence I com Convert to query vector; W K Represents the key-value weight matrix, and converts the text feature sequence T com Convert to key vector; T represents transpose operation; W V Represents the value weight matrix, which converts the text feature sequence T com Converted to a value vector; softmax() is the activation function used to normalize weights;

[0098] In the same way, we get the embedding X from text to image T→I , the obtained X I→T and X T→I connected together and through a shared projection head f shared (·) and the self-attention layer projected into the latent space as the final joint embedding X joint :

[0099]

[0100] Where SelfAttention() represents the self-attention layer;

[0101] In the same potential space The image-specific embedding X is obtained from I , text-specific embeddings X T and jointly embed X joint .

[0102] The quality of joint representation can also be improved by using multimodal contrastive learning methods (such as multimodal contrastive matching loss MCM Loss and representation alignment edge loss RAM Loss). Knowledge transfer is achieved by freezing the pre-trained model parameters to improve the efficiency and performance of joint representation learning.

[0103] The data circulation module is used for data transmission and communication between modules, and manages the access rights and usage records of each module;

[0104] The privacy computing platform is used for data encryption, decryption and secure computing. Based on the encrypted cluster centers and fingerprint arrays, it reconstructs the weight mapping and performs secure aggregation to generate a global model and distribute it to data users.

[0105] Specifically, the privacy computing platform uses the fingerprint array H n And the only seeds n Reconstruct the Bloom filter of each data provider to estimate the weight map of each data provider. The weight map P of the nth data provider n The formula is:

[0106] P n ={j|Member(i,j)=true};

[0107] Where, the Member() function represents the execution of a member query to determine whether weight i belongs to cluster center j;

[0108] Replace the corresponding position in the weight map with the encrypted cluster center to generate the encrypted model weight, expressed as:

[0109]

[0110] Where W n The encrypted model weight for the nth data provider, The jth cluster center after encryption for the nth data provider;

[0111] Combined with the received functional decryption key, secure aggregation is performed to obtain the global model parameters to generate the global model, which is expressed as:

[0112] θ r+1 ={Dec({W rn} n∈N ,dk r )};

[0113] In the formula, θ r+1 is the global model parameter of the rth round, Dec() represents the decryption operation; W rn is the encrypted model weight of the nth data provider in the rth round; N is the set of data providers; dk r The decryption key for the function in round r, where dk r =dKeyComb({f(sk n ,|D n ∣)} n∈N ), sk nrepresents the key of the nth data provider, |D n | is the number of data samples of data provider n, f() is the function decryption key function, and dKeyComb() represents combining the function decryption keys of all data providers to generate a complete function decryption key function;

[0114] Data consumers receive and use the global model in a trusted execution environment for analysis and decision making.

[0115] Furthermore, a clustering algorithm is used to cluster the optimized model weights. Specifically, the k-means clustering algorithm is used to assign the model weights to different cluster centers. The weight clustering loss function L wc It is expressed as:

[0116]

[0117] In the formula, The optimized model weights of the local model of the nth data provider, is the model weight The i-th weight in Z n is the cluster center of the nth data provider, Z n,j is the jth cluster center of the nth data provider; K is the number of cluster centers, d is the dimension of the model weight; u ij is the indicator function, the i-th weight When u belongs to the jth cluster center ij =1, otherwise u ij =0.

[0118] As a preferred embodiment of the present invention, training a local model on the original medical data also includes applying a multimodal joint learning method to process and fuse multiple data modalities of the original medical data and perform joint learning, specifically including:

[0119] For raw medical data with missing modalities, the missing image data is generated from the text description through the pre-trained image generation model, and the missing text description is generated from the image through the pre-trained text generation model;

[0120] Use pre-trained image encoder and text encoder to extract image features and text features respectively, and use cross-modal attention mechanism to fuse image features and text features to generate joint representation. Specifically: given an image-text pair {x I ,x T} as input, using a non-overlapping patch embedding layer and a pre-trained text encoder f T E (·) to obtain the image feature sequence I com and text feature sequence Tcom T com , the image feature sequence I com and text feature sequence T com All belong to the common dimension d com After the position embedding operation, the image feature sequence and the text feature sequence are input into the cross-modal fusion layer. The cross-modal fusion layer includes a visual-to-language attention module and a language-to-visual attention module. Both attention modules are based on cross-modal attention and can effectively fuse the representations between the two input modal embeddings to embed the image into text. I→T To demonstrate cross-modal attention:

[0121]

[0122] In the formula, CM I→T W represents the cross-modal attention mechanism from image to text, which is used to fuse the image feature sequence and the text feature sequence to generate a joint representation; Q Represents the query weight matrix, and the image feature sequence I com Convert to query vector; W K Represents the key-value weight matrix, and converts the text feature sequence T com Convert to key vector; T represents transpose operation; W V Represents the value weight matrix, which converts the text feature sequence T com Converted to a value vector; softmax() is the activation function used to normalize weights;

[0123] In the same way, we get the embedding X from text to image T→I , the obtained X I→T and X T→I connected together and through a shared projection head f shared (·) and the self-attention layer projected into the latent space as the final joint embedding X joint :

[0124]

[0125] Where SelfAttention() represents the self-attention layer.

[0126] To summarize, the present invention predicts and completes the data of missing modalities through the data of existing modalities, thereby ensuring the integrity and effectiveness of the joint model; through encryption technology and privacy computing platform, the high security of medical data in the circulation process is achieved to protect patient privacy; clustering algorithm and joint learning technology are used to improve the processing efficiency of large-scale medical data and reduce the consumption of computing resources; by establishing a trusted privacy computing platform, different medical institutions can achieve efficient sharing and utilization of data while ensuring data security. The encryption method in the present invention has good scalability and can adapt to data sets of different sizes and complexities to achieve efficient joint learning.

[0127] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, it can be implemented in whole or in part in the form of a computer program product, which includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0128] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0129] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A trusted medical data security circulation system based on privacy computing, characterized in that: The system comprises: An initialization module is used to initialize the privacy computing platform and all participants, and generate encryption keys and private keys. The participants include data providers and data users; A data provider is used to provide and process raw medical data, train a local model and perform weight clustering using the raw medical data, generate cluster centers and weight maps, and a fingerprint array for encrypting cluster centers and generating weight maps; The data circulation module is used for data transmission and communication between modules, and manages the access rights and usage records of each module. The privacy computing platform is used for data encryption, decryption and secure computing. Based on the encrypted cluster center and fingerprint array, the weight map is reconstructed and securely aggregated to generate a global model and distribute it to data users. Data consumers, which receive and use the global model in a trusted execution environment for analysis and decision making; The initialization module includes: The platform unit is used to perform the initialization process on the privacy computing platform and generate public parameters that can be accessed by all participants; The participant unit is used to perform the initialization process for all participants. All participants use the unique identifier and public parameters provided by the privacy computing platform to independently create their own key pairs, including private keys and public keys, use the public key as the encryption key, and generate a functional decryption key; The data provider includes a local model training unit, a clustering unit and an encryption unit; The local model training unit is used to train the local model on the original medical data to obtain the optimized model weights; the clustering unit is used to cluster the optimized model weights using a clustering algorithm to generate cluster centers and corresponding weight mappings; The encryption unit is used to encrypt the cluster center using an encryption key to generate an encrypted cluster center, insert the weight map into the Bloom filter, and generate a fingerprint array through a hash operation; The formula for the hash operation is: In the formula, H n Represents the fingerprint array of the nth data provider; U n is the set of keys that the nth data provider needs to perform hash operations on; f(a) represents the position of the calculated key a in the fingerprint array H. The fingerprint array H n The construction of depends on the unique seed s of the nth data provider n ; Uploading the encrypted cluster center and the fingerprint array to the privacy computing platform through a data circulation module; The privacy computing platform uses the fingerprint array H n And the only seeds n Reconstruct the Bloom filter of each data provider to estimate the weight map of each data provider. The weight map P of the nth data provider n The formula is: P n ={j∣Member(i,j)=true}; Where, the Member() function represents the execution of a member query to determine whether weight i belongs to cluster center j; Replace the corresponding position in the weight map with the encrypted cluster center to generate the encrypted model weight, expressed as: Where W n The encrypted model weight for the nth data provider, The jth cluster center after encryption for the nth data provider; Combined with the received functional decryption key, secure aggregation is performed to obtain the global model parameters to generate the global model, which is expressed as: θ r+1 {Dec({W rn } n∈N ,min r )}? In the formula, θ r+1 is the global model parameter of the rth round, Dec() represents the decryption operation; W rn is the encrypted model weight of the nth data provider in the rth round; N is the set of data providers; dk r The decryption key for the function in round r, where dk r =dKeyComb({f(sk n ,∣D n ∣)} n∈N ), sk n represents the key of the nth data provider, |D n | is the number of data samples of data provider n, f() is the functional decryption key function, and dKeyComb() means combining the functional decryption keys of all data providers to generate a complete functional decryption key function.

2. According to claim 1, a trusted medical data security circulation system based on privacy computing is characterized in that: The clustering algorithm is used to cluster the optimized model weights, specifically using the k-means clustering algorithm to assign the model weights to different cluster centers, and the weight clustering loss function L wc It is expressed as: In the formula, The optimized model weights of the local model of the nth data provider, is the model weight The i-th weight in Z n is the cluster center of the nth data provider, Z n,j is the jth cluster center of the nth data provider; K is the number of cluster centers, d is the dimension of the model weight; u ij is the indicator function, the i-th weight When u belongs to the jth cluster center ij =1, otherwise u ij =0.

3. According to claim 1, a trusted medical data security circulation system based on privacy computing is characterized in that: The training of the local model on the original medical data also includes applying a multimodal joint learning method to process and fuse multiple data modalities of the original medical data and perform joint learning, specifically including: For raw medical data with missing modalities, the missing image data is generated from the text description through the pre-trained image generation model, and the missing text description is generated from the image through the pre-trained text generation model; Use pre-trained image encoder and text encoder to extract image features and text features respectively, and use cross-modal attention mechanism to fuse the image features and text features to generate a joint representation. Specifically, given an image-text pair {x I ,x T } as input, using a non-overlapping patch embedding layer and a pre-trained text encoder To obtain the image feature sequence I com and text feature sequence T com T com , the image feature sequence I com and text feature sequence T com All belong to the common dimension d com After the position embedding operation, the image feature sequence and the text feature sequence are input into the cross-modal fusion layer, which includes a visual-to-language attention module and a language-to-visual attention module. Both attention modules are based on cross-modal attention and can effectively fuse the representations between the two input modal embeddings to form an image-to-text embedding X. I→T To demonstrate cross-modal attention: In the formula, CM I→T W represents the cross-modal attention mechanism from image to text, which is used to fuse the image feature sequence and the text feature sequence to generate a joint representation; Q Represents the query weight matrix, and the image feature sequence I com Convert to query vector; W K Represents the key-value weight matrix, and converts the text feature sequence T com Convert to key vector; T represents transpose operation; W V Represents the value weight matrix, which converts the text feature sequence T com Converted to a value vector; softmax() is the activation function used to normalize weights; In the same way, we get the embedding X from text to image T→I , the obtained X I→T and X T→I connected together and through a shared projection head f shared (·) and the self-attention layer projected into the latent space as the final joint embedding X joint : Where SelfAttention() represents the self-attention layer.

4. A trusted medical data secure circulation method based on privacy computing, characterized in that: The method comprises: Initialize the privacy computing platform and generate public parameters that are accessible to all participants. Perform an initialization process for all participants. All participants use the unique identifier and public parameters provided by the privacy computing platform to independently create their own key pairs, including private keys and public keys, use the public key as the encryption key, and generate a functional decryption key; The data provider trains the local model on the original medical data to obtain the optimized model weights, clusters the optimized model weights using a clustering algorithm, generates cluster centers and corresponding weight mappings, encrypts the cluster centers using an encryption key, generates encrypted cluster centers, and inserts the weight mapping into a Bloom filter, generating a fingerprint array through a hash operation; Uploading the encrypted cluster center and the fingerprint array to the privacy computing platform through a data circulation module; The privacy computing platform uses the fingerprint array H n And the only seeds n Reconstruct the Bloom filter of each data provider to estimate the weight map of each data provider. The weight map P of the nth data provider n The formula is: P n ={j∣Member(i,j)=true}; Where, the Member() function represents the execution of a member query to determine whether weight i belongs to cluster center j; Replace the corresponding position in the weight map with the encrypted cluster center to generate the encrypted model weight, expressed as: Where W n The encrypted model weight for the nth data provider, The jth cluster center after encryption for the nth data provider; Combined with the received functional decryption key, secure aggregation is performed to obtain the global model parameters to generate the global model, which is expressed as: θ r+1 {Dec({W rn } n∈N ,min r )}? In the formula, θ r+1 is the global model parameter of the rth round, Dec() represents the decryption operation; W rn is the encrypted model weight of the nth data provider in the rth round; N is the set of data providers; dk r The decryption key for the function in round r, where dk r =dKeyComb({f(sk n ,∣D n ∣)} n∈N ), sk n represents the key of the nth data provider, |D n ∣ is the number of data samples of data provider n, f() is the function decryption key function, and dKeyComb() represents combining the function decryption keys of all data providers to generate a complete function decryption key function; Data consumers receive and use the global model in a trusted execution environment for analysis and decision making.

5. According to claim 4, a method for secure circulation of trusted medical data based on privacy computing is characterized in that: The clustering algorithm is used to cluster the optimized model weights, specifically using the k-means clustering algorithm to assign the model weights to different cluster centers, and the weight clustering loss function L wc It is expressed as: In the formula, The optimized model weights of the local model of the nth data provider, is the model weight The i-th weight in Z n is the cluster center of the nth data provider, Z n,j is the jth cluster center of the nth data provider; K is the number of cluster centers, d is the dimension of the model weight; u ij is the indicator function, the i-th weight When u belongs to the jth cluster center ij =1, otherwise u ij =0.

6. According to claim 4, a method for secure circulation of trusted medical data based on privacy computing is characterized in that: The training of the local model on the original medical data also includes applying a multimodal joint learning method to process and fuse multiple data modalities of the original medical data and perform joint learning, specifically including: For raw medical data with missing modalities, the missing image data is generated from the text description through the pre-trained image generation model, and the missing text description is generated from the image through the pre-trained text generation model; Use pre-trained image encoder and text encoder to extract image features and text features respectively, and use cross-modal attention mechanism to fuse the image features and text features to generate a joint representation. Specifically, given an image-text pair {x I ,x T } as input, using a non-overlapping patch embedding layer and a pre-trained text encoder To obtain the image feature sequence I com and text feature sequence T com T com , the image feature sequence I com and text feature sequence T com All belong to the common dimension d com After the position embedding operation, the image feature sequence and the text feature sequence are input into the cross-modal fusion layer, which includes a visual-to-language attention module and a language-to-visual attention module. Both attention modules are based on cross-modal attention and can effectively fuse the representations between the two input modal embeddings to form an image-to-text embedding X. I→T To demonstrate cross-modal attention: In the formula, CM I→T W represents the cross-modal attention mechanism from image to text, which is used to fuse the image feature sequence and the text feature sequence to generate a joint representation; Q Represents the query weight matrix, and the image feature sequence I com Convert to query vector; W K Represents the key-value weight matrix, and converts the text feature sequence T com Convert to key vector; T represents transpose operation; W V Represents the value weight matrix, which converts the text feature sequence T com Converted to a value vector; softmax() is the activation function used to normalize weights; In the same way, we get the embedding X from text to image T→I , the obtained X I→T and X T→I connected together and through a shared projection head f shared (·) and the self-attention layer projected into the latent space as the final joint embedding X joint : Where SelfAttention() represents the self-attention layer.

Images