Multi-modal data fusion-oriented medical meta-space infrastructure and privacy protection method

Abstract

The invention provides a medical meta-space infrastructure and privacy protection method for multi-modal data fusion, and the method comprises the following steps: collecting multi-source health data, and forming standardized multi-modal medical meta-data based on the multi-source health data; constructing a decentralized distributed network infrastructure to realize intelligent management and control of user health data; identity-based encryption and zero-knowledge proof are integrated on the basis of a decentralized distributed network infrastructure, a PKG is responsible for initializing a system and generating a secret key, and public and private key pairs of care entities are issued by using secure bilinear mapping and a hash function; the edge server performs preliminary encryption on local user multi-source data and transmits the data to the central server; when a data request is sent out, a requester issues an intelligent contract, and a demand for data and zero-knowledge proof are attached to the intelligent contract; a requested node in a network can select a corresponding request, the compliance of the requested node is verified by using zk-SNARKs through an edge server of a group where the requested node is located under the condition that the data does not need to be disclosed, a conversion key is generated by a PKG, and the data is converted into a format which can only be decrypted by a requester through an agent re-encryption service on a central server.

Description

Technical Field

[0001] This invention relates to the field of smart healthcare and health management technology, and in particular to a medical metaspace infrastructure and privacy protection method for multimodal data fusion. Background Technology

[0002] In modern medical systems, data sources are diverse, including but not limited to physiological, psychological, environmental, behavioral, and medical imaging data. These data sources are characterized by multi-source, multi-layered, and multi-scale nature, with varying formats and standards, making direct fusion and analysis difficult. Therefore, achieving standardized fusion of multi-source heterogeneous data is one of the key challenges facing smart healthcare.

[0003] The management of medical metadata involves the collaboration of multiple network nodes on heterogeneous platforms. In this case, traditional data management methods may have potential cybersecurity risks and vulnerabilities. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a medical metaspace infrastructure and privacy protection method for multimodal data fusion, comprising the following steps: S1: Collect multi-source health data and form standardized multimodal medical metadata based on the multi-source health data; S2: Build a decentralized distributed network infrastructure to achieve intelligent management and control of user health data; S3: Based on a decentralized distributed network infrastructure, it integrates identity-based encryption and zero-knowledge proofs. PKG is responsible for initializing the system and generating keys, and uses secure bilinear mapping and hash functions to distribute public and private key pairs to each care entity. S4: The edge server performs preliminary encryption on local user multi-source data before transmitting it to the central server.

[0005] S5: When a data request is issued, the requesting party publishes a smart contract, attaching the data requirement and the method for... The zero-knowledge proof constituted; S6: Requested nodes in the network can select the appropriate request and use it through the edge server of their respective group. The compliance of the data is verified without disclosing the data itself, and a conversion key is generated by PKG. The proxy re-encryption service on the central server converts the data into a format that only the requester can decrypt.

[0006] S1 includes the following steps: S11: Utilizes wearable medical motion sensors, biosensors, and environmental sensors to collect individual physiological indicators, while integrating multi-source health data from electronic medical records, medical images, and medication records; S12: Through the methods of extracting all historical data and augmenting the extraction of new data, multi-source health data is converted into a standardized file format for storage; S13: Integrate multi-source health data into a unified structure through a multimodal medical metadata aggregation model; The multimodal medical metadata aggregation model is represented as follows: in, For medical metadata aggregation, Scale information representing medical metadata, Representing data information, wearable devices Collection time and space , physiological signals Composition of user feedback information and its health data The information set that they together constitute; S14: Standardized multimodal medical metadata is formed through a unified functional data integration model.

[0007] Specifically, the decentralized distributed network infrastructure adopts a hybrid data storage and "cloud-edge-device" collaborative computing approach. It utilizes resource management gateways, network transmission base stations, data management centers, and multi-access cloud computing to build web servers that run decentralized applications, call distributed network interfaces to establish connections with heterogeneous platforms, and realize intelligent management and control of user health data through smart contracts.

[0008] The steps to form a zero-knowledge proof are as follows: S51: Based on user identity Heyuan Space Medical Data Building an extended dataset S52: Generate random numbers r ,calculate .

[0009] S53: Generate user digital signature .

[0010] S54: Based on smart contracts Processing medical datasets Output result set And hash value ℎ: S55: Transfer system security parameters and smart contract input to key generation algorithm Obtain proof to generate key and verify the generated key : .

[0011] S56: By generating a key Medical datasets User digital signature Output result set A reliable zero-knowledge proof is obtained by performing a ℎ operation on the hash value. : Also includes S7; S7: Multi-source data fusion and visualization interaction.

[0012] In S7, the specific steps include: S71: Based on the digital twin model, it integrates multi-source user health data and cognitive and behavioral indicators to achieve a dynamic twin mapping of individual health status for self-care. S72: Using virtual digital clusters, 3D modeling, and extended reality (XR) technologies, a multi-dimensional data interaction interface is constructed to develop digital twin basic scenarios based on users' personalized self-care needs; a physical space perception system for self-care implementation is constructed through IoT sensing modules such as motion capture and environmental monitoring. S73: Acquires dynamic 3D interactive data of the medical metaspace during the user's interaction with the system through a head-mounted extended reality device, and then realizes real-time data transmission between the terminal device and the XR interactive data stream platform through 5G, WLAN or Bluetooth communication protocols; S74: During user interaction, the short-term fluctuations and long-term changes of different monitoring indicators are highlighted by the dynamic quantification method of virtual events, realizing the virtual and real superposition and integration of user self-care scenarios and medical metaspace; and various indicator parameters and real-time health information are fed back to the user interaction interface through an interactive digital twin.

[0013] Beneficial effects: Based on identity-based encryption technology, this method implements user data encryption and authentication, enhancing the security of data management and data exchange processes.

[0014] This research aims to construct a medical metadata aggregation model for multi-source, multi-layered, and multi-scale health and medical data, exploring methods for data set classification, computation, and modeling. Based on multimodal data aggregated from wearable health devices and electronic medical records, a distributed data sharing mechanism is studied through cloud-edge-device collaborative computing to achieve standardized data storage and management methods. Furthermore, for data transformation models, standardized terminology specifications and information fusion data queues are established to achieve an efficient, secure, and unified framework for describing medical metaspace resources.

[0015] Based on self-care application scenarios, a dynamic data twin model of user health data is constructed. Extended reality development tools are integrated to achieve immersive human-computer interaction methods and enhance user perception and cognitive abilities during the interaction process. Addressing the data link security issue between the metaspace and the real space, this invention utilizes distributed technology and zero-knowledge proofs to construct a medical metadata storage and interaction method, building an "on-chain" and "off-chain" combined storage model to ensure effective verification of data integrity. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This invention provides a standardized flowchart for a multimodal medical metadata method. Figure 3 This is a flowchart of the zero-knowledge proof formation method of the present invention. Detailed Implementation

[0018] See Figure 1 This invention proposes a medical metaspace infrastructure and privacy protection method for multimodal data fusion, including the following steps: S1: Collect multi-source health data and form standardized multimodal medical metadata based on the multi-source health data; Specifically, see Figure 2 This includes the following steps: S11: Utilizes wearable medical motion sensors, biosensors, and environmental sensors to collect individual physiological indicators, while integrating multi-source health data from electronic medical records, medical images, and medication records; S12: Through the methods of extracting all historical data and augmenting the extraction of new data, multi-source health data is converted into a standardized file format for storage; Specifically, through data interface adapters deployed in various medical information systems such as hospital information systems, electronic medical record systems, and medical image archiving and communication systems, historical health data is extracted in a single, full volume using ETL tools or dedicated APIs, simultaneously performing deduplication, invalid record filtering, and timestamp alignment. Meanwhile, new health data is continuously collected from data sources such as wearable devices, sensors, and mobile applications through real-time or scheduled data interfaces, with data completion, outlier handling, and quality verification performed during this process.

[0019] S13: Integrate multi-source health data into a unified structure through a multimodal medical metadata aggregation model; The multimodal medical metadata aggregation model is represented as follows: in, For medical metadata aggregation, Scale information representing medical metadata, Representing data information, wearable devices Collection time and space , physiological signals Composition of user feedback information and its health data The information set that they together constitute; S14: Standardized multimodal medical metadata is formed through a unified functional data integration model.

[0020] Specifically, the health data from different sources is first processed to unify the format, and then the data fields from different sources are mapped to a unified medical metadata model through predefined data mapping rules. The integrated data is then cleaned and deduplicated, and finally the cleaned data is encapsulated into standardized multimodal medical metadata.

[0021] S2: Build a decentralized distributed network infrastructure to achieve intelligent management and control of user health data; Specifically, the decentralized distributed network infrastructure adopts a hybrid data storage and "cloud-edge-device" collaborative computing approach. It utilizes resource management gateways, network transmission base stations, data management centers, and multi-access cloud computing to build web servers that run decentralized applications, call distributed network interfaces to establish connections with heterogeneous platforms, and realize intelligent management and control of user health data through smart contracts.

[0022] The management of medical metadata involves the collaboration of multiple network nodes across heterogeneous platforms. In this context, traditional data management methods may pose potential cybersecurity risks. This invention addresses this issue by constructing a decentralized distributed network infrastructure for medical metadata management. To resolve common problems such as the complexity, latency, and congestion of distributed network data exchange, this invention employs hybrid data storage and a "cloud-edge-device" collaborative computing approach to improve data transmission and storage efficiency. Utilizing resource management gateways, network transmission base stations, a data management center, and multi-access cloud computing, a web server runs decentralized applications, establishing connections with heterogeneous platforms through distributed network interfaces, and achieving intelligent management and control of user health data through smart contracts.

[0023] S3: Based on a decentralized distributed network infrastructure, it integrates identity-based encryption and zero-knowledge proofs. PKG is responsible for initializing the system and generating keys, and uses secure bilinear mapping and hash functions to distribute public and private key pairs to each care entity. S4: The edge server performs preliminary encryption on local user multi-source data before transmitting it to the central server; S5: When a data request is issued, the requesting party publishes a smart contract, attaching the data requirement and the method for... The zero-knowledge proof constituted; See Figure 3 The steps for forming a zero-knowledge proof are as follows: S51: Based on user identity Heyuan Space Medical Data Building an extended dataset S52: Generate random numbers r ,calculate ; S53: Generate user digital signature ; S54: Based on smart contracts Processing medical datasets Output result set And hash value ℎ: S55: Transfer system security parameters and smart contract input to key generation algorithm Proof of key generation and verify the generated key : .

[0024] S56: By generating a key Medical datasets User digital signature Output result set A reliable zero-knowledge proof is obtained by performing a ℎ operation on the hash value. : S6: In a decentralized distributed network, the requested node can select the appropriate request and use it through the edge server of its group. The compliance of the data is verified without disclosing the data itself, and a conversion key is generated by PKG. The proxy re-encryption service on the central server converts the data into a format that only the requester can decrypt.

[0025] Furthermore, it also includes S7; S7: Multi-source data fusion and visualization interaction, specifically including the following steps: S71: Based on the digital twin model, it integrates multi-source user health data and cognitive and behavioral indicators to achieve dynamic twin mapping of the health status of self-care individuals; based on the interactive feedback mechanism between the virtual avatar of the self-care individual and the virtual avatar of the self-care service provider, it generates and optimizes intelligent decisions for self-care intervention. S72: Utilizing virtual digital clusters, 3D modeling, and extended reality (XR) technologies, a multi-dimensional data interaction interface is constructed to develop digital twin-based scenarios based on users' personalized self-care needs. A physical space perception system for self-care implementation is constructed using IoT sensing modules such as motion capture and environmental monitoring. Through interactive data visualization, users can intuitively understand their overall health status, including physiological indicators and disease risks, as well as dynamically updated self-care intervention goals based on intelligent decision-making. Simultaneously, personalized health management suggestions are provided based on the analysis of users' personal health data and behavioral patterns, thereby achieving more precise and effective self-care solutions.

[0026] S73: Acquires dynamic 3D interactive data of the medical metaspace during the user's interaction with the system through a head-mounted extended reality device, and then realizes real-time data transmission between the terminal device and the XR interactive data stream platform through communication protocols such as 5G, WLAN, and Bluetooth; S74: During user interaction, the short-term fluctuations and long-term changes of different monitoring indicators are highlighted by the dynamic quantification method of virtual events, so as to realize the virtual and real superposition and integration of user self-care scenarios and medical metaspace; through an interactive digital twin, various indicator parameters and real-time health information are fed back to the user interaction interface to enhance the human-computer interaction experience based on extended reality.

[0027] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A medical metaspace infrastructure and privacy protection method for multimodal data fusion, characterized in that, Includes the following steps: S1: Collect multi-source health data and form standardized multimodal medical metadata based on the multi-source health data; S2: Build a decentralized distributed network infrastructure to achieve intelligent management and control of user health data; S3: Based on a decentralized distributed network infrastructure, it integrates identity-based encryption and zero-knowledge proofs. PKG is responsible for initializing the system and generating keys, and uses secure bilinear mapping and hash functions to distribute public and private key pairs to each care entity. S4: The edge server performs preliminary encryption on local user multi-source data before transmitting it to the central server. S5: When a data request is issued, the requesting party publishes a smart contract, attaching the data requirement and the method for... The zero-knowledge proof constituted; S6: Requested nodes in the network can select the appropriate request and use it through the edge server of their respective group. The compliance of the data is verified without disclosing the data itself, and a conversion key is generated by PKG. The proxy re-encryption service on the central server converts the data into a format that only the requester can decrypt.

2. The medical metaspace infrastructure and privacy protection method for multimodal data fusion as described in claim 1, characterized in that: S1 includes the following steps: S11: Utilizes wearable medical motion sensors, biosensors, and environmental sensors to collect individual physiological indicators, while integrating multi-source health data from electronic medical records, medical images, and medication records; S12: Through the methods of extracting all historical data and augmenting the extraction of new data, multi-source health data is converted into a standardized file format for storage; S13: Integrate multi-source health data into a unified structure through a multimodal medical metadata aggregation model; The multimodal medical metadata aggregation model is represented as follows: in, For medical metadata aggregation, Scale information representing medical metadata, Representing data information, wearable devices Collection time and space , physiological signals Composition of user feedback information and its health data The information set that they together constitute; S14: Standardized multimodal medical metadata is formed through a unified functional data integration model.

3. The medical metaspace infrastructure and privacy protection method for multimodal data fusion as described in claim 1, characterized in that: Specifically, the decentralized distributed network infrastructure adopts a hybrid data storage and "cloud-edge-device" collaborative computing approach. It utilizes resource management gateways, network transmission base stations, data management centers, and multi-access cloud computing to build web servers that run decentralized applications, call distributed network interfaces to establish connections with heterogeneous platforms, and realize intelligent management and control of user health data through smart contracts.

4. The medical metaspace infrastructure and privacy protection method for multimodal data fusion as described in claim 1, characterized in that: The steps to form a zero-knowledge proof are as follows: S51: Based on user identity Heyuan Space Medical Data Building an extended dataset S52: Generate random numbers r ,calculate . S53: Generate user digital signature . S54: Based on smart contracts Processing medical datasets Output result set And hash value ℎ: S55: Transfer system security parameters and smart contract input to key generation algorithm Proof of key generation and verify the generated key : . S56: By generating a key Medical datasets User digital signature Output result set A reliable zero-knowledge proof is obtained by performing a ℎ operation on the hash value. : .

5. The medical metaspace infrastructure and privacy protection method for multimodal data fusion as described in claim 1, characterized in that: Also includes S7; S7: Multi-source data fusion and visualization interaction.

6. The medical metaspace infrastructure and privacy protection method for multimodal data fusion as described in claim 1, characterized in that: In S7, the specific steps include: S71: Based on the digital twin model, it integrates multi-source user health data and cognitive and behavioral indicators to achieve a dynamic twin mapping of individual health status for self-care. S72: Using virtual digital clusters, 3D modeling, and extended reality (XR) technologies, a multi-dimensional data interaction interface is constructed to develop digital twin basic scenarios based on users' personalized self-care needs; a physical space perception system for self-care implementation is constructed through IoT sensing modules such as motion capture and environmental monitoring. S73: Acquires dynamic 3D interactive data of the medical metaspace during the user's interaction with the system through a head-mounted extended reality device, and then realizes real-time data transmission between the terminal device and the XR interactive data stream platform through 5G, WLAN or Bluetooth communication protocols; S74: During user interaction, the short-term fluctuations and long-term changes of different monitoring indicators are highlighted by the dynamic quantification method of virtual events, realizing the virtual and real superposition and integration of user self-care scenarios and medical metaspace; and various indicator parameters and real-time health information are fed back to the user interaction interface through an interactive digital twin.

Images