Establishment method and device of digestive tract database, equipment and storage medium

Abstract

The invention relates to the technical field of medical data, in particular to an alimentary canal database establishment method and device, equipment and a storage medium, and the method comprises the steps: obtaining multi-modal alimentary canal data and environment information, constructing a space-time semantic fusion multi-dimensional data model, and generating a causal relationship enhancement influence propagation graph; carrying out feature extraction based on a cross-modal attention mechanism according to the multi-modal alimentary canal data, and carrying out feature enhancement in combination with a space-time semantic fusion multi-dimensional data model and a causal relationship enhancement influence propagation graph to obtain enhanced data features; constructing a global annotation model based on federated learning, and complementing missing information in the multi-modal digestive tract data through hierarchical feature reconstruction and a multi-model cooperation strategy; a block chain technology is adopted to carry out right confirmation and encryption, and sharing and updating are carried out based on the block chain technology. Therefore, the problems that in the prior art, multi-modal integration is insufficient, causal mining is weak, missing completion is low in efficiency, right confirmation sharing is unsafe, and feature representation is one-sided are solved.

Description

Technical Field

[0001] This application relates to the technical field of medical data, and particularly relates to a method, device, equipment and storage medium for establishing a digestive tract database. Background Art

[0002] A digestive tract database is a system that integrates multi-modal digestive tract data and environmental information. It provides core data support and solid technical guarantee for medical research and clinical diagnosis by accurately collecting data to ensure comprehensiveness, modeling to ensure correlation, feature processing to ensure effectiveness, complementing information to ensure integrity, and blockchain to ensure security. This guarantee plays an important role in aspects such as accurate diagnosis of digestive tract diseases, personalized treatment, and research on pathogenesis.

[0003] The existing construction of digestive tract databases relies on traditional equipment, manual entry, and basic algorithms. However, these methods have the following problems: inconsistent collection standards, large differences in data such as endoscopic images, easy errors and incompleteness in environmental information; imperfect spatio-temporal semantic models, poor construction of causal propagation graphs, and difficulty in deeply exploring data connections; poor cross-modal feature extraction, lack of systematic operations in the enhancement link, and limited assistance for diagnosis; slow federated learning collaboration, weak privacy, poor reconstruction and multi-model collaboration effects; lack of a rights confirmation mechanism, weak encryption, no dynamic update, and high risks in storage and sharing; in summary, these problems lead to low data quality, weak analysis ability, and no security guarantee in digestive tract databases, making it difficult to meet the needs of medical research and clinical diagnosis. Summary of the Invention

[0004] This application provides a method, device, equipment and storage medium for establishing a digestive tract database to solve problems such as insufficient multi-modal integration, weak causal mining, inefficient missing complementation, insecure rights confirmation and sharing, and one-sided feature representation in the prior art.

[0005] An embodiment of the first aspect of the present application provides a method for establishing a digestive tract database, including the following steps: obtaining multimodal digestive tract data and environmental information; constructing a spatio-temporal semantic fusion multi-dimensional data model and generating a causal relationship enhanced influence propagation graph according to the multimodal digestive tract data and the environmental information; performing feature extraction on the multimodal digestive tract data based on a cross-modal attention mechanism, and combining the spatio-temporal semantic fusion multi-dimensional data model and the causal relationship enhanced influence propagation graph to perform feature enhancement, obtaining enhanced data features; based on the enhanced data features, constructing a global annotation model based on federated learning, and complementing missing information in the multimodal digestive tract data through a hierarchical feature reconstruction and multi-model collaboration strategy, where the federated learning includes distributed collaboration and privacy protection, the hierarchical feature reconstruction includes coarse-grained feature restoration, fine-grained feature refinement, and multimodal alignment, and the multi-model collaboration strategy includes model diversification and a collaboration mechanism; performing rights confirmation and encryption on the complemented multimodal digestive tract data using blockchain technology, and performing sharing and updating based on the blockchain technology, where the blockchain technology includes data hash on-chain storage, smart contracts, and homomorphic encryption.

[0006] Preferably, performing rights confirmation on the complemented multimodal digestive tract data using blockchain technology includes: calculating the complemented multimodal digestive tract data according to a hash algorithm to obtain a unique data identifier; storing the unique data identifier in the blockchain, completing the rights confirmation operation, and obtaining a rights confirmation result; writing a smart contract according to the rights confirmation result and the blockchain, deploying the smart contract to a blockchain network, and using an automatic execution mechanism to associate the data access permission rule with the rights confirmation result to achieve access control.

[0007] Preferably, after achieving access control, it includes: encrypting the complemented multimodal digestive tract data based on a homomorphic encryption algorithm to obtain encrypted data; sharing and updating the encrypted data on the blockchain, where when the encrypted data is updated, the update content is recorded based on a timestamp and a hash chain to obtain an updated blockchain data record.

[0008] Preferably, based on the enhanced data features, constructing a global annotation model based on federated learning, and complementing missing information in the multimodal digestive tract data through a hierarchical feature reconstruction and multi-model collaboration strategy includes: obtaining local model parameters; encrypting and transmitting the local model parameters, and aggregating them at a central node, and constructing a global annotation model according to the aggregated data; sequentially performing coarse-grained feature restoration and fine-grained feature refinement on the missing information in the multimodal digestive tract data based on the global annotation model to obtain a missing information restoration result; using the processing results of different models for fusion according to the missing information restoration result to complete the complementation of the missing information.

[0009] Preferably, feature extraction is performed on the multi-modal digestive tract data based on a cross-modal attention mechanism, and feature enhancement is carried out in combination with the spatio-temporal semantic fusion multi-dimensional data model and the causal relationship enhanced influence propagation graph to obtain enhanced data feature information, including: inputting the multi-modal digestive tract data into a cross-modal attention model to obtain the attention weight distribution within each modal data; based on the attention weight distribution, obtaining a cross-modal attention fusion feature; according to the cross-modal attention fusion feature, combining the spatio-temporal semantic features in the spatio-temporal semantic fusion multi-dimensional data model to obtain spatio-temporal semantic attention features; and performing feature enhancement on the spatio-temporal semantic attention features and the causal association information in the causal relationship enhanced influence propagation graph to obtain enhanced data feature information.

[0010] Preferably, based on the multi-modal digestive tract data and the environmental information, a spatio-temporal semantic fusion multi-dimensional data model is constructed and a causal relationship enhanced influence propagation graph is generated, including: obtaining feature vectors based on different modal features of the multi-modal digestive tract data, combining the timestamp in the environmental information, and encoding the time information into a time feature vector; based on the time feature vector, combining the spatial environmental information to obtain a spatial feature vector, and passing the spatial feature vector through a semantic fusion gate to obtain a feature vector integrating semantic information, thereby constructing a spatio-temporal semantic fusion multi-dimensional data model; based on the spatio-temporal semantic fusion multi-dimensional data model, using a graph neural network to analyze the causal relationship between data to obtain causal association information, and based on the causal association information, generating a causal relationship enhanced influence propagation graph through a multi-hop path reasoning mechanism.

[0011] The second aspect of the present application provides an apparatus for establishing a digestive tract database, including: an acquisition module for acquiring multi-modal digestive tract data and environmental information; a construction module for constructing a spatio-temporal semantic fusion multi-dimensional data model and generating a causal relationship enhanced influence propagation graph according to the multi-modal digestive tract data and the environmental information; an enhancement module for performing feature extraction on the multi-modal digestive tract data based on a cross-modal attention mechanism, and performing feature enhancement by combining the spatio-temporal semantic fusion multi-dimensional data model and the causal relationship enhanced influence propagation graph to obtain enhanced data features; a completion module for constructing a global annotation model based on federated learning according to the enhanced data features, and completing the missing information in the multi-modal digestive tract data through a hierarchical feature reconstruction and multi-model collaboration strategy, where the federated learning includes distributed collaboration and privacy protection, the hierarchical feature reconstruction includes coarse-grained feature restoration, fine-grained feature refinement, and multi-modal alignment, and the multi-model collaboration strategy includes model diversification and a collaboration mechanism; an update module for authenticating and encrypting the completed multi-modal digestive tract data using blockchain technology, and sharing and updating it based on blockchain technology, where the blockchain technology includes data hash on-chain storage, smart contracts, and homomorphic encryption.

[0012] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement a method for establishing a digestive tract database as in the above embodiment.

[0013] The fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to implement a method for establishing a digestive tract database as in the above embodiment.

[0014] The fifth aspect of the present application provides a computer program product, including a computer program or instruction for implementing a method for establishing a digestive tract database as in the above embodiment.

[0015] Therefore, the present application has the following beneficial effects: In the embodiments of the present application, by constructing a spatio-temporal semantic fusion multi-dimensional data model, the comprehensive analysis and deep fusion of multi-source data are realized. A causal relationship enhanced influence propagation graph is generated, providing rich information for feature extraction and data analysis. In the feature extraction stage, cross-modal attention mechanism is combined for feature enhancement, improving the accuracy and robustness of data features. A global annotation model is constructed based on federated learning, protecting data privacy and improving the generalization ability and annotation accuracy of the model. At the same time, missing information is effectively complemented through hierarchical feature reconstruction and multi-model collaboration strategy. Finally, blockchain technology is used to confirm the rights and encrypt the complemented data, realizing data sharing and updating, and ensuring the traceability and immutability of data. Thus, the problems of insufficient multi-modal integration and low feature utilization rate in the prior art are solved.

[0016] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Brief Description of the Drawings

[0017] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the accompanying drawings, where: Figure 1 is a flowchart of a method for establishing a digestive tract database according to an embodiment of the present application; Figure 2 is a schematic diagram of a scenario application of a certain technology company according to an embodiment of the present application; Figure 3 is a schematic diagram of a hash algorithm of a certain foreign technology company according to an embodiment of the present application; Figure 4 is a flowchart of a method for establishing a digestive tract database according to an embodiment of the present application; Figure 5 is a schematic structural diagram of a device for establishing a digestive tract database according to an embodiment of the present application; Figure 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Detailed Description of the Embodiments

[0018] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation of the present application.

[0019] The following describes a method, apparatus, device, and storage medium for establishing a digestive tract database according to embodiments of the present application. In view of the problem of low feature utilization rate mentioned in the above background art, the present application provides a method for establishing a digestive tract database. In this method, by constructing a spatio-temporal semantic fusion multi-dimensional data model, comprehensive analysis and deep fusion of multi-source data are realized. A causal relationship enhanced influence propagation graph is generated, providing rich information for feature extraction and data analysis. In the feature extraction stage, cross-modal attention mechanism is combined for feature enhancement, improving the accuracy and robustness of data features. A global annotation model is constructed based on federated learning, protecting data privacy and improving the generalization ability and annotation accuracy of the model. At the same time, missing information is effectively complemented through hierarchical feature reconstruction and multi-model collaborative strategies. Finally, blockchain technology is used to confirm the rights and encrypt the complemented data, realizing data sharing and updating, and ensuring the traceability and immutability of data. Thus, the problems of insufficient multi-modal integration and low feature utilization rate in the prior art are solved.

[0020] The following describes a method, apparatus, device, and storage medium for establishing a digestive tract database according to embodiments of the present application.

[0021] Specifically, Figure 1 is a schematic diagram of the method for establishing a digestive tract database provided by an embodiment of the present application.

[0022] As Figure 1 shown, the method for establishing the digestive tract database includes the following steps: In step S101, multi-modal digestive tract data and environmental information are obtained.

[0023] Among them, multi-modal digestive tract data refers to a composite data set that integrates multiple dimensions and forms such as digestive tract images, physiological signals, and biochemical indicators, and is used to comprehensively reflect the digestive tract structure, function, and physiological state.

[0024] It can be understood that in the embodiments of the present application, by integrating multi-dimensional information such as images, physiological signals, and biochemical indicators, the digestive tract structure, function, and physiological and pathological states are accurately presented, improving the accuracy of disease diagnosis, and providing a data basis for the research on the mechanism of digestive tract diseases and the optimization of treatment plans.

[0025] For example, in the diagnosis of digestive tract diseases in a certain medical institution, by integrating multi-modal data such as endoscopic images, electrophysiological signals, biochemical tests, and sensor monitoring, the physiological state, lesion characteristics, and functional abnormalities of the digestive tract are analyzed, providing multi-dimensional data support for accurately identifying early lesions and evaluating treatment effects.

[0026] In step S102, according to the multi-modal digestive tract data and environmental information, a spatio-temporal semantic fusion multi-dimensional data model is constructed and a causal relationship enhanced influence propagation graph is generated.

[0027] Among them, spatio-temporal semantics refers to the analysis and expression of the meaning contained in data, events or phenomena in the time and space dimensions, emphasizing the integration of spatio-temporal context and semantic information.

[0028] It can be understood that by integrating multi-modal digestive tract data and environmental information, the embodiments of the present application can comprehensively reflect the health status of patients and reduce misdiagnosis. The spatio-temporal semantic fusion model further helps doctors understand the disease evolution and formulate effective treatment plans. At the same time, these data also provide personalized management and prevention strategies for patients, realizing early warning and avoiding disease deterioration.

[0029] For example, in epidemic prevention and control, by integrating virus transmission path data at different time points (time dimension) and geographical information of the activity areas of infected persons (space dimension), combined with semantic features such as population flow and place types, an epidemic spatio-temporal transmission map is accurately drawn to identify high-risk areas and transmission chains, providing decision-making basis in the spatio-temporal dimension for formulating isolation measures, resource allocation and prevention and control strategies, and improving the timeliness and accuracy of epidemic monitoring and response.

[0030] In the embodiments of the present application, according to multi-modal digestive tract data and environmental information, a spatio-temporal semantic fusion multi-dimensional data model is constructed and a causal relationship enhanced influence propagation graph is generated, including: obtaining feature vectors based on different modal features of multi-modal digestive tract data, combining the time stamps in the environmental information, and encoding the time information into time feature vectors; based on the time feature vectors, combining the spatial environmental information, obtaining spatial feature vectors, and for the spatial feature vectors, through a semantic fusion gate, obtaining feature vectors integrating semantic information, and constructing a spatio-temporal semantic fusion multi-dimensional data model; based on the spatio-temporal semantic fusion multi-dimensional data model, using a graph neural network to analyze the causal relationship between data, obtaining causal association information, and based on the causal association information, through a multi-hop path reasoning mechanism, generating a causal relationship enhanced influence propagation graph.

[0031] Among them, the multi-hop path reasoning mechanism is a reasoning method that gradually conducts logical deductions on multiple associated paths to solve complex problems or mine deep relationships between data.

[0032] It can be understood that in the embodiments of the present application, by extracting different modal features of data and combining timestamp information, the time dimension is encoded as a feature vector, realizing the fusion of time and data features, enabling the model to capture the dynamic characteristics of data changing over time. By fusing spatial environmental information, a spatial feature vector is obtained, thus realizing the integration of the spatial dimension and data features. This multi-dimensional feature fusion enables the model to understand data more comprehensively. In terms of semantic fusion and model construction, the spatial feature vector is processed through a semantic fusion gate to obtain a feature vector integrating semantic information, enhancing the semantic understanding ability of the model. Based on this feature vector, a spatio-temporal semantic fusion multi-dimensional data model is constructed. This model comprehensively considers the time, space, and semantic characteristics of data, improving the expression ability and generalization ability of the model. Finally, in terms of causal relationship analysis and reasoning, a graph neural network is used to analyze the causal relationship between data to obtain causal association information, and a causal relationship enhanced influence propagation graph is generated through a multi-hop path reasoning mechanism, deepening the model's understanding of causal relationships and enabling it to predict and explain the mutual influence and propagation path between data.

[0033] For example, in a certain technology company, by applying the multi-hop path reasoning mechanism, a collaborative system consisting of a super brain and an intelligent cerebellum is constructed through the independently developed brain network software architecture for humanoid robot swarms. Among them, the super brain is based on a multi-modal embodied reasoning large model and relies on deep reasoning technology to be able to perform multi-hop logical deductions in complex production line-level tasks, achieve high-dimensional decision-making, and efficiently disassemble, schedule, and coordinate the operation processes of multiple robots; the intelligent cerebellum supports distributed learning and skill transfer through cross-field fusion perception and multi-robot cooperative control technology. This mechanism enables multiple WalkerS1 robots to successfully complete collaborative sorting, handling, and precision assembly tasks in multiple scenarios such as the general assembly workshop and the quality inspection area, promoting the evolution of humanoid robots from single-robot autonomy to swarm intelligence.

[0034] In step S103, based on the multi-modal digestive tract data, feature extraction is performed according to the cross-modal attention mechanism, and feature enhancement is performed in combination with the spatio-temporal semantic fusion multi-dimensional data model and the causal relationship enhanced influence propagation graph to obtain enhanced data features.

[0035] Among them, the cross-modal attention mechanism is an artificial intelligence technology that establishes associations between different data modalities, enabling the model to focus on the key information of other modalities, thereby enhancing the information understanding and interaction capabilities in multi-modal tasks.

[0036] It can be understood that in the embodiments of the present application, by establishing semantic associations and interactions between modalities, the comprehensive understanding ability of the model for complex scenarios is improved, the depth and accuracy of information fusion in multi-modal tasks are enhanced, enabling the model to more accurately capture the complementary features between different modalities, and optimizing decision support and intelligent applications.

[0037] In the embodiments of the present application, feature extraction is performed on multimodal digestive tract data based on a cross-modal attention mechanism, and feature enhancement is carried out by combining a spatio-temporal semantic fusion multi-dimensional data model and a causal relationship enhanced influence propagation graph to obtain enhanced data feature information, including: inputting the multimodal digestive tract data into a cross-modal attention model to obtain the attention weight distribution within each modality data; based on the attention weight distribution, obtaining the cross-modal attention fusion feature; according to the cross-modal attention fusion feature, combining the spatio-temporal semantic features in the spatio-temporal semantic fusion multi-dimensional data model to obtain the spatio-temporal semantic attention feature; performing feature enhancement on the spatio-temporal semantic attention feature and the causal association information in the causal relationship enhanced influence propagation graph to obtain the enhanced data feature information.

[0038] Among them, the influence propagation graph represents the relationships and information propagation paths between individuals or groups through nodes and edges, and is a graphical model used to analyze and predict the diffusion laws of information, opinions or behaviors in a network.

[0039] It can be understood that by inputting the multimodal digestive tract data into the cross-modal attention model in the embodiments of the present application, the attention weight distribution within each modality data can be calculated, thereby highlighting important information and suppressing irrelevant information. Based on these attention weight distributions, the cross-modal attention fusion feature is further obtained, realizing the effective fusion between different modality data. Combining the cross-modal attention fusion feature with the spatio-temporal semantic features in the spatio-temporal semantic fusion multi-dimensional data model, the spatio-temporal semantic attention feature is extracted, which contains the time, space, semantic and cross-modal association information of the data. Finally, feature enhancement processing is performed on the spatio-temporal semantic attention feature and the causal association information in the causal relationship enhanced influence propagation graph to obtain the enhanced data feature information, which helps to improve the accuracy and reliability of subsequent analysis or decision-making.

[0040] In step S104, based on the enhanced data features, a global annotation model is constructed based on federated learning, and the missing information in the multimodal digestive tract data is complemented through a hierarchical feature reconstruction and multi-model collaboration strategy, where federated learning includes distributed collaboration and privacy protection, hierarchical feature reconstruction includes coarse-grained feature recovery, fine-grained feature refinement and multimodal alignment, and the multi-model collaboration strategy includes model diversification and collaboration mechanism.

[0041] Among them, the global annotation model is a machine learning model that performs unified annotation by integrating multi-party distributed data based on the federated learning framework, ensuring data privacy, and realizing cross-institutional or cross-domain collaborative annotation and knowledge sharing on the premise of protecting data security.

[0042] It can be understood that in the embodiments of the present application, by utilizing the enhanced data features and constructing a global annotation model based on federated learning technology, multi-party collaboration and knowledge sharing are achieved while protecting data privacy. At the same time, a hierarchical feature reconstruction strategy is adopted, including coarse-grained feature restoration, fine-grained feature refinement, and multimodal alignment, to effectively complete the missing information in multimodal digestive tract data. In addition, a multi-model collaboration strategy is introduced. Through model diversification and collaboration mechanisms, the advantages of different models are fully utilized to improve the accuracy and efficiency of data processing.

[0043] In the embodiments of the present application, according to the enhanced data features, a global annotation model is constructed based on federated learning, and the missing information in multimodal digestive tract data is completed through hierarchical feature reconstruction and multi-model collaboration strategies, including: obtaining local model parameters; encrypting and transmitting the local model parameters, and aggregating them at the central node, and constructing a global annotation model according to the aggregated data; based on the global annotation model, sequentially performing coarse-grained feature restoration and fine-grained feature refinement on the missing information in multimodal digestive tract data to obtain a missing information restoration result; according to the missing information restoration result, fusing the processing results of different models to complete the completion of the missing information.

[0044] Among them, the local model parameters refer to the numerical variables learned through training in a machine learning model running on a local device, which are used to describe the relationship between data features and prediction results, and directly determine the performance and prediction ability of the model.

[0045] It can be understood that in the embodiments of the present application, by obtaining local model parameters and encrypting and transmitting them, the central node aggregates them to construct a global annotation model; using this model to perform coarse-grained and fine-grained feature restoration on the missing information in multimodal digestive tract data to obtain a restoration result; finally, fusing the processing results of different models to complete the completion of the missing information.

[0046] For example, as Figure 2 shown, a certain technology co., ltd., based on the independently developed AI module MS372Q, which adopts an 8-core 64-bit heterogeneous architecture, has a powerful computing power of 12 TOPS, supports a variety of industrial interfaces, and can efficiently complete model inference and operation in scenarios such as low-altitude economy, autonomous driving, and industrial robots. Through local deployment, BILIN ensures data privacy and security, improves the real-time response speed of terminal devices. When actually running a model with a parameter scale of 1.5B, the Token generation speed reaches 15 Tokens / s, providing solutions for edge computing and local intelligent applications, and promoting the large-scale implementation and innovative application of AI technology in complex scenarios.

[0047] In step S105, the blockchain technology is used to confirm the rights and encrypt the complemented multi-modal digestive tract data, and sharing and updating are carried out based on the blockchain technology. Among them, the blockchain technology includes data hash on-chain storage, smart contracts, and homomorphic encryption.

[0048] Among them, the blockchain technology is a computer technology that records data through a decentralized distributed ledger, uses cryptography to ensure immutability, transparency and traceability, and realizes secure sharing through a consensus mechanism.

[0049] It can be understood that in the embodiment of the present application, the blockchain technology is used to confirm the rights and encrypt the complemented multi-modal digestive tract data to ensure the security and traceability of the data; at the same time, based on the blockchain technology, the sharing and updating of the data are realized, and technical means such as data hash on-chain storage, smart contracts, and homomorphic encryption are used to improve the efficiency and security of data sharing, providing a strong guarantee for the compliant use and management of data.

[0050] In the embodiment of the present application, confirming the rights of the complemented multi-modal digestive tract data by using the blockchain technology includes: calculating the complemented multi-modal digestive tract data according to the hash algorithm to obtain a unique data identifier, storing the unique data identifier into the blockchain, completing the right confirmation operation, and obtaining a right confirmation result; writing a smart contract according to the right confirmation result and the blockchain, deploying the smart contract to the blockchain network, and using an automatic execution mechanism to associate the data access permission rules with the right confirmation result to achieve access control.

[0051] Among them, the hash algorithm is an algorithm that converts input data of any length into a hash value of a fixed length, has irreversibility, and different inputs usually produce unique hash values.

[0052] It can be understood that in the embodiment of the present application, by mapping any data to a unique and fixed-length hash value, the data integrity is efficiently verified, and the data tampering is prevented; its irreversibility makes it a security barrier for password storage to avoid the leakage of the original information; at the same time, the hash algorithm supports fast data indexing and retrieval, improves the data processing efficiency, and is applied to fields such as information security, blockchain, and file verification, providing a security and efficiency guarantee for data management and transmission.

[0053] For example, as Figure 3As shown, a foreign technology company combined the efficient compression characteristics of the Huffman algorithm with the fast search ability of the hash algorithm, and introduced a cache to optimize the data storage and retrieval process, successfully achieving efficient compression and fast search of similar objects in the EDA field; by performing Huffman coding on the data to generate compressed binary data, and then using the hash algorithm to generate unique hash values for indexing, the data processing efficiency was significantly improved, providing a solution for large-scale data management in complex EDA scenarios, demonstrating the key role of the hash algorithm in optimizing data storage and retrieval.

[0054] In the embodiment of the present application, after implementing access control, it includes: encrypting the complemented multi-modal digestive tract data based on the homomorphic encryption algorithm to obtain the encrypted data; sharing and updating the encrypted data on the blockchain, where when the encrypted data is updated, the update content is recorded based on the timestamp and the hash chain to obtain the updated blockchain data record.

[0055] It can be understood that the embodiment of the present application supports encrypted state computing under the premise of ensuring data privacy through homomorphic encryption, ensuring the security of sensitive medical data during the sharing process; using the decentralized characteristics of the blockchain and the timestamp and hash chain mechanisms to achieve transparent recording and immutability of data updates, enhancing the traceability and integrity of data; at the same time, the distributed storage and consensus mechanism of the blockchain provides a sharing platform for multi-modal medical data, meeting the requirements for data security and privacy protection in the medical field.

[0056] According to the method for establishing a digestive tract database proposed in the embodiment of the present application, by constructing a spatio-temporal semantic fusion multi-dimensional data model, the comprehensive analysis and in-depth fusion of multi-source data are realized. A causal relationship enhanced influence propagation graph is generated, providing rich information for feature extraction and data analysis. In the feature extraction stage, cross-modal attention mechanism is combined for feature enhancement, improving the accuracy and robustness of data features. A global annotation model is constructed based on federated learning, protecting data privacy and improving the generalization ability and annotation accuracy of the model. At the same time, missing information is effectively complemented through hierarchical feature reconstruction and multi-model collaboration strategies. Finally, blockchain technology is used to confirm the rights and encrypt the complemented data, realizing data sharing and updating, and ensuring the traceability and immutability of data. Thus, the problems of insufficient multi-modal integration and low feature utilization rate in the prior art are solved.

[0057] The method for establishing a digestive tract database will be elaborated below through a specific embodiment. Taking the digestive tract disease diagnosis and treatment center of a certain tertiary hospital as an example, as Figure 4 shown, it includes the following content: Step 1: Obtain multi-modal digestive tract data and environmental information; Data collection: Gastrointestinal Tract Images: Mucosal images of the patient's esophagus, stomach, and intestines are collected through a high-definition electronic endoscope with a resolution of 4K, and visual features such as the location and morphology of lesions are recorded; Physiological Signals: Gastric and intestinal electrical signals are obtained using a gastrointestinal motility monitoring device with a sampling frequency of 100Hz to capture the peristaltic rhythm of the gastrointestinal tract; Biochemical Indicators: Biochemical data such as pepsinogen and carcinoembryonic antigen (CEA) detected in blood tests, as well as fecal occult blood test results, are collected; Environmental Information: Spatial environmental information such as the inspection timestamp (accurate to seconds), the patient's visiting department (gastroenterology outpatient clinic / inpatient department), and the inspection equipment number are recorded.

[0058] Step 2: Construct a spatio-temporal semantic fusion multi-dimensional data model and generate a causal relationship enhanced influence propagation graph; Feature Processing: Extract the texture features of endoscopic images (such as LBP features), the frequency domain features of physiological signals (spectrum after FFT transformation), and the numerical features of biochemical indicators, and encode them into time feature vectors in combination with timestamps (such as converting time into temporal embedding vectors); Integrate spatial information such as department location and equipment distribution to generate spatial feature vectors, which are processed through a semantic fusion gate (such as a gated recurrent unit GRU) to obtain feature vectors with fused semantics, and construct a multi-dimensional data model including time, space, and semantics. Causal Analysis: Use a graph neural network (such as GCN) to analyze the causal relationships in the data. For example, discover the association between abnormal gastric electrical signals and decreased pepsinogen levels; Through multi-hop path reasoning, generate a causal relationship enhanced influence propagation graph, such as the propagation path of "gastric mucosal inflammation → gastric electrical rhythm disorder → digestive function decline".

[0059] Step 3: Perform feature extraction and enhancement based on the cross-modal attention mechanism; Feature Extraction: Input endoscopic images, gastric electrical signals, and biochemical indicators into a cross-modal attention model to calculate the internal attention weights of each modality. For example, the weight of the lesion area in the endoscopic image is increased, and the weight of the abnormal rhythm segment in the gastric electrical signal is highlighted; Integrate the cross-modal attention features and combine them with the spatio-temporal semantic features of the spatio-temporal semantic model to obtain spatio-temporal semantic attention features (such as the association features between the lesion image and abnormal physiological signals of a patient at a certain time point and in a certain department). Feature Enhancement: Integrate the causal association information in the causal relationship enhanced influence propagation graph (such as the causal features between inflammation indicators and function indicators) to enhance the richness of the data features.

[0060] Step 4: Construct a global annotation model based on federated learning and complete the missing information; Federated Learning Collaboration: The local model parameters of each department in the hospital (outpatient clinic, inpatient department, laboratory center) (such as the convolution kernel parameters of the endoscopic image classification model) are encrypted and transmitted to the central node, and aggregated to construct a global annotation model; Hierarchical Feature Reconstruction: For the missing intestinal flora detection data, first perform coarse-grained feature recovery (fill in based on the mean flora distribution of similar patients), and then perform fine-grained refinement (adjust the filled values in combination with the patient's diet and medication records); Multi-Model Collaborative Completion: Integrate the prediction results of the random forest model and the generation results of the deep learning model to complete the missing digestive tract motility function score data.

[0061] Step Five: Perform data right confirmation, encryption, sharing, and updating based on blockchain technology; Right Confirmation and Encryption: Calculate the unique identifier of the completed data through the SHA-256 hash algorithm, such as "0x5a3f...9c2b", and store it in the blockchain to complete the right confirmation; Write a smart contract, set the access rule of "only digestive doctors have the right to read the complete patient data", and deploy it to the blockchain network; use a homomorphic encryption algorithm (such as Paillier) to encrypt the data to ensure that statistical analysis can be performed in the ciphertext state (such as calculating the mean CEA of the patient group); Sharing and Updating: When the patient's review data is updated, record the timestamp "2024-08-01 10:30:00", link the old and new data through a hash chain, and generate an updated blockchain record for the research team to share and analyze under authorization.

[0062] Through the embodiments of this application, the multi-modal data parsing efficiency is increased by 40%, the early gastric cancer screening accuracy rate is increased from 75% to 88%; the missing data completion accuracy rate reaches 92%, providing complete data support for the formulation of personalized diagnosis and treatment plans for digestive tract diseases; based on the blockchain-based sharing mechanism, cross-hospital scientific research collaboration is realized, and the risk of privacy leakage during the data sharing process approaches zero, promoting the research progress of digestive tract diseases.

[0063] Through the integration of multi-modal data and environmental information, the present invention constructs a spatio-temporal semantic model and a causal relationship map, combines a cross-modal attention mechanism and a federated learning global annotation model, realizes efficient feature enhancement of digestive tract data, completion of missing information under privacy protection, and ensures data right confirmation, encryption, and traceable sharing with the help of blockchain technology, improving the analysis accuracy, privacy security, and cross-institutional collaboration efficiency of digestive tract data, and providing technical support for the precise diagnosis and research of digestive tract diseases.

[0064] Secondly, describe the establishment device of the digestive tract database according to the embodiments of the present application with reference to the accompanying drawings.

[0065] Figure 5 It is a block diagram of the establishment device of the digestive tract database according to the embodiments of the present application.

[0066] As Figure 5As shown in the figure, the apparatus 10 for establishing the digestive tract database includes: an acquisition module 100, a construction module 200, an enhancement module 300, a completion module 400, and an update module 500.

[0067] Among them, the acquisition module 100 is used to acquire multi-modal digestive tract data and environmental information; the determination module 200 is used to construct a spatio-temporal semantic fusion multi-dimensional data model and generate a causal relationship enhanced influence propagation graph according to the multi-modal digestive tract data and environmental information; the warning module 300 is used to extract features based on the cross-modal attention mechanism according to the multi-modal digestive tract data, and perform feature enhancement in combination with the spatio-temporal semantic fusion multi-dimensional data model and the causal relationship enhanced influence propagation graph to obtain enhanced data features; the completion module 400 is used to construct a global annotation model based on federated learning according to the enhanced data features, and complete the missing information in the multi-modal digestive tract data through a hierarchical feature reconstruction and multi-model collaboration strategy, where federated learning includes distributed collaboration and privacy protection, hierarchical feature reconstruction includes coarse-grained feature recovery, fine-grained feature refinement and multi-modal alignment, and the multi-model collaboration strategy includes model diversification and collaboration mechanism; the update module 500 is used to confirm the rights and encrypt the completed multi-modal digestive tract data using blockchain technology, and perform sharing and updating based on blockchain technology, where blockchain technology includes data hash chain storage, smart contract and homomorphic encryption.

[0068] It should be noted that the foregoing explanation of the embodiments of the method for establishing the digestive tract database also applies to the apparatus for establishing the digestive tract database in this embodiment, and will not be elaborated here.

[0069] According to the apparatus for establishing the digestive tract database provided by the embodiments of the present application, by constructing a spatio-temporal semantic fusion multi-dimensional data model, comprehensive analysis and deep fusion of multi-source data are realized. Generating a causal relationship enhanced influence propagation graph provides rich information for feature extraction and data analysis. In the feature extraction stage, feature enhancement is combined with the cross-modal attention mechanism to improve the accuracy and robustness of data features. Constructing a global annotation model based on federated learning protects data privacy and improves the generalization ability and annotation accuracy of the model. At the same time, missing information is effectively completed through hierarchical feature reconstruction and multi-model collaboration strategy. Finally, blockchain technology is used to confirm the rights and encrypt the completed data, realizing data sharing and updating, and ensuring the traceability and immutability of data. Thus, the problems of insufficient multi-modal integration and low feature utilization rate in the prior art are solved.

[0070] Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include: A memory 601, a processor 602, and a computer program stored in the memory 601 and executable on the processor 602.

[0071] When the processor 602 executes the program, it implements the method for establishing the digestive tract database provided in the above embodiments.

[0072] Furthermore, the electronic device further includes: A communication interface 603 for communication between the memory 601 and the processor 602.

[0073] A memory 601 for storing computer programs that can run on the processor 602.

[0074] The memory 601 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0075] If the memory 601, the processor 602, and the communication interface 603 are implemented independently, the communication interface 603, the memory 601, and the processor 602 can be interconnected through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0076] Optionally, in a specific implementation, if the memory 601, the processor 602, and the communication interface 603 are integrated on a single chip, the memory 601, the processor 602, and the communication interface 603 can communicate with each other through an internal interface.

[0077] The processor 602 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0078] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method for establishing the digestive tract database as described above.

[0079] In addition, an embodiment of the present application further provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed, the method for establishing the digestive tract database described above is implemented.

[0080] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0081] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0082] Any process or method description in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a customized logic function or process. And the scope of the preferred embodiments of the present application includes additional implementations, where the functions may be executed in a manner substantially simultaneous with or in the reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art of the embodiments of the present application.

[0083] It should be understood that each part of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following well-known technologies in the art or a combination thereof can be used: discrete logic circuits with logic gate circuits for implementing logic functions on data signals, application-specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0084] Those of ordinary skill in the art can understand that all or part of the steps carried out in the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0085] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for establishing a digestive tract database, characterized in that, The method includes: Obtaining multimodal digestive tract data and environmental information; Constructing a spatio-temporal semantic fusion multi-dimensional data model and generating a causal relationship enhanced influence propagation graph according to the multimodal digestive tract data and the environmental information; Performing feature extraction on the multimodal digestive tract data based on a cross-modal attention mechanism, and combining the spatio-temporal semantic fusion multi-dimensional data model and the causal relationship enhanced influence propagation graph for feature enhancement to obtain enhanced data features; Based on the enhanced data features, constructing a global annotation model based on federated learning, and complementing the missing information in the multimodal digestive tract data through a hierarchical feature reconstruction and multi-model collaboration strategy, where the federated learning includes distributed collaboration and privacy protection, the hierarchical feature reconstruction includes coarse-grained feature recovery, fine-grained feature refinement, and multimodal alignment, and the multi-model collaboration strategy includes model diversification and a collaboration mechanism; Using blockchain technology to confirm the rights and encrypt the complemented multimodal digestive tract data, and sharing and updating based on the blockchain technology, where the blockchain technology includes data hash chain storage, smart contracts, and homomorphic encryption.

2. The method for establishing a digestive tract database according to claim 1, wherein Using blockchain technology to confirm the rights of the complemented multimodal digestive tract data includes: Calculating the complemented multimodal digestive tract data according to the hash algorithm to obtain a unique data identifier; Storing the unique data identifier in the blockchain, completing the right confirmation operation, and obtaining a right confirmation result; Writing a smart contract according to the right confirmation result and the blockchain, deploying the smart contract to the blockchain network, and using an automatic execution mechanism to associate the data access permission rules with the right confirmation result to achieve access control.

3. The method for establishing a digestive tract database according to claim 2, characterized in that, After achieving access control, it includes: Encrypting the complemented multimodal digestive tract data based on the homomorphic encryption algorithm to obtain encrypted data; Sharing and updating the encrypted data on the blockchain, where when the encrypted data is updated, the update content is recorded based on a timestamp and a hash chain to obtain an updated blockchain data record.

4. The method for establishing a digestive tract database according to claim 1, wherein Based on the enhanced data features, constructing a global annotation model based on federated learning, and complementing the missing information in the multimodal digestive tract data through a hierarchical feature reconstruction and multi-model collaboration strategy, including: Obtaining local model parameters; Encrypting and transmitting the local model parameters, summarizing them at the central node, and constructing a global annotation model according to the summarized data; Based on the global annotation model, sequentially performing coarse-grained feature recovery and fine-grained feature refinement on the missing information in the multimodal digestive tract data to obtain a missing information recovery result; According to the missing information recovery result, fusing the processing results of different models to complete the complementation of the missing information.

5. The method for establishing a digestive tract database according to claim 1, wherein ,Performing feature extraction on the multimodal digestive tract data based on a cross-modal attention mechanism, and combining the spatio-temporal semantic fusion multi-dimensional data model and the causal relationship enhanced influence propagation graph for feature enhancement to obtain enhanced data feature information, including: Inputting the multimodal digestive tract data into a cross-modal attention model to obtain the attention weight distribution inside each modal data; Based on the attention weight distribution, obtain cross-modal attention fusion features; According to the cross-modal attention fusion features, combine with the spatio-temporal semantic features in the spatio-temporal semantic fusion multi-dimensional data model to obtain spatio-temporal semantic attention features; Perform feature enhancement on the spatio-temporal semantic attention features and the causal association information in the causal relationship enhanced influence propagation graph to obtain enhanced data feature information.

6. The method for establishing a digestive tract database according to claim 1, wherein According to the multi-modal digestive tract data and the environmental information, construct a spatio-temporal semantic fusion multi-dimensional data model and generate a causal relationship enhanced influence propagation graph, including: Obtain feature vectors based on different modal features of multi-modal digestive tract data, and combine with the timestamp in the environmental information to encode the time information into a time feature vector; Based on the time feature vector, combine with the spatial environmental information to obtain a spatial feature vector. For the spatial feature vector, through a semantic fusion gate, obtain a feature vector integrating semantic information, and construct a spatio-temporal semantic fusion multi-dimensional data model; Based on the spatio-temporal semantic fusion multi-dimensional data model, use a graph neural network to analyze the causal relationship between data to obtain causal association information. Based on the causal association information, through a multi-hop path reasoning mechanism, generate a causal relationship enhanced influence propagation graph.

7. An apparatus for establishing a digestive tract database, characterized in that, Including: An acquisition module for acquiring multi-modal digestive tract data and environmental information; A construction module for constructing a spatio-temporal semantic fusion multi-dimensional data model and generating a causal relationship enhanced influence propagation graph according to the multi-modal digestive tract data and the environmental information; An enhancement module for performing feature extraction based on the multi-modal digestive tract data using a cross-modal attention mechanism, and combining with the spatio-temporal semantic fusion multi-dimensional data model and the causal relationship enhanced influence propagation graph for feature enhancement to obtain enhanced data features; A completion module for constructing a global annotation model based on federated learning according to the enhanced data features, and complementing the missing information in the multi-modal digestive tract data through a hierarchical feature reconstruction and multi-model collaboration strategy, where the federated learning includes distributed collaboration and privacy protection, the hierarchical feature reconstruction includes coarse-grained feature restoration, fine-grained feature refinement, and multi-modal alignment, and the multi-model collaboration strategy includes model diversification and collaboration mechanism; An update module for authenticating and encrypting the complemented multi-modal digestive tract data using blockchain technology, and sharing and updating based on the blockchain technology, where the blockchain technology includes data hash on-chain storage, smart contracts, and homomorphic encryption.

8. An electronic device, characterized in that, Including: A memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method for establishing a digestive tract database according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instruction is executed by the processor, it is used to implement the method for establishing a digestive tract database according to any one of claims 1-6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instruction is executed, it implements the method for establishing a digestive tract database according to any one of claims 1-6.

Images