Multi-modal process data trusted storage method and system combined with block chain

Abstract

The invention provides a multi-modal process data trusted storage method and system combined with a block chain, and belongs to the technical field of data storage, and the method comprises the following steps: obtaining multi-modal process data; constructing a metadata minimum set covering the full life cycle of the multi-modal process data; performing under-chain storage on the multi-modal process data and the metadata minimum set, and verifying the multi-modal process data to obtain verification process data; recording the digital signature evidence of the verification process data on a block chain to obtain an on-chain shared signature evidence of the verification process data; based on the on-chain shared signature evidence, establishing a mapping relationship between the hash value of the on-chain shared signature evidence and the under-chain verification process data; and storing the verification process data and the mapping relationship in a distributed process database. According to the method, the block chain and the distributed storage are combined to construct a multi-modal process data trusted storage system with coordination of on-chain and off-chain, so that the trusted storage of the multi-modal process data in the whole life cycle is realized.

Description

Technical Field

[0001] The present invention relates to the field of data storage technology, and in particular to a multimodal process data trusted storage method and system combined with blockchain. Background Art

[0002] Process data refers to the detailed records of data generation, use, transmission, storage, verification and destruction in a specific process or system, covering all operations and changes of data throughout its life cycle. In the field of intelligent computing services, due to the large amount and diverse types of multimodal process data (including text, images, videos, structured data, etc.), as well as the need to ensure the integrity, authenticity and traceability of data throughout its life cycle, the management and storage of multimodal process data faces major challenges.

[0003] In the context of existing technologies, in order to ensure the security and credibility of multimodal process data, the following methods are usually used for data management: first, data is stored and managed through off-chain storage; second, the data stored off-chain is secured through access control, data encryption and other means; finally, to ensure the credibility of the data, some systems have introduced blockchain technology to store data hash summaries and digital signatures on the chain, thereby providing guarantees that the data cannot be tampered with and is traceable.

[0004] The existing practices mainly have the following problems: First, although blockchain technology has the advantage of being tamper-proof when used for trusted data storage, due to the huge volume of multimodal data, it is not feasible to directly store all of it on the chain, and it will also lead to excessive burdens of on-chain storage and calculation. Therefore, existing technologies usually store the data itself off-chain and only store the hash value on the chain. However, this method often lacks efficient association between on-chain and off-chain data during data management, positioning and access, making it difficult to efficiently locate and retrieve data during the access process. Secondly, although data stored off-chain can provide a certain degree of security through access control and other methods, the data may still be subject to tampering risks at multiple stages of the entire life cycle.

[0005] Therefore, how to achieve trusted storage of multimodal process data throughout its entire life cycle has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The present invention provides a trusted storage method, system, electronic device and storage medium for multimodal process data combined with blockchain, so as to solve the defects in the prior art and realize the trusted storage of multimodal process data throughout its life cycle.

[0007] The present invention provides a trusted storage method for multimodal process data combined with blockchain, comprising the following steps: obtaining multimodal process data; Constructing a minimum set of metadata covering the entire life cycle of the multimodal process data; The multimodal process data and the minimum set of metadata are stored off-chain, and the multimodal process data is verified to obtain verification process data; The digital signature of the verification process data is recorded on the blockchain to obtain the on-chain shared signature of the verification process data; Based on the on-chain shared signature evidence, a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data is established; The verification process data and the mapping relationship are stored in a distributed process database.

[0008] According to a multimodal process data trusted storage method combined with blockchain provided by the present invention, the multimodal process data is verified to obtain the verification process data, specifically including: Performing authenticity check, integrity check, compliance information check and ownership confirmation information check on the multimodal process data to obtain verification information; According to the minimum set of metadata, the verification information and the multimodal process data are integrated in the form of metadata to obtain the verification process data.

[0009] According to a multimodal process data trusted storage method combined with blockchain provided by the present invention, the digital signature evidence includes a data summary, a timestamp and signature information; the digital signature evidence of the verification process data is recorded on the blockchain to obtain the on-chain shared signature evidence of the verification process data, specifically including: Calculating a first hash value of the verification process data; Using the first hash value as the data digest and generating a timestamp for the data digest; Signing the data digest and the timestamp based on a private key associated with the multimodal process data to generate the signature information; The data summary, the timestamp and the signature information are recorded on the blockchain to obtain the on-chain shared signature evidence of the verification process data.

[0010] According to a multimodal process data trusted storage method combined with blockchain provided by the present invention, based on the on-chain shared signature evidence, a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data is established, specifically including: Extracting a first hash value of the shared signature certificate on the chain from the blockchain, and using the first hash value as the on-chain identifier; Determining a uniform resource locator corresponding to off-chain storage of the multimodal process data and the minimum set of metadata; Based on a preset data structure, a mapping relationship between the on-chain identifier and the uniform resource locator is established, wherein the on-chain identifier is used as a key of the mapping relationship, and the uniform resource locator is used as a value of the mapping relationship.

[0011] According to a multimodal process data trusted storage method combined with blockchain provided by the present invention, the distributed process database includes a process data storage interface, a process data load balancing mechanism and a distributed database; the verification process data and the mapping relationship are stored in the distributed process database, specifically including: Based on the process data load balancing mechanism, the verification process data and the mapping relationship are distributed to each database node in the distributed database for storage through the process data storage interface, wherein the process data load balancing mechanism distributes the verification process data and the mapping relationship to the appropriate database node according to the data characteristics and the performance of each database node.

[0012] According to a multimodal process data trusted storage method combined with blockchain provided by the present invention, the method further includes: When a user accesses the multimodal process data, a first hash value of the shared signature evidence on the chain and a second hash value of the multimodal process data stored off-chain are obtained; Calculating a third hash value of the multimodal process data; Compare the third Hash value with the first Hash value and the second Hash value respectively to determine whether the third Hash value is consistent with the first Hash value and / or the second Hash value; When the third hash value is consistent with the first hash value, and / or the second hash value, allowing the user to access the multimodal process data; When the third hash value is inconsistent with both the first hash value and the second hash value, the user is denied access to the multimodal process data.

[0013] The present invention also provides a multimodal process data trusted storage system combined with blockchain, comprising the following modules: an acquisition module and a processing module; The acquisition module is used to acquire multimodal process data; The processing module is used to construct a minimum set of metadata covering the entire life cycle of the multimodal process data; The processing module is further used to store the multimodal process data and the minimum set of metadata off-chain, and verify the multimodal process data to obtain verification process data; The processing module is further used to record the digital signature of the verification process data on the blockchain to obtain the on-chain shared signature of the verification process data; The processing module is further used to establish a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data based on the on-chain shared signature evidence; The processing module is further used to store the verification process data and the mapping relationship in a distributed process database.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a multimodal process data trusted storage method combined with blockchain as described in any one of the above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a multimodal process data trusted storage method combined with blockchain as described in any one of the above.

[0016] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the multimodal process data trusted storage methods combined with blockchain as described above.

[0017] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: By constructing a minimum set of metadata covering the entire life cycle of multimodal process data, a standardized data description framework is established to record the key information of each data in its life cycle, thereby ensuring that the status, operation records, and ownership information of each multimodal process data can be uniformly managed and tracked throughout the life cycle, improving the accuracy and operability of data management. By storing the multimodal process data and the minimum set of metadata off-chain and verifying the multimodal process data, the verification process data is obtained. First, the authenticity, integrity, and ownership information of the data are verified to ensure that the data has been fully verified before storage, thereby reducing the security risks in data storage and ensuring the security and credibility of off-chain data storage. By recording the digital signature of the verification process data on the blockchain, the on-chain shared signature of the verification process data is obtained. By using the tamper-proof characteristics of the blockchain, the signature, hash value, and timestamp of the data are stored on the chain, thereby providing a transparent storage certificate for each verification process data, ensuring the tamper-proof and traceability of the data, and enhancing data security. By establishing a mapping relationship between the hash value of the shared signature evidence on the chain and the off-chain verification process data based on the on-chain shared signature evidence, the system builds an effective connection between the on-chain and the off-chain, so that the on-chain evidence can quickly locate the actual data resources stored off-chain, thereby realizing on-chain and off-chain collaboration, ensuring efficient management and fast query of data throughout the life cycle, and improving the efficiency and accuracy of data access. By storing the verification process data and mapping relationships in a distributed process database, relying on the scalability and efficient query capabilities of the distributed database, the verification process data and its mapping relationships are stored on multiple nodes, thereby realizing the trusted storage of multimodal process data throughout the life cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is a schematic diagram of a multimodal process data trusted storage method combined with blockchain provided by the present invention.

[0020] Figure 2 This is one of the flow charts of the multimodal process data trusted storage method combined with blockchain provided by the present invention.

[0021] Figure 3 It is a schematic diagram of the minimum set of metadata fields and identifiers provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the distributed process database provided by the present invention.

[0023] Figure 5 It is a structural diagram of the multimodal process data trusted storage system combined with blockchain provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] It should be noted that, in the description of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the system or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0027] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" means at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0028] Combine the following Figure 1-Figure 6The present invention describes the trusted storage method, system, electronic device and storage medium of multimodal process data combined with blockchain. Figure 1 As shown, Figure 1 This is a schematic diagram of a multimodal process data trusted storage method combined with blockchain provided by the present invention. The server side provides a remote call interface based on services such as HTTP, and a distributed process database is deployed on the server side.

[0029] Reference Figure 2 , Figure 2 This is one of the flow charts of the multimodal process data trusted storage method combined with blockchain provided by the present invention, and the method includes but is not limited to the following steps: Step 1: Acquire multimodal process data.

[0030] Specifically, multimodal process data refers to data from different data sources and different modalities (such as text, images, audio, video, sensor data, etc.), which are continuously generated in intelligent computing services or complex processes and involve operation records at different stages.

[0031] Specifically, the acquisition of multimodal process data is not just a simple data collection, but a dynamic process covering the entire life cycle of data. In intelligent computing services, data is usually accompanied by different states and changes, such as training data, inference results, data changes in the model generation process, etc., which are all part of multimodal process data. Therefore, when acquiring this data, the continuity and integrity of the data from generation to final storage must be guaranteed.

[0032] Step 2: Construct a minimal set of metadata that covers the entire life cycle of multimodal process data.

[0033] The implementation example of step 2 is to construct a minimum set of metadata covering the entire life cycle of multimodal process data. The purpose is to provide a standardized description for each multimodal process data and each link in its entire life cycle, thereby ensuring that the data maintains consistency and integrity throughout the entire generation, use, storage and verification process, and supports collaborative storage and verification on and off the chain.

[0034] In intelligent computing services, multimodal process data includes text, images, videos, model prediction results and other forms, and these data will be processed multiple times in multiple stages such as generation, operation, transmission, storage, and verification. In order to ensure the security and credibility of data, it is necessary to design a structured metadata model to record the core information of these data. The key purpose of building a minimal set of metadata is to simplify the data description process, ensure the uniqueness, integrity, and clarity of data ownership, and reduce unnecessary data redundancy and complexity.

[0035] In order to simplify the description and processing, the minimum set of metadata is usually encapsulated in JSON format. This structured format can be used to uniformly describe and process the various attributes of data resources. The field design of the minimum set of metadata not only refines the data description, but also covers the key information of each data in the generation, transmission and use of the intelligent computing process, effectively reducing the storage and transmission burden on and off the chain.

[0036] Reference Figure 3 , Figure 3 It is a schematic diagram of the minimum set of metadata fields and identifiers provided by the present invention. Figure 3 The fields are described as follows: 1. Data Resource ID (DID): Meaning: Used to uniquely identify each data resource.

[0037] Format: String.

[0038] Content Description: A unique identifier, which is a linear number in the order in which it is generated.

[0039] 2. Data resource location information (DataLocation): Meaning: Describes the storage location of data resources.

[0040] Format: String.

[0041] Content description: Uniform resource description URL.

[0042] 3. Data resource description (DataDescription): Meaning: Provides a brief description of the data resource, etc.

[0043] Format: String.

[0044] Content description: data resource type (data, model, computing power), data resource category (relational database storage, big data format data, model prediction results, etc.), data resource form (support multi-modality, such as text, image, video, structured data, etc.).

[0045] 4. Data resource usage time (UsageTime): Meaning: Records the usage time of data resources.

[0046] Format: DateTime.

[0047] Content description: The start and end time of data resource use, in the ISO8601 format, for example: 2024-08-03T10:15:30Z.

[0048] 5. Data resource owner information (OwnerInfo): Meaning: Describes the owner information of the data resource.

[0049] Format: JSON object (JSONObject).

[0050] Content description: Relevant information of the data resource owner, such as the owner's ID, name, contact information, etc., for example: {"ownerID":"SJTU-user1","name":"seiee-ljh","contact":"seiee-ljh@example.com"}.

[0051] 6. Integrity Verification Information (IntegrityVerification): Meaning: Provides integrity verification information for data resources.

[0052] Format: JSON object (JSONObject).

[0053] Content description: Describe the integrity verification method and verification result, for example: {"method":"SHA-256","result":"abc123"}.

[0054] 7. Ownership Information (OwnershipInfo): Meaning: Records the ownership information of data resources.

[0055] Format: JSON object (JSONObject).

[0056] Content description: Describes the relevant information of title confirmation, such as owner ID, title confirmation time, etc. The format is: {"ownerID":"user123","timestamp":"2024-08-03T10:15:30Z"}.

[0057] 8. Data resources participating in intelligent computing information (AIComputationInfo): Meaning: Records information related to data resources participating in intelligent computing.

[0058] Format: JSON object (JSONObject).

[0059] Content description: describes the intelligent computing task ID, computing time, computing results, etc., for example: {"taskID":"task123","computeTime":"2024-08-03T11:00:00Z","result":"success"}.

[0060] 9. Acquisition and storage of life cycle information (LifecycleInfo): Meaning: Describes the life cycle information of data resources during the collection, storage and use process.

[0061] Format: JSON object (JSONObject).

[0062] Content description: including the creation time, modification time, storage time and usage time of the data, for example: {"createdTime":"2024-08-01T08:00:00Z","modifiedTime":"2024-08-02T09:00:00Z","storageTime":"2024-08-01T08:00:00Z","usageTime":"2024-08-03T10:15:30Z"}.

[0063] 10.Data resource hash value (DataHash): Meaning: Provides the hash value of data resources to verify data integrity and prevent tampering.

[0064] Format: String.

[0065] Content description: The hash value of a data resource is used to retrieve data and verify information ownership, such as a hash string generated using the SHA-256 algorithm.

[0066] Step 3: Store the minimum set of multimodal process data and metadata off-chain, and verify the multimodal process data to obtain verified process data.

[0067] The implementation example of step 3 is to store the minimum set of multimodal process data and metadata off-chain and verify the multimodal process data. The purpose is to ensure the authenticity, integrity, compliance and ownership of the multimodal process data when it is stored off-chain, so as to ensure the security and reliability of the data throughout its life cycle, and lay the foundation for subsequent on-chain signature storage and on-chain and off-chain collaborative storage.

[0068] In a possible implementation, step 3 specifically includes the following steps: Perform authenticity checks, integrity checks, compliance information checks, and ownership information checks on multimodal process data to obtain verification information.

[0069] According to the minimum set of metadata, the verification information is integrated with the multimodal process data in the form of metadata to obtain the verification process data.

[0070] Specifically, first, an authenticity check is performed to ensure that the source and generation process of the multimodal process data are legal and credible. The key to authenticity verification is to verify that the data is generated or operated by an authorized party and has not been tampered with or forged. This is usually achieved through a signature mechanism. The specific approach is to sign the data using the private key of the entity that generates the data, generate a digital signature, and verify the validity of the signature through the public key. If the signature verification passes, it proves that the identity of the data generator is credible, and the data source is legal and has not been tampered with. This process ensures that the multimodal process data processed by the system is authentic and reliable.

[0071] Next, an integrity check is performed to ensure that the data has not been tampered with or damaged during generation, transmission, and storage. Integrity checks are usually implemented by calculating hash values ​​(such as the SHA-256 algorithm). The hash value of the multimodal process data is calculated and compared with the pre-stored hash value of the original data. If the hash values ​​are consistent, it proves that the data has maintained integrity and has not been modified or damaged during transmission and storage. If they are inconsistent, it means that the data may have been tampered with or damaged, and the storage of the data needs to be re-verified or rejected. This step is an important means to ensure that data remains consistent between different processing links.

[0072] After the legitimacy and integrity of the data have been verified, compliance information verification is required. The purpose of compliance verification is to ensure that multimodal process data complies with relevant laws, regulations and industry standards, especially in areas involving sensitive data or subject to strict data protection regulations (such as GDPR). This step usually includes a review of the privacy, confidentiality and use permissions of the data. For example, whether certain data contains sensitive personal information, whether it has been authorized for processing, etc. Compliance verification confirms that the data complies with relevant regulations by analyzing data metadata and its related permissions, ensuring that the storage and use of data are compliant and legal.

[0073] Finally, there is the verification of ownership information, which aims to ensure that the ownership and use rights of the data are clear and traceable. In the verification of ownership, the system will check the identity information and ownership records of the data resource owner. Usually, through the recorded owner ID and ownership timestamp, the ownership of the data can be traced and it can be ensured that the data has not been illegally occupied or used infringingly. This process relies on the on-chain or off-chain ownership mechanism to ensure that the data ownership is clear and avoid disputes caused by unclear data ownership.

[0074] Through the four-fold verification of authenticity, integrity, compliance and ownership information, the credibility of multimodal process data is ensured before off-chain storage.

[0075] Next, based on the pre-defined minimum set of metadata, the basic information of the multimodal process data (such as data resource ID, data resource location information, data description, etc.) is combined with various verification information (such as hash value, signature information, ownership information, etc.). The field design of the minimum set of metadata covers the core attributes of the data and ensures that all key information can be accurately accessed during subsequent storage, transmission and verification. For example, the data resource ID (DID) uniquely identifies each piece of multimodal process data to ensure that it can maintain consistency and traceability throughout the data life cycle; the data resource location information points to the actual data location stored off-chain, so that the data can be quickly located; the data hash value recorded in the integrity verification information ensures the integrity comparison of the data during subsequent retrieval, and the ownership information records the owner of the data and the time of ownership confirmation to ensure clear ownership of the data.

[0076] In this process, the role of the minimum set of metadata is not only to simplify the description and storage of data, but also to ensure the unified management of multimodal process data by providing a set of standardized formats. During the integration process, each field of metadata is encapsulated in JSON format to ensure its efficient storage and transmission in different storage nodes or distributed databases. Integrating multimodal data and verification information into structured metadata helps the system process data in a standardized manner in subsequent on-chain evidence storage, data storage, and query links, avoiding inconsistencies in the processing of different data types or verification results.

[0077] Step 4: Record the digital signature of the verification process data on the blockchain to obtain the on-chain shared signature of the verification process data.

[0078] The embodiment of step 4 is to record the digital signature of the verification process data on the blockchain, in order to ensure that the data is tamper-proof and traceable during storage and use. By putting the key information of the verification process data on the chain, the decentralized and tamper-proof characteristics of the blockchain are utilized to enhance the security and credibility of the multimodal process data. This step is crucial in the entire solution because it provides a basis for on-chain and off-chain collaborative storage, enabling the system to efficiently store data off-chain while ensuring the credibility of the data through on-chain evidence.

[0079] In a possible implementation, the digital signature evidence includes a data summary, a timestamp, and signature information. Step 4 specifically includes: A first hash value of the authentication process data is calculated.

[0080] The first hash value is used as the data digest, and a timestamp is generated for the data digest.

[0081] The data digest and the timestamp are signed based on a private key associated with the multimodal process data to generate signature information.

[0082] The data summary, timestamp and signature information are recorded on the blockchain to obtain the on-chain shared signature evidence of the verification process data.

[0083] Specifically, first obtain the integrated verification process data, which already contains the multimodal process data and its corresponding minimum set of metadata, including verification information, data resource ID, location information, integrity verification information, ownership information, etc. In order to ensure efficient storage and security of data, the system will not directly store or transmit this data, but use a hash algorithm to calculate its summary. Hash algorithms usually use algorithms with high encryption strength such as SHA-256, which can map data input of any size to a fixed-length string. This string is the first hash value of the verification process data.

[0084] A key feature of hashing algorithms is that even if the input data changes slightly, the output hash value will be completely different. This means that any attempt to tamper with any part of the multimodal data (whether it is content, metadata, or verification information) will result in a change in the hash value. Therefore, the first hash value can effectively reflect the integrity of the verification process data. Once the hash value is calculated, this first hash value will serve as a unique identifier for the verification process data and will be used for on-chain storage and comparison in subsequent steps.

[0085] Next, it is necessary to confirm the current time, that is, the exact time point when the verification process data is generated or verified. This time will be recorded as a timestamp and associated with the first hash value. Timestamps generally use a standard time format, such as ISO 8601 format, accurate to seconds or milliseconds, to ensure its compatibility across platforms and systems. Through this standardized time format, the system can ensure that the timestamp of the verification process data can be accurately understood and parsed no matter when and where. This timestamp not only marks the generation time of the data, but also provides a reliable basis for the time tracing and auditing of subsequent data.

[0086] The process of generating timestamps is usually automated. After the system completes the calculation of the first hash value, it immediately calls the system clock to obtain the current precise time and formats it into a standard timestamp. The timestamp is combined with the first hash value to form a unique time-data association pair. This timestamp adds a time dimension to the verification process data, so that each piece of data can be clearly marked at any time in its life cycle when it was generated, stored or used.

[0087] Next, the system needs to digitally sign these two key pieces of information using the private key associated with the data owner. The private key is a unique encryption key held by the data owner, and only the legitimate owner can use the private key for signing operations, thereby ensuring the origin and integrity of the data.

[0088] The specific process of signing uses asymmetric encryption technology, usually RSA or elliptic curve algorithm. The system uses the first hash value and timestamp as the data to be signed and encrypts them with the private key. The encrypted data is the signature information, which is bound to the original data. Any modification of the hash value or timestamp will cause the signature verification to fail, thereby effectively preventing tampering.

[0089] Finally, the system packages the data summary, timestamp, and signature information and submits them to the blockchain through a smart contract. A smart contract is an automatically executed program on the blockchain that is responsible for automatically recording data to the blockchain when certain conditions are met. In this process, the smart contract writes the data summary (first hash value), timestamp, and signature information into the block and generates a new block containing this information. The characteristic of the blockchain is that each block forms an unalterable chain through the previous and next links, and any tampering with the recorded information will destroy the integrity of the chain. Therefore, recording these key information on the blockchain can effectively prevent the data from being tampered with after the evidence is stored.

[0090] A key point of this process is that the blockchain does not store the original content of the verification process data, but only records its hash value and signature information. This design effectively reduces the burden of on-chain storage, because the length of the hash value and signature is fixed and small, and compared with the large amount of original data of multimodal process data, storing these summary information greatly saves space. At the same time, since the blockchain is a public, distributed ledger, all participants can access these records at any time and verify their validity. Through the public key, anyone can verify the authenticity of the signature, thereby verifying the source and integrity of the data.

[0091] Step 5: Based on the on-chain shared signature evidence, establish a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data.

[0092] The embodiment of step 5 is based on the on-chain shared signature evidence, and establishes a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data, with the purpose of ensuring the effective association between the on-chain and off-chain data, so that the data can be efficiently and securely managed and verified through on-chain and off-chain collaboration throughout the life cycle. By establishing this mapping relationship, the system can quickly trace back to the off-chain data resources from the on-chain identifier when necessary, realize the collaborative operation of verification, storage and query, and ensure the credibility and integrity of the data.

[0093] In a possible implementation, step 5 specifically includes the following steps: Extract the first hash value of the shared signature certificate on the blockchain and use the first hash value as the on-chain identifier.

[0094] Determine the Uniform Resource Locator (URL) for off-chain storage of a minimal set of multimodal process data and metadata.

[0095] Based on a preset data structure, a mapping relationship between an on-chain identifier and a uniform resource locator is established, wherein the on-chain identifier is used as a key of the mapping relationship and the uniform resource locator is used as a value of the mapping relationship.

[0096] Specifically, first, the system needs to extract the shared signature evidence information of the verification process data from the blockchain, especially the first hash value. This hash value, as the unique identifier of the verification process data, has been recorded in the blockchain in the previous step. After extracting this hash value, the system uses it as the on-chain identifier. This on-chain identifier not only represents the summary information of the data, but also points to the signature evidence and timestamp of the data, ensuring the data cannot be tampered with and can be traced back to time.

[0097] Next, the system needs to determine the Uniform Resource Locator (URL) used when storing verification process data off-chain. This URL points to the data resources stored off-chain, including multimodal process data and its corresponding verification information. Since off-chain storage can be distributed in different servers or database nodes, each data resource needs to be located by a unique URL. Here, the role of the Uniform Resource Locator is to bind the data resources stored off-chain with the identifier recorded on the chain, ensuring that the corresponding data resources off-chain can be quickly found when accessing the on-chain signature evidence.

[0098] After obtaining the on-chain identifier (i.e., the first hash value) and the off-chain resource locator (URL), the system maps the two through a preset data structure. Usually, the system uses efficient data structures such as hash tables or red-black trees to establish this mapping relationship. The hash table can perform fast searches within O(1) time complexity, while the red-black tree structure can maintain high performance stability under complex query operations. Through this mapping structure, the system uses the on-chain identifier as the key (Key) and the off-chain uniform resource locator as the value (Value), thereby achieving a close connection between the on-chain and off-chain.

[0099] The establishment of this mapping relationship is very important because it ensures that the system can quickly locate data resources off-chain from the chain when needed. In a distributed environment, the storage of multimodal process data is distributed across multiple nodes or databases, while the chain only stores the hash summary and signature information of the data. Through the mapping relationship, the system can easily find the data stored off-chain through the on-chain identifier and verify its authenticity and integrity. In particular, when users need to verify or access data, the system can quickly retrieve the corresponding off-chain data based on the hash value provided by the on-chain signature certificate, and compare it to ensure that the data has not been tampered with or damaged during storage and transmission.

[0100] The effect of establishing a mapping relationship is reflected in many aspects. First, it ensures the efficiency and operability of on-chain and off-chain collaborative storage. Users can quickly locate off-chain data resources through on-chain information, avoiding tedious manual query and positioning operations. Secondly, through this mapping relationship, the system can maintain the consistency of data management in a distributed environment. No matter where the data is stored, it can be quickly verified through the on-chain identifier. Finally, the mapping relationship effectively reduces the storage and transmission burden of the system. Only key information such as the hash value, signature, and timestamp of the data is stored on the chain, while the complete data content is stored off-chain. Through this division of labor, the system not only ensures the immutability of on-chain data, but also effectively protects the privacy and security of off-chain data.

[0101] Step 6: Store the verification process data and mapping relationship in the distributed process database.

[0102] The embodiment of step 6 is to store the verification process data and mapping relationship in a distributed process database, the purpose of which is to achieve efficient and secure management of the verification process data through the storage advantages of the distributed database, and to ensure the scalability, reliability and rapid response capability of the data query. The distributed database can handle large-scale multimodal process data, and optimize resource allocation and data storage efficiency through a load balancing mechanism. In this way, the system can provide efficient storage and query capabilities while ensuring data integrity and consistency.

[0103] In one possible implementation, refer to Figure 4 , Figure 4 Schematic diagram of the distributed process database provided by the present invention. The distributed process database includes a process data storage interface, a process data load balancing mechanism and a distributed database. Step 6 specifically includes the following steps: Based on the process data load balancing mechanism, the verification process data and mapping relationships are distributed to each database node in the distributed database for storage through the process data storage interface. Among them, the process data load balancing mechanism distributes the verification process data and mapping relationships to the appropriate database nodes according to the data characteristics and the performance of each database node.

[0104] Specifically, first, the system obtains the verified process data and the mapping relationship between the on-chain identifier and the off-chain resource locator constructed in the previous step. At this point, the verified process data contains the minimum set of multimodal process data and its related metadata, and the mapping relationship corresponds to the uniform resource locator (URL) of the off-chain resource through the on-chain hash value. In order to ensure efficient storage of data in a distributed environment, the system will hand over this data to the distributed database for storage and processing.

[0105] The structural design of the distributed process database can adopt a distributed NoSQL database such as MongoDB, because it can support the efficient storage of unstructured data and multimodal data. First, the system passes the verification process data and mapping relationships to the database through the process data storage interface. The storage interface is responsible for formatting this data into a form suitable for distributed storage and calling the storage function of the database in the background. Through this interface mechanism, the system can ensure that the data format is unified and the interface is universal during the storage process, and can provide support for future queries and statistical analysis.

[0106] Next, the process data load balancing mechanism is activated to ensure that the verification process data can be allocated to the appropriate database node. The role of the load balancing mechanism is to reasonably allocate data resources based on the characteristics of different data (such as data volume, access frequency, data type, etc.) and the performance and load of each distributed database node. Specifically, the system will dynamically adjust the storage location of the data based on factors such as the node's processing power, storage capacity, and network bandwidth to ensure the overall storage and query efficiency of the system. Through this mechanism, the system can ensure the optimal allocation of resources, reduce storage bottlenecks, and improve query response speed when dealing with large-scale, multimodal data.

[0107] Once the verification process data and mapping relationships are successfully stored in the distributed database, the system will have the ability to quickly retrieve and query. When users access the verification process data, they can quickly locate the data resources off the chain through the mapping relationship through the hash value in the shared signature certificate on the chain. In the distributed database, the system establishes an index for the verification process data to speed up query and positioning operations. Especially for queries on multimodal data, the system can quickly find the corresponding process data through the uniform resource locator (URL) and hash value index to achieve efficient retrieval. When processing large-scale data, distributed databases such as MongoDB can store data on different nodes through a sharding mechanism to ensure that the system can still provide stable performance and response when facing a large number of concurrent requests.

[0108] In a possible implementation manner, after step 6, the following steps are further included: When a user accesses multimodal process data, the first hash value of the shared signature evidence on the chain and the second hash value of the multimodal process data stored off-chain are obtained; calculating a third hash value of the multimodal process data; Compare the third Hash value with the first Hash value and the second Hash value respectively to determine whether the third Hash value is consistent with the first Hash value and / or the second Hash value; When the third hash value is consistent with the first hash value, and / or the second hash value, allowing the user to access the multimodal process data; When the third hash value is inconsistent with both the first hash value and the second hash value, the user is denied access to the multimodal process data.

[0109] Specifically, when a user initiates a request to access multimodal process data, the system first extracts the first hash value of the shared signature on the blockchain. This first hash value is the unique identifier of the verification process data, which has been recorded in the blockchain through the on-chain signature in the previous step. The purpose of extracting this hash value is to verify whether the data accessed by the user is consistent with the data recorded on the chain, ensuring that the data has not been tampered with or modified.

[0110] Next, the system extracts the multimodal process data stored off-chain from the distributed process database through the mapping relationship and calculates its second hash value. The verification process data off-chain contains the data requested by the user and the minimum set of its metadata. The system calculates the data stored off-chain through the same hash algorithm (such as SHA-256) to generate the hash value of the data, i.e. the second hash value. This hash value represents the integrity and status of the off-chain data and is the basis for the comparison of on-chain and off-chain data.

[0111] To further ensure the authenticity of the data, the system also needs to recalculate the third hash value of the multimodal process data off-chain. This step is to perform real-time calculations on the off-chain data to ensure that the current state of the data has not changed when the system responds to user requests. The third hash value is obtained by real-time calculation of the off-chain data, and it is a key step for the final comparison with the hash values ​​stored twice before.

[0112] The system then compares the third hash value with the first and second hash values. This step is the core of the verification process. By comparing the hash value of the current off-chain data with the hash value stored on the chain (the first hash value) and the hash value originally calculated off-chain (the second hash value), the system can determine whether the data has maintained consistency during storage and transmission. If the three hash values ​​are consistent, it means that the data has not been tampered with or modified on or off the chain.

[0113] If the comparison result shows that the third hash value is consistent with the first hash value and / or the second hash value, the system will allow the user to access the multimodal process data. At this time, the system will respond to the user's access request and provide the corresponding multimodal process data. Users can safely access verified data to ensure the authenticity, integrity and immutability of the data. This process ensures that the data accessed by the user is credible and can be traced back to the historical records of on-chain evidence storage and off-chain verification.

[0114] If the comparison result shows that the third hash value is inconsistent with the first hash value and the second hash value, the system will deny the user access to the multimodal process data. In this case, the system believes that the data may have been tampered or erroneous during storage or transmission, so access is denied to ensure that the user does not obtain incomplete or tampered data. The access denial mechanism is to protect the security of the system and data and prevent forged data from entering the system, thereby damaging the credibility of the system.

[0115] Reference Figure 5 , Figure 5 It is a structural diagram of a multimodal process data trusted storage system combined with blockchain provided by the present invention, the system comprising: an acquisition module and a processing module; An acquisition module, used to acquire multimodal process data; The processing module is used to construct a minimal set of metadata covering the entire life cycle of multimodal process data; The processing module is also used to store the multimodal process data and the minimum set of metadata off-chain, and to verify the multimodal process data to obtain the verified process data; The processing module is also used to record the digital signature of the verification process data on the blockchain to obtain the on-chain shared signature of the verification process data; The processing module is also used to establish a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data based on the on-chain shared signature evidence; The processing module is also used to store the verification process data and the mapping relationship in the distributed process database.

[0116] In a possible implementation manner, the processing module is further configured to: Conduct authenticity check, integrity check, compliance information check, and ownership information check on multimodal process data to obtain verification information; According to the minimum set of metadata, the verification information is integrated with the multimodal process data in the form of metadata to obtain the verification process data.

[0117] In a possible implementation manner, the processing module is further configured to: Calculating a first hash value of the verification process data; Using the first hash value as a data digest and generating a timestamp for the data digest; Signing the data digest and the timestamp based on a private key associated with the multimodal process data to generate signature information; The data summary, timestamp and signature information are recorded on the blockchain to obtain the on-chain shared signature evidence of the verification process data.

[0118] In a possible implementation manner, the processing module is further configured to: Extract the first hash value of the shared signature certificate on the blockchain and use the first hash value as the on-chain identifier; Determine the Uniform Resource Locator (URL) for off-chain storage of a minimum set of multimodal process data and metadata; Based on a preset data structure, a mapping relationship between an on-chain identifier and a uniform resource locator is established, wherein the on-chain identifier is used as a key of the mapping relationship and the uniform resource locator is used as a value of the mapping relationship.

[0119] In a possible implementation manner, the processing module is further configured to: Based on the process data load balancing mechanism, the verification process data and mapping relationships are distributed to each database node in the distributed database for storage through the process data storage interface. Among them, the process data load balancing mechanism distributes the verification process data and mapping relationships to the appropriate database nodes according to the data characteristics and the performance of each database node.

[0120] In a possible implementation, the acquisition module is further used to acquire the first hash value of the shared signature evidence on the chain and the second hash value of the multimodal process data stored off-chain when the user accesses the multimodal process data; The processing module is also used to: calculating a third hash value of the multimodal process data; Compare the third Hash value with the first Hash value and the second Hash value respectively to determine whether the third Hash value is consistent with the first Hash value and / or the second Hash value; When the third hash value is consistent with the first hash value, and / or the second hash value, allowing the user to access the multimodal process data; When the third hash value is inconsistent with both the first hash value and the second hash value, the user is denied access to the multimodal process data.

[0121] It should be noted that the multimodal process data trusted storage system combined with blockchain provided by the present invention can execute the multimodal process data trusted storage method combined with blockchain of any of the above-mentioned embodiments during specific operation, which will not be elaborated in this embodiment.

[0122] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610 (processor), a communication interface 620 (CommunicationsInterface), a memory 630 (memory) and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute a trusted storage method for multimodal process data combined with a blockchain, the method comprising: obtaining multimodal process data; constructing a minimum set of metadata covering the entire life cycle of the multimodal process data; storing the multimodal process data and the minimum set of metadata off-chain, and verifying the multimodal process data to obtain verification process data; recording the digital signature of the verification process data on the blockchain to obtain the on-chain shared signature of the verification process data; based on the on-chain shared signature, establishing a mapping relationship between the hash value of the on-chain shared signature and the off-chain verification process data; storing the verification process data and the mapping relationship in a distributed process database.

[0123] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory 630 (ROM, Read-Only Memory), random access memory 630 (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0124] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the trusted storage method for multimodal process data combined with blockchain provided by the above-mentioned embodiments, and the method includes: obtaining multimodal process data; constructing a minimum set of metadata covering the entire life cycle of multimodal process data; storing the multimodal process data and the minimum set of metadata off-chain, and verifying the multimodal process data to obtain verification process data; recording the digital signature evidence of the verification process data on the blockchain to obtain an on-chain shared signature evidence of the verification process data; based on the on-chain shared signature evidence, establishing a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data; storing the verification process data and the mapping relationship in a distributed process database.

[0125] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by the processor 610, is implemented to execute the trusted storage method for multimodal process data combined with blockchain provided in the above-mentioned embodiments, the method comprising: obtaining multimodal process data; constructing a minimum set of metadata covering the entire life cycle of multimodal process data; storing the multimodal process data and the minimum set of metadata off-chain, and verifying the multimodal process data to obtain verification process data; recording the digital signature evidence of the verification process data on the blockchain to obtain an on-chain shared signature evidence of the verification process data; based on the on-chain shared signature evidence, establishing a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data; and storing the verification process data and the mapping relationship in a distributed process database.

[0126] The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0127] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal process data trusted storage method combined with blockchain, characterized in that: include: Acquire multimodal process data; Constructing a minimum set of metadata covering the entire life cycle of the multimodal process data; The multimodal process data and the minimum set of metadata are stored off-chain, and the multimodal process data is verified to obtain verification process data; The digital signature of the verification process data is recorded on the blockchain to obtain the on-chain shared signature of the verification process data; Based on the on-chain shared signature evidence, a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data is established; The verification process data and the mapping relationship are stored in a distributed process database.

2. The multimodal process data trusted storage method combined with blockchain according to claim 1 is characterized in that: The verifying the multimodal process data to obtain the verification process data specifically includes: Performing authenticity check, integrity check, compliance information check and ownership confirmation information check on the multimodal process data to obtain verification information; According to the minimum set of metadata, the verification information and the multimodal process data are integrated in the form of metadata to obtain the verification process data.

3. The multimodal process data trusted storage method combined with blockchain according to claim 1 is characterized in that: The digital signature evidence includes a data summary, a timestamp, and signature information; the digital signature evidence of the verification process data is recorded on the blockchain to obtain the on-chain shared signature evidence of the verification process data, specifically including: Calculating a first hash value of the verification process data; Using the first hash value as the data digest and generating a timestamp for the data digest; Signing the data digest and the timestamp based on a private key associated with the multimodal process data to generate the signature information; The data summary, the timestamp and the signature information are recorded on the blockchain to obtain the on-chain shared signature evidence of the verification process data.

4. The multimodal process data trusted storage method combined with blockchain according to claim 1 is characterized in that: The step of establishing a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data based on the on-chain shared signature evidence specifically includes: Extracting a first hash value of the shared signature certificate on the chain from the blockchain, and using the first hash value as the on-chain identifier; Determining a uniform resource locator corresponding to off-chain storage of the multimodal process data and the minimum set of metadata; Based on a preset data structure, a mapping relationship between the on-chain identifier and the uniform resource locator is established, wherein the on-chain identifier is used as a key of the mapping relationship, and the uniform resource locator is used as a value of the mapping relationship.

5. The multimodal process data trusted storage method combined with blockchain according to claim 1 is characterized in that: The distributed process database includes a process data storage interface, a process data load balancing mechanism and a distributed database; The storing the verification process data and the mapping relationship in a distributed process database specifically includes: Based on the process data load balancing mechanism, the verification process data and the mapping relationship are distributed to each database node in the distributed database for storage through the process data storage interface, wherein the process data load balancing mechanism distributes the verification process data and the mapping relationship to the appropriate database node according to the data characteristics and the performance of each database node.

6. The multimodal process data trusted storage method combined with blockchain according to claim 1 is characterized in that: The method further comprises: When a user accesses the multimodal process data, a first hash value of the shared signature evidence on the chain and a second hash value of the multimodal process data stored off-chain are obtained; Calculating a third hash value of the multimodal process data; Compare the third Hash value with the first Hash value and the second Hash value respectively to determine whether the third Hash value is consistent with the first Hash value and / or the second Hash value; When the third hash value is consistent with the first hash value, and / or the second hash value, allowing the user to access the multimodal process data; When the third hash value is inconsistent with both the first hash value and the second hash value, the user is denied access to the multimodal process data.

7. A multimodal process data trusted storage system combined with blockchain, characterized in that: include: Acquisition module and processing module; The acquisition module is used to acquire multimodal process data; The processing module is used to construct a minimum set of metadata covering the entire life cycle of the multimodal process data; The processing module is further used to store the multimodal process data and the minimum set of metadata off-chain, and verify the multimodal process data to obtain verification process data; The processing module is further used to record the digital signature of the verification process data on the blockchain to obtain the on-chain shared signature of the verification process data; The processing module is further used to establish a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data based on the on-chain shared signature evidence; The processing module is further used to store the verification process data and the mapping relationship in a distributed process database.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the multimodal process data trusted storage method combined with blockchain as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multimodal process data trusted storage method combined with blockchain as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multimodal process data trusted storage method combined with blockchain as described in any one of claims 1 to 6 is implemented.

Images