A multi-modal process data trusted storage method and system combined with a blockchain

Abstract

The application provides a kind of multi-modal process data credible storage method and system combined with blockchain, belong to data storage technical field, including: obtaining multi-modal process data;Construct the metadata minimum set covering the whole life cycle of multi-modal process data;Multi-modal process data and metadata minimum set are stored off-chain, and multi-modal process data is verified, and verification process data is obtained;The digital signature of verification process data is recorded in evidence on the blockchain, and the on-chain shared signature evidence of verification process data is obtained;Based on the on-chain shared signature evidence, the mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data is established;Verification process data and mapping relationship are stored in distributed process database.The application combines blockchain and distributed storage, and constructs a multi-modal process data credible storage system coordinated on and off the chain, so as to realize the credible storage of multi-modal process data in the whole life cycle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, in particular to a multi-modal process data credible storage method and system combined with a blockchain. BACKGROUND

[0002] Process data refers to detailed records of data from generation, use, transmission, storage, verification and destruction in a certain specific process or system, which covers all operations and changes of data in the entire life cycle. In the field of intelligent computing services, due to the large amount of multi-modal process data (including text, image, video, structured data, etc.), the variety of types, and the need to ensure the integrity, authenticity and traceability of data in the entire life cycle, the management and storage of multi-modal process data face major challenges.

[0003] Under the background of the prior art, in order to ensure the security and credibility of multi-modal process data, the following methods are usually used for data management: first, data is stored and managed through off-chain storage, second, data stored off-chain is secured through access control, data encryption and other means, and finally, to ensure the credibility of data, some systems introduce blockchain technology, hash the data and upload it to the chain for storage, thereby providing data tamper-proof and traceable protection.

[0004] The existing methods mainly have the following problems: first, although the blockchain technology used for data credible storage has the advantage of being tamper-proof, due to the large amount of multi-modal data, directly storing all of them on the chain is not only impractical, but also will cause excessive burden on chain storage and calculation. Therefore, the existing technology usually stores data off-chain, and only uploads the hash value on the chain. However, this method often lacks efficient association of on-chain and off-chain data in data management, positioning and access, resulting in difficulty in efficient positioning and retrieval of data in the access process. Second, although the data stored off-chain can provide certain security through access control and other means, the data may still be subject to tampering risks in multiple stages of the entire life cycle.

[0005] Therefore, how to realize the credible storage of multi-modal process data in the entire life cycle has become a technical problem to be solved. SUMMARY

[0006] The present application provides a multi-modal process data credible storage method, system, electronic device and storage medium combined with a blockchain, to solve the defects in the prior art and realize the credible storage of multi-modal process data in the entire life cycle.

[0007] The present application provides a multi-modal process data credible storage method combined with a blockchain, comprising the following steps: acquiring multi-modal process data;

[0008] constructing a metadata minimum set covering a full life cycle of the multi-modal process data;

[0009] storing the multi-modal process data and the metadata minimum set off-chain, and verifying the multi-modal process data to obtain verified process data;

[0010] recording a digital signature of the verified process data on a blockchain to obtain an on-chain shared signature record of the verified process data;

[0011] based on the on-chain shared signature record, establishing a mapping relationship between a hash value of the on-chain shared signature record and off-chain verified process data;

[0012] storing the verified process data and the mapping relationship in a distributed process database.

[0013] According to the multi-modal process data credible storage method combined with a blockchain provided by the application, the multi-modal process data is verified to obtain verified process data, and the verification specifically includes:

[0014] performing authenticity verification, integrity verification, compliance information verification and right information verification on the multi-modal process data to obtain verification information;

[0015] According to the metadata minimum set, the verification information and the multi-modal process data are integrated in the form of metadata to obtain the verified process data.

[0016] According to the multi-modal process data credible storage method combined with a blockchain provided by the application, the digital signature record includes data digest, timestamp and signature information; the digital signature record of the verified process data is recorded on the blockchain to obtain the on-chain shared signature record of the verified process data, and the recording specifically includes:

[0017] calculating a first hash value of the verified process data;

[0018] the first hash value is taken as the data digest, and a timestamp is generated for the data digest;

[0019] based on a private key associated with the multi-modal process data, the data digest and the timestamp are signed to generate the signature information;

[0020] the data digest, the timestamp and the signature information are recorded on the blockchain to obtain the on-chain shared signature record of the verified process data.

[0021] According to the multi-modal process data credible storage method combined with a blockchain provided by the application, the mapping relationship between the hash value of the on-chain shared signature storage and the off-chain verification process data is established based on the on-chain shared signature storage, and the mapping relationship specifically includes:

[0022] The first hash value of the on-chain shared signature storage is extracted from the blockchain, and the first hash value is taken as an on-chain identifier;

[0023] The uniform resource locator corresponding to the storage of the multi-modal process data and the metadata minimum set is determined;

[0024] Based on a preset data structure, a mapping relationship between the on-chain identifier and the uniform resource locator is established, wherein the on-chain identifier is taken as the key of the mapping relationship, and the uniform resource locator is taken as the value of the mapping relationship.

[0025] According to the multi-modal process data credible storage method combined with a blockchain provided by the application, the distributed process database includes a process data storage interface, a process data load balancing mechanism and a distributed database; the verification process data and the mapping relationship are stored in the distributed process database, and specifically include:

[0026] Based on the process data load balancing mechanism, the verification process data and the mapping relationship are distributed to each database node in the distributed database for storage through the process data storage interface, wherein the process data load balancing mechanism distributes the verification process data and the mapping relationship to appropriate database nodes according to data characteristics and the performance of each database node.

[0027] According to the multi-modal process data credible storage method combined with a blockchain provided by the application, the method further includes:

[0028] When a user accesses the multi-modal process data, the first hash value of the on-chain shared signature storage and the second hash value of the off-chain stored multi-modal process data are obtained;

[0029] The third hash value of the multi-modal process data is calculated;

[0030] The third hash value is compared with the first hash value and the second hash value respectively, and it is judged whether the third hash value is consistent with the first hash value and / or the second hash value;

[0031] When the third hash value is consistent with the first hash value and / or the second hash value, the user is allowed to access the multi-modal process data;

[0032] rejecting user access to the multi-modal process data when the third hash value is inconsistent with the first hash value and the second hash value.

[0033] The application further provides a multi-modal process data credible storage system combined with a blockchain, comprising an acquisition module and a processing module.

[0034] The acquisition module is configured to acquire multi-modal process data.

[0035] The processing module is configured to construct a metadata minimum set covering the whole life cycle of the multi-modal process data.

[0036] The processing module is further configured to store the multi-modal process data and the metadata minimum set off-chain, and verify the multi-modal process data to obtain verified process data.

[0037] The processing module is further configured to record a digital signature of the verified process data in a blockchain to obtain an on-chain shared signature record of the verified process data.

[0038] The processing module is further configured to establish a mapping relationship between a hash value of the on-chain shared signature record and off-chain verified process data based on the on-chain shared signature record.

[0039] The processing module is further configured to store the verified process data and the mapping relationship in a distributed process database.

[0040] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-modal process data credible storage method combined with a blockchain when executing the program.

[0041] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the multi-modal process data credible storage method combined with a blockchain.

[0042] The application further provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the multi-modal process data credible storage method combined with a blockchain.

[0043] To sum up, one or more technical solutions provided in the embodiments of the application have at least the following technical effects or advantages:

[0044] By constructing a metadata minimum set covering the whole life cycle of multi-modal process data, a standardized data description framework is established to record the key information of each data in its life cycle, so that the state, operation record, and right information of each multi-modal process data can be uniformly managed and tracked throughout the life cycle, improving the accuracy and operability of data management. By storing multi-modal process data and metadata minimum set off-chain and verifying multi-modal process data, verified process data is obtained. First, the data is verified for authenticity, integrity, and right information to ensure that the data has been fully verified before storage, thereby reducing security risks in data storage and ensuring the security and credibility of off-chain data storage. By recording the digital signature of the verified process data on the blockchain, the on-chain shared signature record of the verified process data is obtained. By using the tamper-proof feature of the blockchain, the signature, hash value, and timestamp of the data are stored on the chain, thereby providing transparent storage proof for each verified process data, ensuring the tamper-proof and traceability of the data, and enhancing data security. By establishing the mapping relationship between the hash value of the on-chain shared signature record and the off-chain verified process data based on the on-chain shared signature record, the system establishes an effective connection between the on-chain and off-chain, enabling the on-chain record to quickly locate the actual data resources stored off-chain, thereby realizing the cooperation between the on-chain and off-chain, ensuring efficient management and fast query of data throughout the life cycle, and improving the efficiency and accuracy of data access. By storing the verified process data and the mapping relationship in a distributed process database, relying on the scalability and efficient query capability of the distributed database, the verified process data and its mapping relationship are stored on multiple nodes, thereby realizing the trusted storage of multi-modal process data throughout the life cycle. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0046] Figure 1 is a schematic diagram of the multi-modal process data trusted storage method combined with the blockchain provided by the present application.

[0047] Figure 2 is one of the flowcharts of the multi-modal process data trusted storage method combined with the blockchain provided by the present application.

[0048] Figure 3 is a schematic diagram of the metadata minimum set field and identifier provided by the present application.

[0049] Figure 4 is a schematic diagram of a distributed process database provided by the present application.

[0050] Figure 5 is a structural schematic diagram of a multi-modal process data trusted storage system combined with a blockchain provided by the present application.

[0051] Figure 6 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0052] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0053] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement “comprises a” does not exclude the presence of another identical element in the process, method, article or device comprising the element. The terms “upper”, “lower” and the like indicate the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and do not indicate or imply that the systems or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0054] The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by “first”, “second” and the like are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, “and / or” means at least one of the connected objects, and the character “ / ” generally represents a “or” relationship between the front and rear associated objects.

[0055] The following will be described with reference to the drawings Figures 1-6The application provides a multi-modal process data trusted storage method combined with a blockchain, a system, an electronic device, and a storage medium. The overall scheme of the application is shown in Figure 1 Figure 1 The application provides a multi-modal process data trusted storage method combined with a blockchain. The server side provides a remote calling interface based on HTTP or other services, and a distributed process database is deployed on the server side.

[0056] Referring to Figure 2 Figure 2 The application provides a multi-modal process data trusted storage method combined with a blockchain. The server side provides a remote calling interface based on HTTP or other services, and a distributed process database is deployed on the server side.

[0057] Step 1: Obtain multi-modal process data.

[0058] Specifically, multi-modal process data refers to data from different data sources and different modalities (such as text, images, audio, video, sensor data, etc.), which is continuously generated in intelligent computing services or complex processes and involves operation records at different stages.

[0059] Specifically, the acquisition of multi-modal process data is not just simple data collection, but a dynamic process covering the entire life cycle of data. In intelligent computing services, data often accompanies different states and changes, such as training data, inference results, and data changes during model generation, which are all part of multi-modal process data. Therefore, when acquiring these data, it is necessary to ensure the continuity and integrity of data from generation to final storage.

[0060] Step 2: Build a minimal set of metadata covering the entire life cycle of multi-modal process data.

[0061] An embodiment of step 2 is to build a minimal set of metadata covering the entire life cycle of multi-modal process data, which aims to standardize the description of each multi-modal process data and its various links in the entire life cycle, thereby ensuring consistency and integrity of data throughout the generation, use, storage, and verification process, and supporting collaborative storage and verification on and off the chain.

[0062] In intelligent computing services, multi-modal process data includes text, images, video, model prediction results, and other forms, and these data will be processed multiple times in multiple stages such as generation, operation, transmission, storage, and verification. In order to ensure the security and credibility of data, a structured metadata model needs to be designed to record the core information of these data. The key purpose of building a minimal set of metadata is to simplify the data description process, ensure the uniqueness, integrity, and explicitness of data rights, and reduce unnecessary data redundancy and complexity. ​​

[0063] For the sake of simplifying expression and processing, the metadata minimum set is usually encapsulated in JSON format, through which the various attributes of the data resource can be uniformly described and processed. The field design of the metadata minimum set not only refines the data description, but also covers the key information of each data in the generation, transmission and use in the intelligent computing process, effectively reducing the storage and transmission burden on and off the chain.

[0064] Referring to Figure 3 , Figure 3 is the field and identifier schematic diagram of the metadata minimum set provided by the application. Figure 3 The fields in the application are described as follows:

[0065] 1. Data resource ID (DID):

[0066] Meaning: used for uniquely identifying each data resource.

[0067] Format: string (String).

[0068] Content description: a unique identifier, numbered in linear order according to the time sequence.

[0069] 2. Data resource location information (DataLocation):

[0070] Meaning: describes the storage location of the data resource.

[0071] Format: string (String).

[0072] Content description: uniform resource description URL.

[0073] 3. Data resource description (DataDescription):

[0074] Meaning: provides a brief description of the data resource, etc.

[0075] Format: string (String).

[0076] Content description: data resource type (data, model, computing power), data resource category (relational database storage, big data form data, model prediction result, etc.), data resource form (supporting multi-modal, such as text, image, video, structured data, etc.).

[0077] 4. Data resource usage time (UsageTime):

[0078] Meaning: records the usage time of the data resource.

[0079] Format: date and time (DateTime).

[0080] Content Description: Start and end times of data resource usage in ISO 8601 format, e.g., 2024-08-03T10:15:30Z.

[0081] 5. Data Resource Owner Information (OwnerInfo):

[0082] Meaning: Describes the owner information of the data resource.

[0083] Format: JSON Object (JSONObject).

[0084] Content Description: Relevant information of the data resource owner, such as owner ID, name, contact information, etc., e.g., {"ownerID": "SJTU-user1", "name": "seiee-ljh", "contact": "seiee-ljh@example.com"}.

[0085] 6. Integrity Verification Information (IntegrityVerification):

[0086] Meaning: Provides integrity verification information for the data resource.

[0087] Format: JSON Object (JSONObject).

[0088] Content Description: Describes the integrity verification method and result, e.g., {"method": "SHA-256", "result": "abc123"}.

[0089] 7. Ownership Information (OwnershipInfo):

[0090] Meaning: Records the ownership information of the data resource.

[0091] Format: JSON Object (JSONObject).

[0092] Content Description: Describes the relevant information of ownership, such as owner ID, ownership time, etc., in the format: {"ownerID": "user123", "timestamp": "2024-08-03T10:15:30Z"}.

[0093] 8. Data Resource Participation in Intelligent Computing Information (AIComputationInfo):

[0094] Meaning: Records the relevant information of the data resource participating in intelligent computing.

[0095] Format: JSON Object (JSONObject).

[0096] Content Description: Describes the smart computing task ID, computing time, computing result, etc., for example: {"taskID": "task123", "computeTime": "2024-08-03T11:00:00Z", "result": "success"}.

[0097] 9. Lifecycle information for collection and storage (LifecycleInfo):

[0098] Meaning: Describes the lifecycle information of data resources during collection, storage, and use.

[0099] Format: JSON object (JSONObject).

[0100] Content Description: Includes data creation time, modification time, storage time, and usage time, etc., for example: {"createdTime": "2024-08-01T08:00:00Z", "modifiedTime": "2024-08-02T09:00:00Z", "storageTime": "2024-08-01T08:00:00Z", "usageTime": "2024-08-03T10:15:30Z"}.

[0101] 10. Data resource hash value (DataHash):

[0102] Meaning: Provides the hash value of the data resource, used to verify the integrity of the data and prevent tampering.

[0103] Format: String.

[0104] Content Description: Hash value of the data resource, used for data retrieval and information ownership rights, for example, a hash string generated using the SHA-256 algorithm.

[0105] Step 3: Off-chain storage of multi-modal process data and metadata minimal set, and verification of multi-modal process data, obtaining verified process data.

[0106] An embodiment of step 3 is to store the multi-modal process data and the metadata minimal set off-chain, and to verify the multi-modal process data, with the purpose of ensuring the authenticity, integrity, compliance, and ownership of the multi-modal process data when stored off-chain, to ensure the security and credibility of the data throughout its lifecycle, and to lay the foundation for subsequent on-chain signature storage and on-chain and off-chain collaborative storage.

[0107] In one possible implementation, step 3 specifically includes the following steps:

[0108] The authenticity test, integrity test, compliance information test, and right confirmation information test are performed on the multi-modal process data to obtain verification information.

[0109] According to the metadata minimum set, the verification information is integrated with the multi-modal process data in the form of metadata to obtain verified process data.

[0110] Specifically, first, the authenticity test is performed to ensure that the source and generation process of the multi-modal process data are legal and reliable. The key to the authenticity test is to verify that the data is generated or operated by an authorized party and has not been tampered with or falsified. This is usually achieved through a signature mechanism. Specifically, the private key of the entity generating the data is used to sign the data, generating a digital signature, and the validity of the signature is verified by the public key. If the signature verification is passed, it proves that the identity of the data generator is reliable, the data source is legal and has not been tampered with. This process ensures that the multi-modal process data processed by the system is authentic and reliable.

[0111] Next, the integrity test is performed to ensure that the data has not been tampered with or damaged during generation, transmission, and storage. The integrity test is usually implemented through a hash value (such as the SHA-256 algorithm) calculation. The hash value of the multi-modal process data is calculated and compared with the pre-stored original data hash value. If the hash values are consistent, it proves that the data has maintained its integrity during transmission and storage and has not been modified or damaged. If they are not consistent, it indicates that the data may have been tampered with or damaged, and the data needs to be re-verified or rejected for storage. This step is an important means to ensure that the data remains consistent between different processing stages.

[0112] After the legality and integrity of the data are verified, compliance information testing is also required. The purpose of compliance testing is to ensure that the multi-modal process data complies with relevant laws, regulations, and industry standards, especially in areas involving sensitive data or requiring strict data protection regulations (such as GDPR, etc.). This step usually includes a review of the privacy, confidentiality, and usage permissions of the data. For example, whether certain data contains sensitive personal information, whether it has been authorized for processing, etc. Compliance testing confirms that the data complies with relevant regulations by analyzing the data metadata and its related permissions, ensuring that the storage and use of the data are compliant and legal.

[0113] Finally, the right confirmation information test is performed, which aims to ensure that the ownership and usage rights of the data are clear and traceable. In the right confirmation test, the system checks the identity information and right confirmation records of the data resource owner. Usually, through the recorded owner ID and right confirmation timestamp information, the ownership of the data can be traced back, and it is ensured that the data has not been illegally occupied or used. This process relies on on-chain or off-chain right confirmation mechanisms to ensure that the data ownership is clear and to avoid disputes caused by unclear data ownership.

[0114] Through the four tests of authenticity, integrity, compliance and right information, the credibility of the multi-modal process data before off-chain storage is ensured.

[0115] Next, according to the pre-defined minimum set of metadata, the basic information of multi-modal process data (such as data resource ID, data resource location information, data description, etc.) is combined with various verification information (such as hash value, signature information, right information, etc.). The field design of the minimum set of metadata covers the core attributes of the data and ensures that all key information can be accurately accessed in the subsequent storage, transmission and verification process. For example, the data resource ID (DID) uniquely identifies each piece of multi-modal process data, ensuring its consistency and traceability throughout the data life cycle; the data resource location information points to the actual data location stored off-chain, so that the data can be quickly located; the data hash value recorded in the integrity verification information ensures the integrity comparison of the data in subsequent retrieval, and the right information records the owner of the data and the right time, ensuring the clear ownership of the data.

[0116] In this process, the role of the minimum set of metadata is not only to simplify the description and storage of data, but also to ensure the unified management of multi-modal process data by providing a standardized format. In the integration process, each field of the metadata is encapsulated in JSON format to ensure efficient storage and transmission in different storage nodes or distributed databases. Integrating multi-modal data and verification information into structured metadata helps the system to process data in a standardized way in subsequent on-chain storage, data storage and query, etc. It avoids the inconsistency problem caused by different data types or verification results in processing.

[0117] Step 4: Record the digital signature of the verification process data in the blockchain to obtain the on-chain shared signature record of the verification process data.

[0118] The embodiment of step 4 is to record the digital signature of the verification process data in the blockchain, the purpose is to ensure that the data has the characteristics of non-tamperability and traceability in the process of storage and use. By chaining the key information of the verification process data, using the decentralized and tamper-proof characteristics of the blockchain, the security and credibility of the multi-modal process data are enhanced. This step is crucial in the whole scheme, because it provides the basis for off-chain and on-chain collaborative storage, so that the system can efficiently store data off-chain while ensuring the credibility of the data through on-chain storage.

[0119] In one possible implementation, the digital signature record includes data digest, timestamp and signature information. Step 4 specifically includes:

[0120] Calculate the first hash value of the verification process data.

[0121] The first hash value is taken as a data digest, and a timestamp is generated for the data digest.

[0122] Based on the private key associated with the multi-modal process data, the data digest and the timestamp are signed to generate signature information.

[0123] The data digest, timestamp, and signature information are recorded on the blockchain to obtain the on-chain shared signature evidence of the verification process data.

[0124] Specifically, first, the integrated verification process data is obtained, which already contains multi-modal process data and its corresponding metadata minimal set, including verification information, data resource ID, location information, integrity verification information, and right information. To ensure efficient storage and security of the data, the system does not directly store or transmit these data, but uses a hash algorithm to calculate a digest. The hash algorithm, usually using a high encryption strength algorithm such as SHA-256, can map any size of data input to a fixed length string, which is the first hash value of the verification process data.

[0125] A key feature of the hash algorithm is that even if the input data changes slightly, the output hash value will be completely different. This means that any attempt to tamper with or change any part of the multi-modal data, whether it is the content, metadata, or verification information, will result in a change in the hash value. Therefore, the first hash value can effectively reflect the integrity of the verification process data. Once the hash value is calculated, this first hash value will serve as the unique identifier for the verification process data, used for subsequent steps of on-chain evidence and comparison.

[0126] Next, the current time, i.e., the accurate time point of the generation or verification of the verification process data, needs to be confirmed. This time will be recorded as a timestamp and associated with the first hash value. The timestamp generally uses a standard time format, such as the ISO 8601 format, accurate to seconds or milliseconds, ensuring its compatibility across platforms and systems. Through this standardized time format, the system can ensure that the timestamp of the verification process data can be accurately understood and parsed regardless of when and where it is generated. This timestamp not only marks the generation time of the data, but also provides a reliable basis for subsequent time tracing and auditing of the data.

[0127] The process of generating the timestamp is usually automated. When the system calls upon the system clock immediately after completing the first hash calculation, it captures the current precise time and formats it into a standard timestamp. The timestamp is combined with the first hash to form a unique time-data pair. This timestamp adds a temporal dimension to the verification process data, allowing each piece of data to be explicitly labeled with when it was generated, notarized, or used at any point in its lifecycle.

[0128] Next, the system needs to digitally sign both pieces of key information using the private key associated with the data owner. The private key is a unique encryption key held by the data owner, and only the rightful owner can use the private key for signing operations, ensuring the provenance and integrity of the data.

[0129] The specific process of signing uses asymmetric encryption techniques, typically RSA or Elliptic Curve algorithms. The system takes the first hash and the timestamp as the data to be signed and encrypts it using the private key. The encrypted data is the signature information, which is bound to the original data, and any modification of the hash or timestamp will result in a failed signature verification, effectively preventing tampering.

[0130] Finally, the system packages the data digest, timestamp, and signature information and submits them to the blockchain through a smart contract. The smart contract is an automated execution program on the blockchain that is responsible for automatically recording data to the blockchain when certain conditions are met. In this process, the smart contract writes the data digest (first hash), timestamp, and signature information to the block and generates a new block containing this information. The blockchain is characterized by each block being linked to form an unalterable chain, and any tampering with recorded information will destroy the integrity of the chain. Therefore, recording these key information on the blockchain can effectively prevent data from being tampered after notarization.

[0131] A key point of this process is that the blockchain does not store the original content of the verification process data, but only records its hash and signature information. This design effectively reduces the storage burden on the chain, as the length of the hash and signature is fixed and small, compared to the large amount of original data of the multi-modal process data, storing these digest information greatly saves space. At the same time, since the blockchain is a public, distributed ledger, all participants can access these records and verify their validity at any time. Through the public key, anyone can verify the authenticity of the signature, thereby verifying the provenance and integrity of the data.

[0132] Step 5: Based on the chain-shared signature notarization, establish a mapping relationship between the hash value of the chain-shared signature notarization and the off-chain verification process data.

[0133] The embodiment of step 5 is based on the on-chain shared signature evidence, and a mapping relationship between the hash value of the on-chain shared signature evidence and the off-chain verification process data is established, the purpose is to ensure the effective association between the on-chain and off-chain data, so that the data can be efficiently and safely managed and verified through the cooperation of on-chain and off-chain in the whole life cycle. By establishing such a mapping relationship, the system can quickly trace back to the off-chain data resource from the on-chain identification when needed, realize the cooperative operation of verification, storage and query, and ensure the credibility and integrity of the data.

[0134] In one possible implementation, step 5 specifically includes the following steps:

[0135] The first hash value of the on-chain shared signature evidence is extracted from the blockchain, and the first hash value is taken as the on-chain identifier.

[0136] The uniform resource locator corresponding to the off-chain storage of the minimum set of multi-modal process data and metadata is determined.

[0137] Based on the preset data structure, a mapping relationship between the on-chain identifier and the uniform resource locator is established, wherein the on-chain identifier is taken as the key of the mapping relationship, and the uniform resource locator is taken as the value of the mapping relationship.

[0138] Specifically, first, the system needs to extract the on-chain shared signature evidence information of the verification process data from the blockchain, especially the first hash value. The hash value, as the unique identifier of the verification process data, has been recorded in the blockchain through the previous step. After extracting this hash value, the system takes it as the on-chain identifier. This on-chain identifier not only represents the digest information of the data, but also points to the signature evidence and timestamp of the data, ensuring the non-tamperability and time traceability of the data.

[0139] Next, the system needs to determine the uniform resource locator (URL) used for off-chain storage of the verification process data. The URL points to the data resource stored off-chain, including multi-modal process data and its corresponding verification information. Since the off-chain storage can be distributed in different servers or database nodes, each data resource needs to be located through a unique URL. Here, the role of the uniform resource locator is to bind the data resource stored off-chain with the identifier recorded on-chain, ensuring that when accessing the on-chain signature evidence, the corresponding data resource off-chain can be quickly found.

[0140] After obtaining the on-chain identifier (i.e., the first hash value) and the off-chain resource locator (URL), the system maps the two through a pre-set data structure. Typically, the system employs efficient data structures such as hash tables or red-black trees to establish this mapping relationship. Hash tables enable fast lookup within O(1) time complexity, while red-black tree structures maintain high performance stability under complex query operations. Through this mapping structure, the system takes the on-chain identifier as the key (Key) and the off-chain uniform resource locator as the value (Value), thereby realizing the close association between on-chain and off-chain.

[0141] The establishment of this mapping relationship is crucial as it ensures that the system can quickly locate data resources from the off-chain when needed. In a distributed environment, multi-modal process data is stored across multiple nodes or databases, while the on-chain only stores hash digests and signature information of the data. Through the mapping relationship, the system can easily find the data stored off-chain through the on-chain identifier and verify its authenticity and integrity. Especially when users need to verify or access data, the system can quickly retrieve the corresponding off-chain data according to the hash value provided by the on-chain signature evidence and perform comparison, thereby ensuring that the data has not been tampered with or damaged during storage and transmission.

[0142] The effectiveness of the mapping relationship is reflected in several aspects. First, it ensures the efficiency and operability of on-chain and off-chain collaborative storage. Users can quickly locate data resources from the off-chain through on-chain information, avoiding tedious manual queries and positioning operations. Second, through this mapping relationship, the system can maintain data management consistency in a distributed environment. Regardless of where the data is stored, it can be quickly verified through the on-chain identifier. Finally, the mapping relationship effectively reduces the system's storage and transmission burden. The on-chain only stores key information such as hash values, signatures, and timestamps, while the off-chain saves complete data content. Through this division of labor, the system not only ensures the non-tamperability of on-chain data but also effectively protects the privacy and security of off-chain data.

[0143] Step 6: Store the verification process data and mapping relationship in the distributed process database.

[0144] An embodiment of Step 6 is to store the verification process data and mapping relationship in the distributed process database. The purpose is to achieve efficient and secure management of verification process data through the storage advantages of distributed databases, and to ensure data scalability, reliability, and fast query response capabilities. Distributed databases can handle large-scale multi-modal process data and optimize resource allocation and data storage efficiency through load balancing mechanisms. In this way, the system can provide efficient storage and query capabilities while ensuring data integrity and consistency.

[0145] In one possible implementation, with reference to Figure 4 , Figure 4 is a schematic diagram of a distributed process database provided by the present application. The distributed process database comprises a process data storage interface, a process data load balancing mechanism, and a distributed database. Step 6 specifically comprises the following steps:

[0146] Based on the process data load balancing mechanism, the verification process data and the mapping relationship are distributed to each database node in the distributed database for storage through the process data storage interface, wherein the process data load balancing mechanism distributes the verification process data and the mapping relationship to appropriate database nodes according to the characteristics of the data and the performance of each database node.

[0147] Specifically, first, the system obtains the verified process data and the mapping relationship between the on-chain identifier and the off-chain resource locator constructed in the previous steps. At this time, the verification process data contains the multi-modal process data and its related metadata minimum set, and the mapping relationship corresponds to the uniform resource locator (URL) of the off-chain resource through the on-chain hash value. In order to ensure efficient storage of data in a distributed environment, the system hands over these data to the distributed database for storage processing.

[0148] The structure design of the distributed process database can adopt a distributed NoSQL database such as MongoDB, which can support efficient storage of unstructured data and multi-modal data. First, the system transmits the verification process data and the mapping relationship to the database through the process data storage interface. The storage interface is responsible for formatting these data into a form suitable for distributed storage, and calling the storage function of the database in the background. Through this interface mechanism, the system can ensure that the data is uniform in format and general in interface during storage, and can provide support for future query and statistical analysis.

[0149] Next, the process data load balancing mechanism is activated to ensure that the verification process data can be distributed to appropriate database nodes. The role of the load balancing mechanism is to distribute data resources reasonably according to the characteristics of different data (such as data volume, access frequency, data type, etc.) and the performance and load of each distributed database node. Specifically, the system will dynamically adjust the storage location of the data according to the processing capacity, storage capacity, network bandwidth, etc. of the nodes, to ensure the overall storage and query efficiency of the system. Through this mechanism, the system can ensure optimal allocation of resources when dealing with large-scale, multi-modal data, reduce storage bottlenecks, and improve query response speed.

[0150] Once the verification process data and the mapping relationship are successfully stored in the distributed database, the system has the ability to quickly search and query. When a user accesses the verification process data, they can quickly locate the off-chain data resources through the hash value in the on-chain shared signature evidence and the mapping relationship. In the distributed database, the system establishes an index for the verification process data to speed up the query and positioning operations. Especially for multi-modal data queries, the system can quickly find the corresponding process data through the uniform resource locator (URL) and hash value index, achieving efficient retrieval. A distributed database such as MongoDB can store data on different nodes through a sharding mechanism when dealing with large-scale data, ensuring that the system can still provide stable performance and response when facing a large number of concurrent requests.

[0151] In one possible implementation, after step 6, the following steps are further included:

[0152] When a user accesses multi-modal process data, a first hash value of the on-chain shared signature evidence and a second hash value of the off-chain stored multi-modal process data are obtained;

[0153] A third hash value of the multi-modal process data is calculated;

[0154] The third hash value is compared with the first hash value and the second hash value respectively to determine whether the third hash value is consistent with the first hash value and / or the second hash value;

[0155] When the third hash value is consistent with the first hash value and / or the second hash value, the user is allowed to access the multi-modal process data;

[0156] When the third hash value is inconsistent with both the first hash value and the second hash value, the user is denied access to the multi-modal process data.

[0157] Specifically, when a user initiates a request to access multi-modal process data, the system first extracts the first hash value of the on-chain shared signature evidence from the blockchain. This first hash value is the unique identifier of the verification process data, which has been recorded in the blockchain through the on-chain signature evidence in the previous steps. The hash value is extracted to verify whether the data accessed by the user is consistent with the data recorded on the chain, ensuring that the data has not been tampered with or modified.

[0158] Next, the system extracts the off-chain stored multi-modal process data from the distributed process database through the mapping relationship and calculates its second hash value. The off-chain verification process data contains the user's requested data and its metadata minimum set. The system calculates the hash value of the off-chain stored data through the same hash algorithm (such as SHA-256), which is the second hash value. This hash value represents the integrity and status of the off-chain data and is the basis for comparing on-chain and off-chain data.

[0159] To further ensure the authenticity of the data, the system also needs to recalculate a third hash value for the off-chain multi-modal process data. This step is a real-time calculation of off-chain data to ensure that the current state of the data has not changed when the system responds to user requests. The third hash value is obtained by real-time calculation of off-chain data, which is a key step for final comparison with the first two stored hash values.

[0160] The system then compares the third hash value with the first hash value and the second hash value respectively. This step is the core of the verification process, by comparing the hash value of the current off-chain data with the hash value stored on the chain (the first hash value) and the hash value calculated initially off-chain (the second hash value), the system can determine whether the data has remained consistent during storage and transmission. If the three hash values are consistent, it means that the data has not been tampered with or modified on the chain and off-chain.

[0161] If the comparison result shows that the third hash value is consistent with the first hash value and / or the second hash value, the system will allow the user to access the multi-modal process data. At this time, the system will respond to the user's access request and provide the corresponding multi-modal process data. The user can safely access the verified data, ensuring the authenticity, integrity and tamper resistance of the data. This process ensures that the data accessed by the user is trustworthy and can be traced back to the history of on-chain notarization and off-chain verification.

[0162] If the comparison result shows that the third hash value is inconsistent with the first hash value and the second hash value, the system will deny the user access to the multi-modal process data. In this case, the system believes that the data may have been tampered with or errors occurred during storage or transmission, so access is denied to ensure that the user does not obtain incomplete or tampered data. The access denial mechanism is to protect the security of the system and data, to avoid fake data entering the system and thus damaging the credibility of the system.

[0163] Reference Figure 5 , Figure 5 is a structural diagram of a multi-modal process data credible storage system combined with a blockchain provided by the present application, the system comprising: an acquisition module and a processing module;

[0164] The acquisition module is configured to acquire multi-modal process data.

[0165] The processing module is configured to construct a metadata minimal set covering the entire life cycle of the multi-modal process data.

[0166] The processing module is further configured to store the multi-modal process data and the metadata minimal set off-chain, and verify the multi-modal process data to obtain verification process data.

[0167] The processing module is further configured to record a digital signature of the verification process data on the blockchain to obtain an on-chain shared signature record of the verification process data.

[0168] The processing module is further configured to establish a mapping relationship between a hash value of the on-chain shared signature record and the off-chain verification process data based on the on-chain shared signature record.

[0169] The processing module is further configured to store the verification process data and the mapping relationship in a distributed process database.

[0170] In a possible implementation, the processing module is further configured to:

[0171] perform authenticity verification, integrity verification, compliance information verification, and right confirmation information verification on the multi-modal process data to obtain verification information.

[0172] integrate the verification information and the multi-modal process data in the form of metadata according to the metadata minimum set to obtain verification process data.

[0173] In a possible implementation, the processing module is further configured to:

[0174] calculate a first hash value of the verification process data.

[0175] take the first hash value as a data digest and generate a timestamp for the data digest.

[0176] sign the data digest and the timestamp based on a private key associated with the multi-modal process data to generate signature information.

[0177] record the data digest, the timestamp, and the signature information on the blockchain to obtain an on-chain shared signature record of the verification process data.

[0178] In a possible implementation, the processing module is further configured to:

[0179] extract a first hash value of the on-chain shared signature record from the blockchain, and take the first hash value as an on-chain identifier.

[0180] determine a uniform resource locator corresponding to the off-chain storage of the multi-modal process data and the metadata minimum set.

[0181] establish a mapping relationship between the on-chain identifier and the uniform resource locator based on a preset data structure, wherein the on-chain identifier is taken as a key of the mapping relationship, and the uniform resource locator is taken as a value of the mapping relationship.

[0182] In a possible implementation, the processing module is further configured to:

[0183] The verification process data and the mapping relationship are distributed to each database node in the distributed database for storage through a process data storage interface based on a process data load balancing mechanism, wherein the process data load balancing mechanism distributes the verification process data and the mapping relationship to appropriate database nodes according to data characteristics and performance of each database node.

[0184] In a possible implementation, the acquisition module is further configured to acquire a first hash value of the on-chain shared signature evidence and a second hash value of the off-chain stored multi-modal process data when the user accesses the multi-modal process data.

[0185] The processing module is further configured to:

[0186] calculate a third hash value of the multi-modal process data;

[0187] compare the third hash value with the first hash value and the second hash value respectively, and determine whether the third hash value is consistent with the first hash value and / or the second hash value;

[0188] allow the user to access the multi-modal process data when the third hash value is consistent with the first hash value and / or the second hash value;

[0189] reject the user to access the multi-modal process data when the third hash value is inconsistent with the first hash value and the second hash value.

[0190] It should be noted that the multi-modal process data trusted storage system combined with the blockchain provided by the present application can execute the multi-modal process data trusted storage method combined with the blockchain of any of the above embodiments when specifically running, and the present embodiment will not be described here.

[0191] Figure 6 is a structural schematic diagram of an electronic device provided by the present application, such as Figure 6As shown, the electronic device can include a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can invoke the logical instructions in the memory 630 to execute the method for trusted storage of multi-modal process data combined with a blockchain, which includes: acquiring multi-modal process data; constructing a minimum set of metadata covering the full life cycle of the multi-modal process data; storing the multi-modal process data and the minimum set of metadata off-chain, and verifying the multi-modal process data to obtain verified process data; recording the digital signature of the verified process data on the blockchain to obtain an on-chain shared signature record of the verified process data; based on the on-chain shared signature record, establishing a mapping relationship between the hash value of the on-chain shared signature record and the off-chain verified process data; and storing the verified process data and the mapping relationship in a distributed process database.

[0192] In addition, the logical instructions in the memory 630 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory 630 (ROM, Read-Only Memory), a random access memory 630 (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0193] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the method for trusted storage of multi-modal process data combined with a blockchain provided by each of the above embodiments, and the method comprises: acquiring multi-modal process data; constructing a metadata minimum set covering the whole life cycle of the multi-modal process data; performing off-chain storage on the multi-modal process data and the metadata minimum set, and verifying the multi-modal process data to obtain verified process data; recording a digital signature of the verified process data in a blockchain to obtain an on-chain shared signature record of the verified process data; based on the on-chain shared signature record, establishing a mapping relationship between a hash value of the on-chain shared signature record and the off-chain verified process data; and storing the verified process data and the mapping relationship in a distributed process database.

[0194] In another aspect, the present application also provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor 610 to implement a method for trusted storage of multi-modal process data combined with a blockchain provided by each of the above embodiments, and the method comprises: acquiring multi-modal process data; constructing a metadata minimum set covering the whole life cycle of the multi-modal process data; performing off-chain storage on the multi-modal process data and the metadata minimum set, and verifying the multi-modal process data to obtain verified process data; recording a digital signature of the verified process data in a blockchain to obtain an on-chain shared signature record of the verified process data; based on the on-chain shared signature record, establishing a mapping relationship between a hash value of the on-chain shared signature record and the off-chain verified process data; and storing the verified process data and the mapping relationship in a distributed process database.

[0195] The system embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0196] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and of course, the implementation can also be through hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method of each embodiment or some parts of the embodiment.

[0197] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features thereof; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for trusted storage of multi-modal process data in conjunction with a blockchain, the method comprising: The method comprises: acquiring multi-modal process data; constructing a metadata minimum set covering the whole life cycle of the multi-modal process data; storing the multi-modal process data and the metadata minimum set off-chain, and verifying the multi-modal process data to obtain verified process data; storing a digital signature of the verified process data on a blockchain to obtain an on-chain shared signature record of the verified process data; the digital signature record comprises a data digest, a timestamp, and signature information; the storing of the digital signature of the verified process data on the blockchain to obtain the on-chain shared signature record of the verified process data specifically comprises: calculating a first hash value of the verified process data; taking the first hash value as the data digest and generating a timestamp for the data digest; signing the data digest and the timestamp based on a private key associated with the multi-modal process data to generate the signature information; storing the data digest, the timestamp, and the signature information on the blockchain to obtain the on-chain shared signature record of the verified process data; based on the on-chain shared signature record, establishing a mapping relationship between the hash value of the on-chain shared signature record and the off-chain verified process data; the establishing of the mapping relationship between the hash value of the on-chain shared signature record and the off-chain verified process data based on the on-chain shared signature record specifically comprises: extracting a first hash value of the on-chain shared signature record from the blockchain and taking the first hash value as an on-chain identifier; determining a uniform resource locator corresponding to the storage of the multi-modal process data and the metadata minimum set off-chain; based on a preset data structure, establishing a mapping relationship between the on-chain identifier and the uniform resource locator, wherein the on-chain identifier is taken as a key of the mapping relationship and the uniform resource locator is taken as a value of the mapping relationship; storing the verified process data and the mapping relationship in a distributed process database.

2. The multi-modal process data trustful storage method based on blockchain of claim 1, wherein, The verifying of the multi-modal process data to obtain verified process data specifically comprises: performing authenticity verification, integrity verification, compliance information verification, and right verification on the multi-modal process data to obtain verification information; integrating the verification information and the multi-modal process data in the form of metadata according to the metadata minimum set to obtain the verified process data.

3. The multi-modal process data trusted storage method integrated with blockchain according to claim 1, characterized in that, The distributed process database comprises a process data storage interface, a process data load balancing mechanism, and a distributed database; the storing of the verified process data and the mapping relationship in the distributed process database specifically comprises: based on the process data load balancing mechanism, distributing the verified process data and the mapping relationship to each database node in the distributed database for storage through the process data storage interface, wherein the process data load balancing mechanism distributes the verified process data and the mapping relationship to appropriate database nodes according to data characteristics and the performance of each database node.

4. The multi-modal process data trusted storage method integrated with blockchain according to claim 1, wherein, The method further comprises: When a user accesses the multi-modal process data, a first hash value of the on-chain shared signature record and a second hash value of the off-chain stored multi-modal process data are obtained; A third hash value of the multi-modal process data is calculated; The third hash value is compared with the first hash value and the second hash value respectively, to determine whether the third hash value is consistent with the first hash value and / or the second hash value; When the third hash value is consistent with the first hash value and / or the second hash value, the user is allowed to access the multi-modal process data; When the third hash value is inconsistent with the first hash value and the second hash value, the user is denied to access the multi-modal process data. 5.A multi-modal process data trusted storage system combined with a blockchain, characterized in that, Comprise: An acquisition module and a processing module; The acquisition module is configured to acquire multi-modal process data; The processing module is configured to construct a metadata minimum set covering the whole life cycle of the multi-modal process data; The processing module is further configured to store the multi-modal process data and the metadata minimum set off-chain, and verify the multi-modal process data to obtain verified process data; The processing module is further configured to record a digital signature record of the verified process data on a blockchain to obtain an on-chain shared signature record of the verified process data; the digital signature record comprises a data digest, a timestamp, and signature information; The processing module is further configured to calculate a first hash value of the verified process data; take the first hash value as the data digest, and generate a timestamp for the data digest; sign the data digest and the timestamp based on a private key associated with the multi-modal process data to generate the signature information; and record the data digest, the timestamp, and the signature information on the blockchain to obtain the on-chain shared signature record of the verified process data; The processing module is further configured to establish a mapping relationship between a hash value of the on-chain shared signature record and off-chain verified process data based on the on-chain shared signature record; the mapping relationship is established based on the on-chain shared signature record, and comprises: extracting a first hash value of the on-chain shared signature record from the blockchain, and taking the first hash value as an on-chain identifier; determining a uniform resource locator corresponding to the storage of the multi-modal process data and the metadata minimum set off-chain; and establishing a mapping relationship between the on-chain identifier and the uniform resource locator based on a preset data structure, wherein the on-chain identifier is taken as a key of the mapping relationship, and the uniform resource locator is taken as a value of the mapping relationship; The processing module is further configured to store the verified process data and the mapping relationship in a distributed process database.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the multi-modal process data trusted storage method combined with a blockchain according to any one of claims 1-4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, which is executed by a processor, implements the method for trusted storage of multi-modal process data in combination with a blockchain according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program, which is executed by a processor, implements the method for trusted storage of multi-modal process data in combination with a blockchain according to any one of claims 1 to 4.

Images