Large model training data sharing platform based on block chain decentration architecture and data authenticity guarantee method and system

Abstract

The invention relates to the technical field of artificial intelligence and block chains, in particular to a block chain decentralized architecture-based large model training data sharing platform and a data authenticity guarantee method and system, and the method comprises the following steps: a data provider uploads training data to the platform through a data provider node, the size and format of the data are preliminarily checked; carrying out encryption processing on the uploaded data, and carrying out associated storage on an encryption key and data related information; generating a hash value of the encrypted data, and recording source information of the data; the method has the beneficial effects that through the non-tampering characteristic of the block chain, it is ensured that the source and content of data are real and reliable, and the influence of false data on large model training is avoided. The block chain records uploading, sharing and using information of the data, so that the source and the flow direction of the data are clear and can be checked, and supervision and auditing are facilitated. The data is stored and transmitted in an encrypted form, and only authorized users can access the data, so that the privacy of data providers and users is effectively protected.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence and blockchain, and specifically provides a large model training data sharing platform based on a blockchain decentralized architecture, a method and a system for ensuring data authenticity. Background Art

[0002] With the rapid development of artificial intelligence technology, large models have achieved remarkable results in many fields such as natural language processing, computer vision, and speech recognition. However, the training of large models requires a large amount of high-quality data, which is often scattered in different institutions and individuals, and data sharing has become a key factor in promoting the development of large models.

[0003] Currently, there are many problems in traditional data sharing platforms. On the one hand, it is difficult to guarantee the authenticity and reliability of data. Data providers may provide false or low-quality data, thus affecting the training effect of large models. On the other hand, it is difficult to trace and audit the source and usage of data, which easily leads to data security and privacy issues.

[0004] Blockchain technology has characteristics such as decentralization, immutability, and traceability, providing a new idea for solving the problems in large model training data sharing. Combining blockchain technology with large model training data sharing can effectively guarantee the authenticity and reliability of data and achieve traceability and auditability of data. Summary of the Invention

[0005] The purpose of the present invention is to provide a large model training data sharing platform based on a blockchain decentralized architecture, a method and a system for ensuring data authenticity, so as to solve the problems raised in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solutions: A large model training data sharing platform based on a blockchain decentralized architecture and a method for ensuring data authenticity, including the following steps:

[0007] Data providers upload training data to the platform through data provider nodes and perform a preliminary check on the data size and format;

[0008] Encrypt the uploaded data and store the encryption key in association with the data-related information;

[0009] Generate a hash value of the encrypted data and record the source information of the data;

[0010] Record the hash value and source information in the blockchain network;

[0011] Data users submit data requests through data user nodes and perform identity verification;

[0012] Evaluate and authorize the requests of data users according to the rules of smart contracts;

[0013] After authorization, the data user accesses the authorized data.

[0014] Preferably, the specific steps in the data uploading process further include: before uploading the data, the data provider signs the data using the private key; the platform checks the format and validates the content of the uploaded data to ensure that the data meets the platform requirements; compare the public key provided by the data provider with the signature to verify the ownership and authenticity of the data.

[0015] Preferably, the data storage and management steps include: storing the encrypted data in a distributed storage system; recording the storage location and access permissions of the data on the blockchain network; ensuring that only authorized users can access and decrypt the data.

[0016] Preferably, the data usage auditing steps include: the data user submits a usage application to the platform, stating the purpose and scope of use; the platform reviews the usage application according to the rules of the smart contract; after the review is passed, the data user is authorized to access the data, and the usage behavior is recorded on the blockchain network; the usage record includes the usage time, usage method, and usage result, which are used for subsequent auditing and supervision.

[0017] Preferably, the functions of the smart contract include: defining the data sharing rules, authorization mechanism, and fee settlement method; automatically executing the data upload verification program to check the data format, content, hash value, and source information; performing authorization evaluation on the requests of data users according to the rules set by the data provider; conducting fee settlement to ensure the security and transparency of transactions; supporting the update and maintenance of rules, and ensuring the fairness and reasonableness of rules through auditing and consensus mechanisms.

[0018] A system for a large model training data sharing platform and data authenticity guarantee method based on the blockchain decentralized architecture, including a data provider node, a data user node, a blockchain network, and a smart contract, where:

[0019] The data provider node is used to upload training data and perform data encryption, hash value generation, and source information recording;

[0020] The data user node is used to submit data requests, perform identity verification, and obtain data access permissions according to the rules of the smart contract;

[0021] The blockchain network is used to record the upload, sharing, and usage information of data to ensure the immutability and traceability of data;

[0022] The smart contract is used to define the data sharing rules, authorization mechanism, and fee settlement method to implement an automated business process.

[0023] Preferably, when uploading training data, the data provider node performs the following steps to ensure data authenticity: check the data format and content verification to ensure that the data meets the platform requirements; sign the data with a private key, and the data user can use the public key to verify the data ownership and authenticity; generate a hash value of the data and record the hash value and the data source information on the blockchain to ensure that the data is not tampered with during storage and transmission.

[0024] Preferably, the data storage and management methods include: storing the encrypted data in a distributed storage system such as IPFS to ensure high availability, fault tolerance, and scalability of the data; recording the storage location and access rights of the data on the blockchain network to ensure that only authorized users can access the data.

[0025] Preferably, the data usage auditing process includes: the data user submits a usage application to the platform, specifying the usage purpose and scope in detail; the platform reviews the usage application according to the rules of the smart contract, considering the sharing rules of the data provider, data security, and privacy factors; after the review is passed, the data user is authorized to access the data, and the usage records including usage time, usage method, and usage results are recorded on the blockchain for subsequent auditing and supervision.

[0026] Preferably, the functions of the smart contract further include: automatically executing the data upload verification program to check whether the data format, content, hash value, and source information meet the platform requirements; authorizing and evaluating the requests of data users according to the rules set by the data provider and performing corresponding fee settlements; supporting the update and maintenance of the rules to ensure the fairness and reasonableness of the rules, and at the same time undergoing strict testing and auditing to prevent loopholes and security risks.

[0027] Compared with the prior art, the beneficial effects of the present invention are:

[0028] The large model training data sharing platform, data authenticity guarantee method, and system based on the blockchain decentralized architecture proposed by the present invention ensure the authenticity and reliability of the data source and content through the immutable characteristics of the blockchain, avoiding the impact of false data on large model training. The blockchain records the data upload, sharing, and usage information, making the data source and flow clearly traceable, facilitating supervision and auditing. The data is stored and transmitted in encrypted form, and only authorized users can access the data, effectively protecting the privacy of data providers and users. A fair and transparent data sharing platform is established to encourage data providers to actively share data and promote the development of large model technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flowchart of the method of the present invention. Detailed implementation manners

[0030] In order to clearly and completely describe the objectives, technical solutions of the present invention and make the advantages clearer, the following further elaborates on the embodiments of the present invention with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are some, rather than all, embodiments of the present invention, and are merely used to explain the embodiments of the present invention, rather than limiting the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0031] Embodiment 1, please refer to Figure 1 , the present invention provides a technical solution: a large model training data sharing platform and a data authenticity guarantee method based on the blockchain decentralized architecture, including:

[0032] It consists of a data provider node, a data user node, a blockchain network, and a smart contract. Each part works together to achieve the safe and efficient sharing of large model training data.

[0033] 1. Data provider node

[0034] The data provider node is the interface for the data provider to interact with the platform, and undertakes the important tasks of uploading training data, preprocessing, and information recording. Data uploading: The data provider uploads the locally stored training data to the platform through this node. During the uploading process, the system will conduct a preliminary check on the size, format, etc. of the data to ensure that the data can be normally processed by the platform. For example, for image data, it will check whether it is a common picture format (such as JPEG, PNG, etc.); for text data, it will check whether it is UTF-8 encoded, etc. Data encryption processing: To ensure the security and privacy of the data, the node will use an advanced encryption algorithm (such as the AES symmetric encryption algorithm) to encrypt the uploaded data. The encryption key will be associated with the relevant information of the data and stored. Only authorized users can obtain and decrypt the data. In this way, even if the data is intercepted during transmission or storage, attackers cannot obtain the sensitive information therein. Hash value generation: The node will calculate the hash value of the encrypted data, and the commonly used hash algorithm is SHA-256. The hash value is unique, and even if there are minor changes in the data, the hash value will be very different. This hash value will be used as the unique identifier of the data for subsequent data verification and integrity check. Source information recording: The node will record the source information of the data in detail, including the time, location, device, etc. when the data is generated. For example, if the data is collected by a certain sensor, the model number, serial number of the sensor, and the geographical location where it is located will be recorded; if the data is manually annotated, the identity information of the annotator and the annotation time will be recorded. These source information will be recorded on the blockchain together with the hash value of the data to ensure the traceability of the data.

[0035] 2. Data User Node

[0036] The data user node provides a convenient data request and usage path for data users, while ensuring the security and compliance of data access. Data Request: Data users submit requests for the required training data to the platform through this node. The request content includes the type of data (such as images, text, audio, etc.), quantity, quality requirements, and usage purposes. For example, a natural language processing research team may request a large amount of labeled text data for training a new language model. Authentication: Before requesting data, the node strictly verifies the identity of the data user. The verification methods can include various forms such as username, password, digital certificate, etc. Only users who pass the authentication can continue with the data request operation, preventing unauthorized users from obtaining data. Data Authorization and Access Control: The node evaluates and authorizes the data user's request according to the rules of the smart contract. The smart contract takes into account factors such as the sharing rules, usage fees, usage period, etc. set by the data provider. For example, if the data provider stipulates that the data can only be used for academic research, and the user requests to use it for commercial purposes, the smart contract will reject the request. Once the authorization is passed, the node assigns corresponding access rights to the data user, ensuring that the user can only access the data for which they are authorized.

[0037] 3. Blockchain Network

[0038] The blockchain network is the core infrastructure of the platform. It uses distributed ledger technology to record information on data upload, sharing, and usage, ensuring the immutability and traceability of data. Node Composition: The blockchain network consists of multiple nodes, which can be distributed in different geographical locations and institutions. Each node stores a complete copy of the blockchain and synchronizes data and communicates through the network. A consensus mechanism (such as PoW, PoS, etc.) is adopted among the nodes to ensure data consistency and security. Distributed Ledger Record: Each block in the blockchain network contains important information such as the hash value of the data, source information, usage records, etc. When the data provider uploads data, its hash value and source information are packaged into a new block and added to the blockchain; when the data user uses the data, usage records (such as usage time, usage method, usage result, etc.) are also recorded on the blockchain. Due to the immutability of the blockchain, these records cannot be modified or deleted once written, ensuring the authenticity and integrity of the data. Data Synchronization and Verification: When new transactions (such as data upload, data usage) occur, the nodes in the blockchain network perform data synchronization and verification. The nodes verify whether the new block complies with the consensus rules, including the correctness of the hash value, the integrity of the data source information, etc. Only the blocks that pass the verification can be added to the blockchain, thus ensuring the data consistency and security of the entire blockchain network.

[0039] 4. Smart Contract

[0040] The smart contract is the core rules engine of the platform, which defines the data sharing rules, authorization mechanisms, and fee settlement methods in the form of code, and realizes automated business processes. Data upload verification and storage: When a data provider uploads data, the smart contract will automatically execute the data verification program. It will check whether the data format, content, hash value, and source information meet the requirements of the platform. If the verification passes, the smart contract will store the relevant information of the data on the blockchain and store the data in the distributed storage system; if the verification fails, the smart contract will reject the data upload and feedback error information to the data provider. Data authorization and fee settlement: When a data user requests data, the smart contract will conduct an authorization assessment according to the sharing rules set by the data provider. For example, the data provider can set limits on the number of times the data can be used, the usage period, usage fees, etc. The smart contract will judge whether to grant the data user access rights according to these rules and conduct corresponding fee settlements. The fee settlement can be carried out through cryptocurrencies, etc., to ensure the security and transparency of the transaction. Rule update and maintenance: The rules of the smart contract can be updated and maintained according to the development of the platform and the needs of users. The update process requires certain review and consensus mechanisms to ensure the fairness and reasonableness of the rules. At the same time, the code of the smart contract will be strictly tested and audited to prevent loopholes and security risks.

[0041] Not only the platform is needed, but also methods are required to guarantee data authenticity. Through the collaborative work of these three stages, the authenticity and reliability of the large model training data are ensured.

[0042] 1. Data Upload Verification

[0043] Data upload verification is the first line of defense to ensure data authenticity, using various means to ensure the reliable source and true content of data. Data signature: Before uploading data, the data provider needs to sign the data using their own private key. The signature process uses an asymmetric encryption algorithm (such as RSA) to ensure that only the data provider can sign the data. The data user can verify the signature using the data provider's public key to confirm the ownership and authenticity of the data. Format check and content verification: The platform will perform format checks and content verification on the uploaded data. Format checks mainly check whether the file format, encoding method, etc. of the data meet the requirements of the platform; content verification will evaluate the quality, integrity, etc. of the data. For example, for image data, it will check whether the image is damaged and whether the resolution meets the requirements; for text data, it will check whether the text contains sensitive information and whether the grammar is correct. Hash value recording: Generate the hash value of the data and record the hash value and the source information of the data on the blockchain. The recording of the hash value can ensure that the data is not tampered with during storage and transmission. If the data changes in any way, its hash value will also change accordingly. By comparing the hash value recorded on the blockchain with the current hash value of the data, it can be found whether the data has been tampered with.

[0044] 2. Data storage

[0045] During the data storage stage, the main focus is on data security and traceability, achieving secure data management through encrypted storage and blockchain recording. Encrypted storage: The data is stored in an encrypted form in a distributed storage system such as IPFS (InterPlanetary File System). The distributed storage system has advantages such as high availability, fault tolerance, and scalability, which can ensure the long-term preservation of data. The use of encryption algorithms further guarantees data security, and only authorized users can obtain and decrypt the data. Storage location and access permission recording: The blockchain network records the storage location and access permissions of the data. The storage location information can help data users quickly locate and obtain the required data; the access permission recording ensures that only authorized users can access the data. For example, the blockchain will record information such as the data range and access time that a certain data user can access, realizing data traceability and controllability.

[0046] 3. Data usage auditing

[0047] During the data usage audit phase, the usage behaviors of data users are supervised and recorded to ensure that the data is used in compliance with regulations and purposes. Submission of usage application: When using data, data users need to submit a usage application to the platform and explain the purpose and scope of use. The application content includes the data usage scenario, usage period, expected usage results, etc. For example, when a medical research team uses patients' medical data, it needs to elaborate on the research purpose, methods, and expected outcomes. Application review: The platform will review the usage application according to the rules of the smart contract. The review process will consider factors such as the sharing rules of data providers, data security, and privacy. If the application meets the requirements, the platform will authorize the data user to access the data; if the application does not meet the requirements, the platform will reject the application and feedback the reason for rejection to the data user. Usage record keeping: The usage records of data users will be recorded on the blockchain, including usage time, usage method, usage results, etc. These records can be used for subsequent audits and supervision to ensure that data users use data within the specified purpose and scope. For example, if a data user exceeds the authorized usage scope, it can be discovered and corresponding measures can be taken by checking the usage records on the blockchain.

[0048] Embodiment 2, based on Embodiment 1, proposes a system for a large model training data sharing platform and a data authenticity guarantee method according to claim 1, which includes a data provider node, a data user node, a blockchain network, and a smart contract, where:

[0049] The data provider node is used to upload training data and perform data encryption, hash value generation, and source information recording; when uploading training data, the data provider node performs the following steps to guarantee data authenticity: perform format check and content verification on the data to ensure that the data meets the platform requirements; sign the data with a private key, and the data user can use the public key to verify the data ownership and authenticity; generate the hash value of the data and record the hash value and the source information of the data on the blockchain to ensure that the data is not tampered with during storage and transmission. The data storage and management methods include: storing the encrypted data in a distributed storage system such as IPFS to ensure the high availability, fault tolerance, and scalability of the data; recording the storage location and access permissions of the data on the blockchain network to ensure that only authorized users can access the data.

[0050] The data user node is used to submit data requests, authenticate identities, and obtain data access rights according to the rules of the smart contract; the data user submits a usage application to the platform, specifying in detail the purpose and scope of use; the platform reviews the usage application according to the rules of the smart contract, considering the sharing rules of the data provider, and the security and privacy factors of the data; after the review is passed, the data user is authorized to access the data, and the usage records including the usage time, usage method, and usage results are recorded on the blockchain for subsequent auditing and supervision.

[0051] The blockchain network is used to record the upload, sharing, and usage information of data, ensuring the immutability and traceability of data; automatically execute the data upload verification program to check whether the data format, content, hash value, and source information meet the platform requirements; conduct authorization evaluation on the requests of data users according to the rules set by the data provider and perform corresponding fee settlement; support the update and maintenance of rules to ensure the fairness and reasonableness of the rules, while undergoing strict testing and auditing to prevent loopholes and security risks.

[0052] The smart contract is used to define the data sharing rules, authorization mechanism, and fee settlement method, realizing an automated business process.

[0053] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A large model training data sharing platform based on the decentralized architecture of blockchain and a method for ensuring data authenticity, characterized in that: It includes the following steps: The data provider uploads the training data to the platform through the data provider node and conducts a preliminary check on the data size and format; Encrypt the uploaded data and store the encryption key associated with the data-related information; Generate the hash value of the encrypted data and record the source information of the data; Record the hash value and source information to the blockchain network; The data user submits a data request through the data user node and conducts identity authentication; Evaluate and authorize the data user's request according to the rules of the smart contract; After the authorization is passed, the data user accesses the authorized data.

2. The large model training data sharing platform and data authenticity guarantee method based on the blockchain decentralized architecture according to claim 1, characterized in that: The specific steps in the data upload process also include: before uploading the data, the data provider signs the data with the private key; the platform conducts a format check and content verification on the uploaded data to ensure that the data meets the platform requirements; compare the public key provided by the data provider with the signature to verify the ownership and authenticity of the data.

3. A large model training data sharing platform and data authenticity guarantee method based on the blockchain decentralized architecture according to claim 2, characterized in that: The data storage and management steps include: storing the encrypted data in a distributed storage system; recording the storage location and access rights of the data on the blockchain network; ensuring that only authorized users can access and decrypt the data.

4. A large model training data sharing platform and data authenticity guarantee method based on the blockchain decentralized architecture according to claim 3, characterized in that: The data usage audit steps include: the data user submits a usage application to the platform, stating the usage purpose and scope; the platform reviews the usage application according to the rules of the smart contract; after the review is passed, authorizes the data user to access the data and records the usage behavior to the blockchain network; the usage record includes the usage time, usage method, and usage result, which are used for subsequent audit and supervision.

5. A large model training data sharing platform and data authenticity guarantee method based on the blockchain decentralized architecture according to claim 4, characterized in that: The functions of the smart contract include: defining the data sharing rules, authorization mechanism, and fee settlement method; automatically executing the data upload verification program to check the data format, content, hash value, and source information; authorizing and evaluating the data user's request according to the rules set by the data provider; conducting fee settlement to ensure the security and transparency of the transaction; supporting the update and maintenance of the rules, and ensuring the fairness and reasonableness of the rules through the review and consensus mechanism.

6. A system for a large model training data sharing platform and a data authenticity guarantee method based on the blockchain decentralized architecture according to claim 1, characterized in that: It includes a data provider node, a data user node, a blockchain network, and a smart contract, where: The data provider node is used to upload training data and conduct data encryption, hash value generation, and source information recording; The data user node is used to submit data requests, conduct identity authentication, and obtain data access rights according to the rules of the smart contract; The blockchain network is used to record the upload, sharing, and usage information of the data to ensure the immutability and traceability of the data; The smart contract is used to define the data sharing rules, authorization mechanism, and fee settlement method to implement an automated business process.

7. A system according to claim 6, wherein: When the data provider node uploads the training data, it performs the following steps to ensure data authenticity: conduct a format check and content verification on the data to ensure that the data meets the platform requirements; sign the data with the private key, and the data user can use the public key to verify the data ownership and authenticity; generate the hash value of the data and record the hash value and the source information of the data to the blockchain to ensure that the data is not tampered with during storage and transmission.

8. A system according to claim 7, wherein: The data storage and management methods include: storing the encrypted data in a distributed storage system such as IPFS to ensure the high availability, fault tolerance, and scalability of the data; recording the storage location and access rights of the data on the blockchain network to ensure that only authorized users can access the data.

9. A system according to claim 8, wherein: The data usage auditing process includes: the data user submitting a usage application to the platform, specifying in detail the purpose and scope of use; the platform reviewing the usage application according to the rules of the smart contract, considering the sharing rules of the data provider, and the security and privacy factors of the data; after the review is passed, authorizing the data user to access the data and recording the usage records including the usage time, usage method, and usage results on the blockchain for subsequent auditing and supervision.

10. A system according to claim 9, wherein: The functions of the smart contract also include: automatically executing the data upload verification program to check whether the format, content, hash value, and source information of the data meet the platform requirements; authorizing and evaluating the requests of the data users according to the rules set by the data provider and performing corresponding fee settlement; supporting the update and maintenance of the rules to ensure the fairness and reasonableness of the rules, and at the same time undergoing strict testing and auditing to prevent loopholes and security risks.

Images