Proof-of-quality-based generative model blockchain inference deployment method and system

Abstract

The application discloses a kind of generative model blockchain inference deployment method and system based on quality proof, method includes: input query to generative model and output response;Cross encoder model is run respectively by multiple quality assessors in blockchain, based on query-response pair, response is quality evaluated and multiple quality evaluation scores encrypted using private key are output;Based on quality evaluation score, incentive mechanism is used to give different reward values to each quality assessor;Quality evaluation score higher than predetermined threshold is encrypted using public key and then written into shared file, all quality evaluation scores in the shared file are recovered and average value is taken as the final quality evaluation score of response;Based on final quality evaluation score, generative model is optimized and deployment is completed.The new paradigm of quality proof proposed by the application can ensure the credibility of the output and reduce the computational overhead, further improving the robustness to support the deployment of any large generative model on the blockchain architecture.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a generative model blockchain inference deployment method and system based on quality proof. BACKGROUND

[0002] In recent years, deep learning has triggered a revolutionary change in the field of artificial intelligence, making unprecedented progress in natural language processing (NLP), image recognition, and audio processing. Generative AI models, such as the GPT series, DALL·E, and Stable Diffusion, have achieved highly innovative tasks by generating text, images, and audio. These models have shown great application potential in text generation, dialogue systems, content creation, personalized recommendations, artistic creation, programming assistance, and education, and have become important tools for improving productivity and efficiency in key areas such as medical health, financial forecasting, and legal assistance.

[0003] At the same time, blockchain technology, as a decentralized and secure ledger system, provides strong support for data integrity and transparency. Its combination with artificial intelligence not only enhances the security and credibility of AI using data, but also alleviates the risks of data leakage and fraud. The transparent auditing capability of blockchain can enhance the credibility of AI systems, making decisions more stable and responsible. Decentralized AI models can also avoid technological monopolies and promote more fair distribution of technological benefits.

[0004] However, despite the great potential of the combination of artificial intelligence and blockchain, the deployment of generative AI models in decentralized networks faces serious challenges. In particular, generative model inference usually requires a large amount of computing resources, while traditional blockchain consensus mechanisms require repeated computation on multiple nodes, resulting in high costs and non-negligible delays.

[0005] To solve these problems, researchers are exploring advanced encryption technologies such as zero-knowledge proof. However, existing technologies have significant limitations in scalability and floating-point number processing, making them unsuitable for real-world generative AI models. Among them, ZKML (Zero-Knowledge Machine Learning) technology attempts to convert deep neural networks into circuits and generate proofs of inference operations, but is only suitable for small models with simple structure and fewer layers. Its technical defects include limited scalability, floating-point number processing difficulties, and high computational overhead.

[0006] Another solution is OPML (Optimistic Machine Learning), which is based on the optimistic verification idea and aims to improve the efficiency of on-chain inference by reducing redundant calculations. However, OPML also has technical defects such as high verification delay, vulnerability of optimistic assumptions, and high computational cost. In particular, when dealing with complex generative models such as multi-layer Transformer models, OPML still faces similar challenges as ZKML, such as floating-point operation difficulties and scalability limitations.

[0007] In summary, although the combination of generative AI models and blockchain technology has great potential and application value, it still faces many technical challenges. In order to promote the further development of this field, it is necessary to continuously explore new technologies and methods to overcome the limitations of existing technologies, achieve low-latency, high-performance, and trustworthy generative model inference. This is not only a hot and difficult topic of current research, but also an important direction for the development of artificial intelligence and blockchain technology in the future. SUMMARY

[0008] In view of the above, the purpose of the present application is to provide a quality proof-based generative model blockchain inference deployment method and system, a new inference paradigm called quality proof is proposed, which directly evaluates the quality of model inference results to ensure the credibility of the output and reduce computational overhead, combined with an incentive mechanism and a two-stage consensus mechanism to ensure the robustness of quality proof, to support the deployment of any large generative model on a blockchain architecture.

[0009] To achieve the above-mentioned purpose of the application, the technical solutions provided by the present application are as follows:

[0010] The quality proof-based generative model blockchain inference deployment method provided by the embodiments of the present application comprises the following steps:

[0011] Input the query into the generative model and output the response to obtain a query-response pair;

[0012] Run the cross-encoder model by multiple quality assessors in the blockchain, based on the query-response pair, assess the quality of the response and output multiple quality assessment scores encrypted with private keys;

[0013] Encrypt the quality assessment scores higher than the predetermined threshold with a public key and write them into a shared file, restore all quality assessment scores in the shared file, and take the average of all quality assessment scores in the shared file as the final quality assessment score of the response;

[0014] Further optimize the generative model based on the final quality assessment score to complete the model deployment.

[0015] Preferably, it further comprises:

[0016] The label quality evaluation score is obtained by manually performing quality evaluation based on the query-response pair, the label quality evaluation score is compared with the quality evaluation scores generated by the plurality of quality evaluators, different reward values are assigned to the quality evaluators according to the comparison result, and the quality evaluators are optimized.

[0017] Preferably, the quality evaluator adopts a large language model.

[0018] Preferably, the cross-encoder model adopts a BERT model.

[0019] Preferably, the quality evaluator performs quality evaluation on the response according to the consistency between the response and the query.

[0020] Preferably, the cross-encoder model supports efficient operation on a CPU or a GPU.

[0021] In a second aspect, to achieve the above-mentioned object, an embodiment of the present application further provides a generative model blockchain inference deployment system based on quality proof, which is implemented by using the generative model blockchain inference deployment method based on quality proof, and includes an inference module, an evaluation module, a consensus module, and an optimized deployment module.

[0022] The inference module is configured to input a query into a generative model and output a response, so as to obtain a query-response pair.

[0023] The evaluation module is configured to run a cross-encoder model by a plurality of quality evaluators in a blockchain, perform quality evaluation on the response based on the query-response pair, and output a plurality of quality evaluation scores encrypted by private keys.

[0024] The consensus module is configured to write the quality evaluation scores higher than a predetermined threshold into a shared file after being encrypted by a public key, restore all the quality evaluation scores in the shared file, and take the average value of all the quality evaluation scores in the shared file as a final quality evaluation score of the response.

[0025] The optimized deployment module is configured to further optimize the generative model based on the final quality evaluation score, and complete model deployment.

[0026] Preferably, the system further includes an incentive module.

[0027] The incentive module is configured to obtain a label quality evaluation score by manually performing quality evaluation based on the query-response pair, compare the label quality evaluation score with the quality evaluation scores generated by the plurality of quality evaluators, assign different reward values to the quality evaluators according to the comparison result, and optimize the quality evaluators.

[0028] In a third aspect, to achieve the above-mentioned object, the embodiment of the present application further provides an electronic device comprising a memory and a processor, wherein the memory is configured to store a computer program, and the processor is configured to implement the above-mentioned quality proof based generative model blockchain inference deployment method when executing the computer program.

[0029] In a fourth aspect, to achieve the above-mentioned object, the embodiment of the present application further provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is configured to implement the above-mentioned quality proof based generative model blockchain inference deployment method when being executed by a computer.

[0030] Compared with the prior art, the present application has at least the following beneficial effects:

[0031] (1) Systematically guarantee the integrity of generative model services in a decentralized environment: Unlike traditional model centralized deployment protocols, it is both important and difficult to systematically guarantee the integrity of artificial intelligence model services in a fully decentralized environment, especially on a trustless blockchain. The present application proposes a quality proof (Proof of Quality, PoQ) method to guarantee the integrity of generative model output through an innovative inference framework, rather than verifying the entire inference process. Traditional verification methods, such as ZKML and OPML, focus on the process of each inference step, while PoQ directly evaluates the quality of the generative model inference result, ensuring the credibility of the output.

[0032] (2) Verify the output quality of the generative model without increasing the computational overhead: A lightweight quality evaluation model is the key to solving the problem of computational overhead for PoQ. The PoQ proposed by the present application uses a BERT-based cross-encoder model to evaluate the quality of the input and output. These evaluation models are much lighter than large generative models, so they can significantly reduce computational overhead and improve processing efficiency.

[0033] (3) Maintain the robustness of the protocol when facing lazy or malicious participants: The present application adopts an incentive mechanism and a two-stage consensus mechanism to motivate participants to follow the protocol and prevent lazy or malicious behavior, ensuring the robustness of PoQ. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0035] Figure 1is a flowchart of a quality proof-based generative model blockchain inference deployment method provided by an embodiment of the present application.

[0036] Figure 2 is a PoQ inference framework schematic diagram provided by an embodiment of the present application.

[0037] Figure 3 is an evaluation process schematic diagram in the PoQ inference framework provided by an embodiment of the present application.

[0038] Figure 4 is a structural schematic diagram of a quality proof-based generative model blockchain inference deployment system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the protection scope of the present application.

[0040] The inventive concept of the present application is that, in view of the problems of high verification delay, high computational complexity and insufficient robustness in the combination of generative models and blockchains in the prior art, an embodiment of the present application provides a quality proof-based generative model blockchain inference deployment method and system, proposes a new paradigm called quality proof (PoQ), which focuses on verifying the quality of the inference output of the generative model rather than the inference process itself, thereby ensuring the credibility of the output; a lightweight cross-encoder model is used for quality evaluation, which can reduce computational overhead and improve processing efficiency; through an incentive mechanism and a two-stage consensus mechanism, the robustness of PoQ can be ensured, thereby realizing the optimized deployment of any large generative model on the blockchain architecture.

[0041] Figure 1 is a flowchart of a quality proof-based generative model blockchain inference deployment method provided by an embodiment of the present application. As shown in Figure 1 , an embodiment provides a quality proof-based generative model blockchain inference deployment method, comprising the following steps:

[0042] S1, input a query into a generative model and output a response to obtain a query-response pair.

[0043] In generative AI services, the output quality is much more important than the inference process. First, users are only willing to pay for the service when the generated response is satisfactory. Second, the model output quality is not necessarily proportional to the computational effort of the model, although the scaling law usually indicates that larger models perform better, the actual model behavior can be different, and some small but well-optimized models are superior to larger models, therefore, it is more reasonable to allocate rewards to service providers on the blockchain according to the output quality rather than the computational effort. Finally, verifying the model inference output is usually much easier than the inference process itself, the most advanced NLP models, such as Llama 3 and Mixtral, usually contain tens of billions of parameters, however, more compact models with millions of parameters can effectively evaluate the output quality of these larger models.

[0044] As shown in Figure 2 A new paradigm called Proof of Quality (PoQ) is proposed, which focuses on verifying the quality of the model inference output rather than the inference process itself. PoQ aims to support trustworthy model inference services on the blockchain with minimal overhead. Specifically, given a user query q, such as a pure text question, a single participant with strong computing power (e.g., GPU) generates and outputs a corresponding response r, such as a text answer or an image, by running a generative model with the query as input through a generative function F within the generative model, and submits the query and response as a query-response pair (q, r).

[0045] S2, a plurality of quality assessors in the blockchain respectively run a cross-encoder model, based on the query-response pair, to assess the quality of the response and output a plurality of quality assessment scores encrypted with a private key.

[0046] In the new framework of PoQ reasoning proposed in the embodiments, it is crucial to select the appropriate evaluation method based on the response domain R. The specific implementation of this framework will vary, as different response domains R require very different quality evaluation methods. There is a huge difference between the quality of the generated images and the quality of the chatbot responses. Cross-encoder technology is now commonly used as a filtering method for the Retrieval-Augmented Generation (RAG) scheme, with the basic idea being to concatenate a text query and a text document, and the combined string is input into a Transformer-based model, such as BERT, to generate a scalar score that measures the consistency between the query and the response, indicating how good or bad the query answer is. In PoQ, the embodiments of the present application directly use fully optimized cross-encoder models available on the Huggingface platform, which are optimized based on large entities containing question pairs and relevant documents during model training. The performance statistics of the mainstream cross-encoder models (all BERT models) available in the open source community are summarized in Table 1, and the inference delay and running cost of these models are much lower than common large language models (LLM), such as Llama 2 and Mixtral. All these cross-encoder models can run efficiently on CPUs and GPUs, and such low resource requirements enable the quality evaluation step in PoQ to reduce computational overhead and improve processing efficiency.

[0047] Table 1 Performance statistics of cross-encoder models

[0048]

[0049] The performance results in Table 1 show that cross-encoder models are very efficient, so it is economically feasible to deploy a simpler consensus mechanism with replicated computation to build a robust and fast-response decentralized system.

[0050] In this embodiment, each query-response pair (q, r) is sent to k>1 quality assessors (using a large language model). The assessors run a quality assessment method M: Q×R→S, where Q is the query domain, R is the response domain, and S is the score domain. The quality assessment method M maps each query q and response r pair to a score s in the score domain. The score domain is a closed set of real numbers, such as [0, 10], with larger numbers indicating better quality of response r. The quality assessment scores from different quality assessors are aggregated to form a collective consensus of the overall quality assessment. A potential risk of collective quality assessment is that lazy quality assessors may copy the scores of diligent quality assessors. These lazy quality assessors may be motivated to reduce computational costs and increase profits. To mitigate this potential issue, this embodiment introduces a two-phase consensus mechanism based on encryption. In the first phase, all quality assessors calculate and encrypt the scores. Only after a minimum number of scores are met do they release the public key to decrypt and reach consensus in the second phase. This mechanism prevents nodes from copying each other's scores and ensures that each quality assessor independently calculates the scores for the incoming query and response pair.

[0051] S3 uses manual quality assessment based on query-response pairs to obtain label quality assessment scores, compares the label quality assessment scores with the quality assessment scores generated by multiple quality assessors, assigns different reward values ​​to each quality assessor based on the comparison results, and optimizes each quality assessor.

[0052] In this embodiment, the blockchain network reaches consensus on the overall quality score of query-response pairs, and rewards for all participants are determined accordingly. Specifically, a manual approach is employed: first, experts perform quality assessments on query-response pairs to obtain label quality assessment scores. These label quality assessment scores are then compared with quality assessment scores generated by multiple quality assessors. Finally, different reward values ​​are assigned to each quality assessor based on the comparison results, allowing for optimization of each quality assessor. This incentive mechanism improves the robustness of PoQ, motivating participants to adhere to the protocol and preventing lazy or malicious behavior.

[0053] S4, encrypting the quality assessment scores higher than the predetermined threshold with the public key and writing them into the shared file, restoring all the quality assessment scores in the shared file, and calculating the average of all the quality assessment scores in the shared file as the final quality assessment score of the response.

[0054] like Figure 3 As shown in the first consensus phase, after receiving the query-response pair, the i-th quality assessor calculates the quality assessment score by running the specified cross-encoder model, i.e., s i =M(q,r). At the same time, the quality assessor generates a pair of public and private keys for encryption. Given the private key Pri and public key Pu i , the quality evaluator encrypts the score by the private key Pr i and writes the encrypted score into an immutable file accessible to every participant in the blockchain network.

[0055] In the second consensus phase, all the quality evaluation scores obtained are filtered by setting a threshold, and the quality evaluation scores higher than the set threshold are written into a shared file by the public key. Any participant can recover the quality evaluation score based on the shared file to obtain the decrypted score, and generate the average value of the decrypted score as the final quality evaluation score of the response r.

[0056] The operation cost of the above two-phase mechanism is low because the public key / private key generation and encryption process is highly efficient on modern CPUs or GPUs. Only minimal delay is generated when waiting for all quality evaluators to finally determine their quality scores.

[0057] S5, further optimizing the generative model based on the final quality evaluation score to complete the model deployment.

[0058] In the embodiment, the performance of the generative model in different aspects is understood according to the obtained final quality evaluation score. For example, it can be analyzed that the model performs better on which query type, performs worse on which query type, or which generation aspect (such as syntax, semantics, fluency, etc.) has a higher score and which has a lower score. According to the optimization direction, the parameters and structure of the model are adjusted, and the retrained model is used to generate responses again, and a new quality evaluation score is calculated, until the iterative optimization reaches a satisfactory performance level, and the generative model is deployed to the actual application scenario.

[0059] In summary, the generative model blockchain inference deployment method based on quality proof provided by the embodiment of the application can ensure the credibility of the output by directly evaluating the quality of the inference result of the generative model based on the new paradigm of quality proof, can greatly reduce the computational overhead and improve the processing efficiency by using the cross-encoder model based on BERT to evaluate the quality of the input and output, and can prevent lazy or malicious behavior and ensure the robustness of PoQ by using the incentive mechanism and two-phase consensus mechanism in the new paradigm of quality proof.

[0060] Based on the same inventive concept, as shown in Figure 4 the embodiment of the application also provides a generative model blockchain inference deployment system 400 based on quality proof, which comprises an inference module 410, an evaluation module 420, an incentive module 430, a consensus module 440 and an optimization and deployment module 450.

[0061] The inference module 410 is used to input a query into a generative model and output a response to obtain a query-response pair.​

[0062] The evaluation module 420 is configured to run the cross-encoder model by multiple quality evaluators in the blockchain respectively, perform quality evaluation on the response based on the query-response pair, and output multiple quality evaluation scores encrypted by a private key.

[0063] The incentive module 430 is configured to obtain a label quality evaluation score by performing quality evaluation on the query-response pair by an artificial manner, compare the label quality evaluation score with the quality evaluation scores generated by the multiple quality evaluators, and assign different reward values to the quality evaluators according to the comparison result, so as to optimize the quality evaluators.

[0064] The consensus module 440 is configured to write the quality evaluation scores higher than a predetermined threshold into a shared file after being encrypted by a public key, restore all the quality evaluation scores in the shared file, and take an average value of the quality evaluation scores in the shared file as a final quality evaluation score of the response.

[0065] The optimization and deployment module 450 is configured to further optimize the generative model based on the final quality evaluation score, and complete model deployment.

[0066] Based on the same inventive concept, the embodiments of the present application also provide an electronic device including a memory and a processor, the memory is configured to store a computer program, and the processor is configured to implement the quality proof based generative model blockchain reasoning and deployment method described above when executing the computer program.

[0067] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium, the storage medium stores a computer program, and when the computer program is executed by a computer, the quality proof based generative model blockchain reasoning and deployment method described above is implemented.

[0068] It should be noted that the quality proof based generative model blockchain reasoning and deployment system, the electronic device and the computer readable storage medium provided by the above embodiments all belong to the same inventive concept as the quality proof based generative model blockchain reasoning and deployment method, and the specific implementation process is described in detail in the quality proof based generative model blockchain reasoning and deployment method embodiments, which will not be repeated here.

[0069] The above specific embodiments have described the technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the most preferred embodiment of the present application, and is not used to limit the present application. Any modification, supplement and equivalent replacement made within the principle range of the present application shall be included in the protection range of the present application.

Claims

1. A method for deploying a generative model blockchain inference based on proof of quality, characterized in that, The method comprises the following steps: inputting a query into a generative model and outputting a response to obtain a query-response pair; running a cross-encoder model by multiple quality assessors in a blockchain respectively, performing quality assessment on the response based on the query-response pair, and outputting multiple quality assessment scores encrypted by private keys; writing the quality assessment scores higher than a predetermined threshold into a shared file after being encrypted by a public key, restoring all the quality assessment scores in the shared file, and taking the average of all the quality assessment scores in the shared file as the final quality assessment score of the response; further optimizing the generative model based on the final quality assessment score to complete model deployment.

2. The proof-of-stake based generative model blockchain inference deployment method according to claim 1, characterized in that, Further comprising: obtaining a label quality assessment score by manually performing quality assessment based on the query-response pair, comparing the label quality assessment score with the quality assessment scores generated by the multiple quality assessors, and giving different reward values to the quality assessors according to the comparison result to optimize the quality assessors.

3. The proof-of-stake based generative model blockchain inference deployment method according to claim 1 or 2, characterized in that, The quality assessors adopt a large language model.

4. The proof-of-stake based generative model blockchain inference deployment method of claim 1, wherein, The cross-encoder model adopts a BERT model.

5. The proof-of-stake based generative model blockchain inference deployment method of claim 1, wherein, The quality assessors perform quality assessment on the response according to the consistency between the response and the query.

6. The proof-of-stake based generative model blockchain inference deployment method of claim 1, wherein, The cross-encoder model supports efficient running on CPU or GPU.

7. A quality proof based generative model blockchain inference deployment system, implemented by using the quality proof based generative model blockchain inference deployment method of any one of claims 1-6. Comprising: an inference module, an evaluation module, a consensus module, and an optimization and deployment module; the inference module is used for inputting a query into a generative model and outputting a response to obtain a query-response pair; the evaluation module is used for running a cross-encoder model by multiple quality assessors in a blockchain respectively, performing quality assessment on the response based on the query-response pair, and outputting multiple quality assessment scores encrypted by private keys; the consensus module is used for writing the quality assessment scores higher than a predetermined threshold into a shared file after being encrypted by a public key, restoring all the quality assessment scores in the shared file, and taking the average of all the quality assessment scores in the shared file as the final quality assessment score of the response; the optimization and deployment module is used for further optimizing the generative model based on the final quality assessment score to complete model deployment.

8. The proof-of-stake based generative model blockchain inference deployment system of claim 7, wherein, Further comprising: an incentive module; the incentive module is used for obtaining a label quality assessment score by manually performing quality assessment based on the query-response pair, comparing the label quality assessment score with the quality assessment scores generated by the multiple quality assessors, and giving different reward values to the quality assessors according to the comparison result to optimize the quality assessors.

9. An electronic device comprising a memory and a processor, the memory for storing a computer program, characterized in that, The processor is used for realizing the quality proof-based generative model blockchain inference and deployment method of any one of claims 1-6 when executing the computer program.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is used for realizing the quality proof-based generative model blockchain inference and deployment method of any one of claims 1-6 when being executed by a computer.

Images