Cloud edge collaborative diffusion model reasoning method and system based on block chain and reinforcement learning

Abstract

The invention provides a cloud edge collaborative diffusion model reasoning method and system based on a block chain and reinforcement learning, and the method comprises the steps: obtaining a current text prompt word submitted by a user in a block chain network composed of a cloud server and an edge server, and calling a semantic matching model through an intelligent contract, and judging whether a historical intermediate result can be reused or not; obtaining a server environment state, generating a collaborative reasoning strategy by using a pre-trained multi-agent attention actor-commentator model, and determining cloud edge denoising step number distribution; if the image cannot be reused, the cloud server executes partial denoising to generate an intermediate result and stores the intermediate result to the block chain, and the edge server continues to complete residual denoising to generate a final image; and if the edge server can be reused, the edge server directly completes denoising based on the historical intermediate result. According to the method, historical intermediate results can be effectively reused, dynamic intelligent task allocation is realized, the diffusion model reasoning efficiency and the image generation quality are improved, and the credibility and the traceability of a distributed reasoning process are guaranteed.

Description

TECHNICAL FIELD

[0001] The application relates to the field of artificial intelligence and information technology, and in particular to a cloud-edge collaborative diffusion model reasoning method and system based on a blockchain and reinforcement learning. BACKGROUND

[0002] In recent years, generative artificial intelligence (GenAI) has developed rapidly, and in particular, diffusion models represented by diffusion models have attracted widespread attention due to their excellent image generation quality and wide range of application scenarios. Diffusion models can generate images with extremely high visual fidelity and rich details by simulating the physical process of gradual transformation of data between noise and clear structure, and their performance far exceeds that of previous generative adversarial networks (GAN) and other models. However, behind this outstanding performance is a huge computational cost. Diffusion models usually need to perform hundreds or even thousands of iterations of denoising processes, and each iteration involves complex deep neural network forward reasoning, resulting in high reasoning delay and huge computational resource consumption, which poses a serious challenge to their deployment and application in edge computing environments with high real-time requirements and limited resources.

[0003] To address this challenge, the industry naturally turns its attention to the cloud-edge collaborative computing paradigm. This paradigm aims to reasonably allocate computationally intensive tasks between powerful cloud servers and edge servers close to data sources, in order to balance the massive computing power of cloud computing and the low-latency advantage of edge computing. Existing technical solutions mainly focus on task offloading and resource scheduling. For example, some research proposes a user-centric interactive AIGC service management method, which realizes efficient collaborative deployment through a distributed diffusion model framework; some research designs an edge user collaborative reasoning framework to optimize service latency and resource consumption and improve user experience quality; and some research explores the use of a distributed collaborative reasoning mechanism between roadside units and vehicles in a vehicle network to promote the generation of diverse content. These studies have made positive progress in optimizing task allocation, reducing network transmission overhead, and improving edge resource utilization.

[0004] However, the existing solutions still have obvious limitations and defects. First, they mostly focus on macro task offloading or computing resource scheduling, generally ignoring the reuse value of intermediate results generated by step-by-step denoising in the diffusion model inference process. Second, in a dynamic and heterogeneous cloud-edge environment, how to finely and adaptively divide the number of denoising steps for each task between the cloud and the edge, and simultaneously consider the system real-time load, network conditions and quality constraints of the final generated content when making decisions, the existing solutions lack an intelligent decision-making mechanism that can perform long-term and joint optimization. Finally, in a distributed and multi-participant collaborative inference environment, how to ensure the traceability of the inference process, the credibility of task execution and the automatic and tamper-proof execution of collaborative logic, the existing technology fails to provide systematic security guarantees. The above defects restrict the widespread application of diffusion models in edge-side efficient, high-quality and trusted inference. SUMMARY

[0005] In view of this, the embodiments of the present application provide a cloud-edge collaborative diffusion model inference method and system based on blockchain and reinforcement learning to solve the problems of ignoring the reuse value of intermediate results in inference, insufficient intelligence of dynamic task offloading decision and weak trust guarantee of distributed inference process in existing cloud-edge collaborative solutions.

[0006] In one aspect, the present application provides a cloud-edge collaborative diffusion model inference method based on blockchain and reinforcement learning, which is executed in a blockchain network composed of a cloud server and at least one edge server as nodes, and the method comprises: obtaining a user-submitted current text prompt, calling a pre-trained semantic matching model through an intelligent contract pre-deployed on the blockchain to calculate the semantic similarity between the current text prompt and the historical text prompts stored in the blockchain, and if the semantic similarity is greater than a preset threshold, it is determined that the corresponding historical intermediate result can be reused, otherwise it cannot be reused; wherein the semantic matching model adopts a twin network architecture, extracts semantic feature vectors of the current prompt and the historical text prompts through shared weight subnetworks respectively, and calculates the similarity between the feature vectors, and the historical intermediate result refers to the diffusion model hidden variable obtained after the cloud server performs part of the denoising steps for the historical text prompt; Obtaining the environment state of the cloud server and the edge server, generating a collaborative inference strategy for the current inference task through a pre-trained multi-agent attention actor-critic model based on a diffusion model, the collaborative inference strategy including the number of denoising steps performed by the cloud server and the number of denoising steps performed by the edge server; wherein the multi-agent attention actor-critic model is constructed with the environment state of the cloud server and the edge server as the state space, the number of denoising steps of the cloud server and the edge server as the action space, generates an action probability distribution through a denoising diffusion process, and aggregates multi-agent information using an attention mechanism, and the reward function is composed of the weighted sum of the total system processing delay and the generated image quality, wherein the environment state includes task queue length, system utilization, data transmission time and computing power. If the historical intermediate result cannot be reused: obtaining the number of denoising steps performed by the cloud server and the number of denoising steps performed by the edge server according to the collaborative inference strategy, performing denoising on the initial noise of the current text prompt by the cloud server according to the corresponding number of denoising steps, generating a current intermediate result and storing it to the blockchain, and then performing denoising on the current intermediate result by the edge server to generate a final image. If the historical intermediate result can be reused, obtain the historical intermediate result from the blockchain, obtain the number of denoising steps of the edge server according to the collaborative inference strategy, and perform denoising on the historical intermediate result by the edge server to generate a final image.

[0007] In some embodiments of the present application, the smart contract includes a node management contract, a prompt word management contract and a prompt word matching contract. The node management contract is used to mark the node type and set the corresponding permissions and responsibilities when a new node joins the blockchain network. The prompt word management contract is used to record, deduplicate and verify the legality of the prompt word submitted by the user, and add a mark to the prompt word with existing historical inference data. The prompt word matching contract is used to call the semantic matching model to perform the operation of calculating the semantic similarity between the current text prompt word and the historical text prompt word, and determine whether the corresponding historical intermediate result can be reused according to the result.

[0008] In some embodiments of the present application, the semantic matching model includes a text encoder and a twin network, the text encoder includes a tokenizer and a Transformer module, and the method further includes extracting the initial semantic feature vector of the current prompt word and the historical text prompt word using the text encoder, including: The tokenizer is used to respectively decompose the current prompt word and the historical text prompt word into a word sequence, embed each word into a word vector, and form a word vector sequence; After adding the word vector sequence to the learnable position encoding, the word vector sequence is input into the Transformer module; The word vector sequence is encoded by the multi-head attention layer and the feedforward neural network stacked in the Transformer module, and the semantic feature vector of the current prompt word and the initial semantic feature vector of the historical text prompt word are output.

[0009] In some embodiments of the present application, the twin network includes two subnetworks sharing weights, each subnetwork including a convolutional network and a multilayer perceptron, and the semantic similarity between the current text prompt word and the historical text prompt word is calculated, including: The semantic feature vector of the current prompt word and the initial semantic feature vector of the historical text prompt word are input into two subnetworks respectively, in each subnetwork, the local semantic pattern feature is extracted by the convolutional network, and then the local semantic pattern feature is nonlinearly transformed and reduced in dimension by the multilayer perceptron, to obtain the final semantic feature vector of the current prompt word and the historical text prompt word; The cosine similarity between the final semantic feature vectors of the current prompt word and the historical text prompt word is calculated, satisfying the following formula: ; Wherein, represents the semantic similarity between the current prompt word and the historical text prompt word ; represents the final semantic feature vector of the current prompt word; represents the final semantic feature vector of the historical text prompt word; represents the transpose; represents the L2 norm.

[0010] In some embodiments of the present application, the multi-agent attention actor-critic model generates an action probability distribution through a denoising diffusion process, including: An initial noise is sampled from a standard Gaussian distribution as an initial action probability distribution; Starting from the initial action probability distribution, the actor network in the multi-agent attention actor-critic model is iterated for a preset number of steps to denoise, and a clear action probability distribution is gradually recovered, and the denoising steps satisfy the following formula: ; Wherein, represents the time slot t . when step de-noising is performed, the action probability distribution of the agent . denotes the noise retention rate of the first step . denotes the cumulative noise retention rate from the first step to the nth step . denotes the noise predicted by the actor network . denotes the environment state observed by the agent at the time slot . denotes the variance adjustment coefficient . the clear action probability distribution is converted into the final action selection distribution by using a Softmax function, and the conversion process satisfies the following formula: . wherein denotes the probability vector of each action made in the environment state at the time slot . denotes the dimension value corresponding to the action in the clear action probability distribution of the agent at the time slot . and denote the index of the action . denotes the natural constant . denotes the summation.

[0011] In some embodiments of the present application, the total processing delay of the system includes the total delay of the cloud server and the total delay of the edge server, and satisfies the following formula: . wherein denotes the total processing delay of the system for performing the inference task at the time slot . denotes the total delay of the cloud server, including the cloud computing processing delay, the task queuing delay of the cloud server, and the data transmission delay of the cloud server to the edge server for transmitting the intermediate result . denotes the total delay of the edge server, including the edge computing processing delay, the task queuing delay of the edge server, and the data transmission delay of the edge server to the user for transmitting the final image

[0012] In some embodiments of the present application, the generated image quality is evaluated by three indicators of structural similarity index, peak signal-to-noise ratio and Fréchet Inception Distance The structural similarity index is used to measure the overall similarity of the generated image and the real image in multiple dimensions, including at least brightness, contrast and structure. The peak signal-to-noise ratio is used to quantify the pixel-level error between the generated image and the real image. The Frechet Inception Distance is used to measure the distribution difference between the generated image and the real image in the deep feature space.

[0013] In some embodiments of the present application, the reward function satisfies the following formula: ; Wherein, represents the reward value obtained by all agents at time slot; and represent the weight coefficients; represents the set of inference tasks, is one of the inference tasks; represents the number of edge servers, is the index of the edge server; represents the total latency of the cloud server; represents the total latency of the edge server; represents the structural similarity index of the generated image of the inference task after normalization; represents the peak signal-to-noise ratio of the generated image of the inference task after normalization; represents the Frechet Inception Distance of the generated image of the inference task after normalization.

[0014] In some embodiments of the present application, the method further comprises training the multi-agent attention actor-critic model through a maximum entropy reinforcement learning framework, and the training step comprises: updating each critic network parameter by minimizing the joint regression loss function of all agent critic networks; wherein the critic network of each agent uses an attention mechanism to dynamically calculate the relevance weight between its own and other agent environmental state and action information, and aggregates the information of other agents according to the relevance weight to cooperatively estimate the action value function; wherein the joint regression loss function satisfies the following formula: ; Wherein, represents the total loss value of all agent critic network parameters ; represents the index of the agent, ; denotes a batch of experience data randomly sampled from an experience replay buffer denotes a mathematical expectation of the loss, denotes a state, an action, a reward and a state of a next time slot, respectively; denotes a critic network of an agent at a time slot denotes a predicted Q value of an action performed by the critic network of the agent at a state denotes a target Q value; updating the actor network parameters by maximizing a weighted sum of an expected return and a policy entropy; wherein the actor network of each agent updates policy parameters by a policy gradient method according to a value guidance provided by the critic network and a multi-agent baseline related to policy of other agents.

[0015] In another aspect, the present application also provides a cloud-edge collaborative diffusion model inference system based on blockchain and reinforcement learning, comprising a processor, a memory and computer programs / instructions stored on the memory, wherein the processor is configured to execute the computer programs / instructions, and when the computer programs / instructions are executed, the system implements the steps of the method according to any one of the above aspects.

[0016] The present application provides a cloud-edge collaborative diffusion model inference method and system based on blockchain and reinforcement learning, which realizes a qualitative and quantitative leap in diffusion model inference in a cloud-edge collaborative environment through deep integration of blockchain and reinforcement learning. Compared with the prior art, the present application can: significantly improve inference efficiency and response speed: by intelligently reusing historical intermediate results, a large amount of repeated calculations is avoided; combined with a dynamic task offloading strategy based on multi-agent reinforcement learning, the fine collaboration of the denoising step at the cloud and edge ends is realized, effectively reducing the total processing delay of the system and meeting the requirements of application scenarios with high real-time requirements.

[0017] generate high-quality images in a complex environment: the collaborative inference strategy used synchronously considers system real-time load, network conditions and multi-dimensional image quality indicators (such as SSIM, PSNR and FID) when making decisions, which can adaptively adjust the strategy in a dynamic and heterogeneous resource environment to ensure high fidelity and visual quality of generated images.

[0018] build a reliable distributed inference ecosystem: using the characteristics of blockchain technology such as non-tamperability and traceability, the inference process, intermediate results and collaborative logic are solidified, and the intelligent contract is automatically executed, effectively preventing interference and tampering by malicious nodes, and providing a solid security and trust foundation for multi-participant collaboration.

[0019] ​​​​In summary, the application not only overcomes the efficiency and quality bottleneck of the diffusion model deployed on the edge side, but also constructs a new paradigm of efficient, high-quality and reliable cloud-edge collaborative reasoning from the system level.

[0020] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following detailed description and drawings in which:

[0021] Those skilled in the art will appreciate that the objects and advantages of the application can be implemented without regard to the specific details of the following description and that the application can be implemented in various ways, as will be apparent to persons skilled in the art from the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0022] The drawings described herein are intended to provide a further understanding of the application, and constitute a part of this application. The drawings are not intended to limit the application, and are provided for explanation only. In the drawings: Figure 1 A step schematic diagram of a cloud-edge collaborative diffusion model reasoning method based on blockchain and reinforcement learning in an embodiment of the application.

[0023] Figure 2 A cloud-edge collaborative diffusion model reasoning architecture diagram based on blockchain and reinforcement learning in an embodiment of the application.

[0024] Figure 3 A structure principle diagram of a semantic matching model in an embodiment of the application.

[0025] Figure 4 A structure principle diagram of a multi-agent attention actor-critic model in an embodiment of the application. DETAILED DESCRIPTION

[0026] To make the objects, technical solutions and advantages of the application clearer, the following further describes the application in conjunction with the embodiments and drawings. Here, the illustrative embodiments of the application and their descriptions are used to explain the application, but are not limiting to the application.

[0027] It should be noted that, in order to avoid obscuring the application due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the application are shown in the drawings, and other details not closely related to the application are omitted.

[0028] It should be emphasized that the term "comprises / comprising" as used herein means the presence of the stated features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0029] It is also noted herein that, if it is not otherwise indicated, the term "connection" can refer not only to a direct connection, but also to an indirect connection via an intermediate object.

[0030] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0031] It is emphasized here that the step marks mentioned hereinafter are not a limitation on the order of the steps, but it should be understood that the steps can be performed in the order mentioned in the embodiments, or in an order different from the embodiments, or several steps can be performed simultaneously.

[0032] In order to solve the problems of ignoring the reuse value of inference intermediate results, the lack of intelligent level of dynamic task offloading decision, and the weak trustworthiness of distributed inference process in the existing cloud-edge collaboration scheme, the present application provides a cloud-edge collaboration diffusion model inference method based on blockchain and reinforcement learning. The method is executed in a blockchain network, as shown in Figure 1 The method comprises the following steps S101-S104: Step S101: Obtain the current text prompt word submitted by the user, and calculate the semantic similarity between the current text prompt word and the historical text prompt word stored in the blockchain by calling the pre-trained semantic matching model through the smart contract pre-deployed on the blockchain. If the semantic similarity is greater than a preset threshold, it is determined that the corresponding historical intermediate result can be reused, otherwise it cannot be reused. The semantic matching model uses a twin network architecture to extract semantic feature vectors of the current prompt word and the historical text prompt word through shared weight subnetworks, and calculates the similarity between the two feature vectors. The historical intermediate result refers to the diffusion model hidden variable obtained after the cloud server performs a certain number of denoising steps for the historical text prompt word.

[0033] Step S102: Obtain the environment state of the current cloud server and edge server, and generate a collaborative inference strategy for the current inference task by using the pre-trained multi-agent attention actor-critic model based on the diffusion model. The collaborative inference strategy includes the number of denoising steps performed by the cloud server and the number of denoising steps performed by the edge server for the current inference task. The multi-agent attention actor-critic model is constructed with the environment state of the cloud server and the edge server as the state space, and the denoising step distribution strategy of the cloud server and the edge server as the action space. The action probability distribution is generated through the denoising diffusion process, and the multi-agent information is aggregated using the attention mechanism. The reward function is composed of the weighted sum of the total processing delay and the generated image quality.

[0034] Step S103: If the historical intermediate result cannot be reused, the number of denoising steps performed by the cloud server and the number of denoising steps performed by the edge server are obtained according to the collaborative reasoning strategy, the initial noise of the current text prompt is denoised by the cloud server according to the corresponding number of denoising steps, the current intermediate result is generated and stored in the blockchain, and the edge server is used to denoise the current intermediate result to generate the final image.

[0035] Step S104: If the historical intermediate result can be reused, the historical intermediate result is obtained from the blockchain, the number of denoising steps of the edge server is obtained according to the collaborative reasoning strategy, and the edge server is used to denoise the historical intermediate result to generate the final image.

[0036] As shown in FIG. 1, it is a reasoning architecture diagram of a cloud-edge collaborative diffusion model based on a blockchain and reinforcement learning. Figure 2

[0037] The cloud server is denoted as CS, which is composed of a large data center, and the edge server is denoted as ES, which is multiple and distributed. The cloud server and the edge server are both nodes in the blockchain network and jointly constitute the blockchain platform.

[0038] In some embodiments, the cloud server is a full node, responsible for recording all historical events related to reasoning on the blockchain. The edge server is a light node, only recording the reasoning process and its corresponding results executed on the edge layer.

[0039] A smart contract is pre-deployed on the blockchain, and all subsequent operations are automatically executed by the smart contract, thereby effectively avoiding human intervention or other external factors from interfering with the reasoning process and improving the credibility and execution consistency of the system.

[0040] In some embodiments, the smart contract includes a node management contract, a prompt management contract, and a prompt matching contract.

[0041] The node management contract is used to mark the node type and set the corresponding permissions and responsibilities of the new node when it joins the blockchain network. The node type includes full nodes and light nodes, and the corresponding permissions and responsibilities are as described above.

[0042] The prompt management contract is used to record, deduplicate, and verify the legality of the prompt submitted by the user, and add a mark to the prompt with existing historical reasoning data.

[0043] The prompt matching contract is used to call the semantic matching model to perform the operation of calculating the semantic similarity between the current text prompt and the historical text prompt, and determine whether the corresponding historical intermediate result can be reused according to the result.

[0044] After the reasoning framework is constructed, the reasoning process is further described below.​

[0045] As shown in Figure 2 , the cloud server and the edge server cooperate to complete the step-by-step denoising inference task of the diffusion model. Under this inference architecture, the inference task is distributedly executed, the cloud server is responsible for the processing of the front inference steps and the generation of the intermediate results, and the edge server continues to complete the subsequent denoising generation task on this basis. Specifically: ① part represents that the user submits a text prompt word to initiate an inference task request. ② part represents that the smart contract calls the semantic matching model to determine whether the current inference task can reuse the historical results. ③ part represents that the cloud server performs global inference on the text prompt word to generate intermediate results. ④ part represents that the cloud server transmits the intermediate results to the edge server. ⑤ part represents that the edge server performs local inference based on the intermediate results provided by the cloud server to obtain the final image.

[0046] In ① and ② parts (corresponding to step S101), the user end submits a text prompt word to the blockchain platform through a smart device such as a mobile phone, computer, tablet, watch, etc., to initiate an inference task request. After receiving the inference task request, the blockchain platform calls the semantic matching model through the smart contract (prompt word matching contract) to determine whether the inference task can reuse the historical intermediate results obtained after the cloud server executed part of the denoising steps for the historical text prompt word.

[0047] If it cannot be reused, ③, ④ and ⑤ parts are executed in turn, corresponding to step S103; if it can be reused, ④ and ⑤ parts are directly executed, corresponding to step S104.

[0048] For example, let the total denoising step number be , the denoising step number executed by the cloud server be , and the denoising step number executed by the edge server be , then .

[0049] In ③ part, the cloud server receives the current inference task request and executes steps of denoising to generate intermediate results. The hash value corresponding to the intermediate results is stored on the blockchain, and the original data of the intermediate results is stored in the form of encryption in the off-chain interplanetary file system (IPFS).

[0050] In ④ part, the cloud server transmits the original data of the intermediate results to the edge server through wireless transmission.

[0051] In ⑤ part, the edge server receives the intermediate results and executes steps of denoising to generate the final image.

[0052] In some embodiments, as shown in section ⑤A, if the cloud server and the edge server perform a denoising step on the same text prompt word, a high-quality image can be generated. As shown in section ⑤B, if the denoising step is not performed on the same text prompt word, but the smart contract determines that the two text prompt words have a high semantic similarity, the quality of the generated image highly depends on the workload of the cloud server. The size. If If the noise level is relatively low, meaning the cloud server only performs basic noise reduction, then the edge server can generate a high-quality image for the new text prompts; if... If the noise level is too high, the correct image content cannot be generated. Therefore, the allocation of noise reduction steps between cloud servers and edge servers is crucial to the generation of high-quality images.

[0053] Therefore, this invention not only determines whether intermediate results can be reused, but also designs a multi-agent attention actor-critic model to determine the cloud-edge collaborative reasoning strategy.

[0054] The following provides further explanation of steps S101 to S104.

[0055] In step S101, as described above, after receiving a new inference task request, the blockchain platform calls the semantic matching model through a smart contract to calculate the semantic similarity between the current text prompt and the historical text prompt, thereby determining whether the historical intermediate results can be reused.

[0056] In some embodiments, such as Figure 3 As shown, the semantic matching model consists of two parts: a text encoder and a Siamese network. For example, the text encoder uses the CLIP text encoder. The text encoder extracts the initial semantic feature vectors of the current prompt word and historical text prompt words. The Siamese network further extracts and encodes the semantic feature vectors of the current prompt word and historical text prompt words, and finally calculates the cosine similarity between the two to determine the degree of semantic similarity.

[0057] The text encoder includes a tokenizer and a Transformer module. For example, the tokenizer uses a Byte Pair Encoding (BPE) tokenizer.

[0058] In the word segmenter, both the current prompt word and the historical text prompt words are decomposed into word element sequences. Each word element is embedded into a word vector using an embedding lookup table, as shown in formula (1): ; (1) in, Indicates the first Word vectors; Indicates an embedded function; Indicates the first wordpiece; denotes a real vector space of dimension

[0059] Thus, the word vector sequence is obtained, as shown in equation (2): ; (2) wherein, denotes the word vector sequence; denotes the word vector, is the total length of the sequence, and an exemplary ; denotes a real vector space of dimension

[0060] Since the Transformer itself does not have built-in sequential sense, the word vector sequence is added to the learnable position encoding before being input into the Transformer module for processing.

[0061] The Transformer module is stacked by multiple layers, and each layer includes two core sub-layers: a multi-head attention layer and a feed-forward neural network.

[0062] The multi-head attention layer is used to allow each word (or position) in the sequence to pay attention to all other words in the sequence, thereby updating its own representation according to the global context.

[0063] The input to the current layer is linearly transformed by three different weight matrices respectively to generate query, key and value matrices, and then attention is calculated, as shown in equation (3): ; (3) wherein, denotes the calculation function of the scaled dot-product attention; denotes the query matrix; denotes the key matrix; denotes the value matrix; denotes the transpose; denotes the dimension of the key vector; denotes the normalization exponential function.

[0064] The feed-forward neural network is used to perform nonlinear transformation and feature integration on the output of the attention layer to increase the expression ability of the model. Specifically, a two-layer fully connected network is used, and a GELU activation function is used in the middle to introduce nonlinearity, and each position in the sequence is independently and identically calculated, as shown in equation (4): ; (4) wherein, denotes the feed-forward neural network function; denotes input matrix of the layer, output of the multi-head attention layer; denotes weight matrix of the first layer; denotes transpose; denotes bias vector of the first layer; denotes weight matrix of the second layer; denotes bias vector of the second layer; denotes Gaussian error linear unit activation function.

[0065] After the above two sub-layers are processed, the text encoder finally outputs the initial semantic feature vectors of the current prompt and the historical text prompt, as shown in formulas (5) and (6) respectively: ; (5) ; (6) wherein, denotes the initial semantic feature vector of the current prompt; denotes the initial semantic feature vector of the historical text prompt; denotes the processing of the text encoder; denotes the current prompt; denotes the historical text prompt.

[0066] The twin network is used to map the high-dimensional semantic vector generated by the text encoder, which may contain redundant information, into a low-dimensional dense vector that is more suitable for direct similarity comparison. The twin network includes two sub-networks that share weights, and each sub-network further includes a convolution network and a multi-layer perceptron.

[0067] In the convolution network, a sliding window is used to extract locally relevant semantic patterns in the sequence length direction (i.e., across several adjacent word units) to capture semantic units like phrases, making the semantic representation more robust.

[0068] After the multi-layer perceptron (MLP) is further used to perform nonlinear transformation and high compression on the features, the final semantic feature vectors of the current prompt and the historical text prompt are obtained, as shown in formulas (7) and (8): ; (7) ; (8) wherein, denotes the final semantic feature vector of the current prompt; denotes the final semantic feature vector of the historical text prompt; denotes the multi-layer perceptron processing; denotes one-dimensional convolution processing.

[0069] Finally, the cosine similarity between the current prompt and the final semantic feature vector of the historical text prompt is calculated to obtain the semantic similarity, as shown in formula (9): (9) wherein, represents the semantic similarity between the current prompt and the historical text prompt . represents the transpose; represents the L2 norm.

[0070] If the semantic similarity is greater than a preset threshold, it is determined that the corresponding historical intermediate result can be reused, otherwise it cannot be reused.

[0071] In some embodiments, the twin network is trained by using a contrast learning method, and different constraints are applied to the loss function according to different sample labels: For a positive sample pair, by minimizing the square of the similarity between the semantic vectors, the model is prompted to learn to narrow the representation distance.

[0072] For a negative sample pair, by minimizing the square of the similarity exceeding the preset boundary value , the model is prompted to learn to pull the representation distance apart by at least interval.

[0073] The loss function is shown in formula (10): (10) wherein, represents the contrast loss value; represents the semantic similarity calculated by the semantic matching model described above; represents the true label, which is a binary signal, and an example is represents a positive sample pair, meaning that the semantics are similar, and the intermediate result can be reused, represents a negative sample pair, meaning that the semantics are not similar, and the intermediate result cannot be reused; represents a preset boundary value or interval, which is a preset hyperparameter; represents the maximum function.

[0074] After determining whether the intermediate result can be reused in step S102, the pre-trained multi-agent attention actor-critic model based on the diffusion model is used to generate a collaborative reasoning strategy for the current reasoning task, that is, to determine the number of denoising steps performed by the cloud server and the number of denoising steps performed by the edge server.

[0075] First, the principle of generating an optimal collaborative reasoning strategy for the current reasoning task by the multi-agent attention actor-critic model based on the diffusion model is described.

[0076] In the present application, the best decision (i.e., collaborative inference strategy) of inference task offloading is generated by using a denoising diffusion probability model (DDPM), which is solved in combination with multi-agent reinforcement learning. For example, the task offloading decision of the cloud server and the edge server is defined as , then the probability distribution of each decision being selected is .

[0077] The DDPM includes a forward process and a reverse process. The forward process is to add Gaussian noise to until the data becomes completely noisy, and the reverse process is to gradually denoise and recover from the beginning of the Gaussian noise.

[0078] The forward process and the reverse process are further described below.

[0079] It should be noted that the forward process is not a step of model running (inference / decision making), but only a process for preparing training data in the training stage.

[0080] In the forward process, at each step, a small amount of Gaussian noise is added based on the result of the previous step to obtain .

[0081] In some embodiments, the mean of the new data obtained at each step is defined as , and the variance is , which indicates that most of the original information is preserved. Wherein, is a preset very small value, which increases with the increase of the step, that is, the amount of noise added at each step gradually increases; represents the unit matrix.

[0082] For easy sampling and derivation, the Gaussian distribution sampling is rewritten as a deterministic calculation formula, as shown in equation (11): ; (11) Wherein, represents the sampled Gaussian noise in the standard normal distribution.

[0083] The entire forward process can be represented as a Markov chain, as shown in equation (12): ; (12) Wherein, represents the probability distribution of the entire sequence from step 0 to step ; represents a continuous multiplication.

[0084] By mathematical induction, formula (11) is applied successively to obtain formula (13): ; (13) wherein, denotes the noise retention rate of the th step, denotes the cumulative noise retention rate from the 1st step to the th step.

[0085] Based on formula (12), the noise data of the th step is actually a linear combination of the original data and the standard noise . The proportion of the combination is determined by . As the step increases, tends to 0, so tends to pure noise .

[0086] In the reverse process, based on formula (12), the relationship between and can be calculated, as shown in formula (14): ; (14) wherein, denotes the new noise source of each denoising step, which is independent of the noise distribution of the forward process.

[0087] In order to infer the probability distribution step by step through the denoising steps, convert and into Gaussian distribution, as shown in formula (15): ; (15) wherein, denotes the Gaussian distribution; denotes the mean vector of the reverse process Gaussian distribution, which is predicted by the neural network, denotes the actor network parameters of the agent ; denotes the environment state observed by the agent at the th time slot; denotes the variance of the reverse process Gaussian distribution; denotes the unit matrix.

[0088] wherein, the calculation formula of the variance of the reverse process Gaussian distribution is shown in formula (16): ; (16) The mean vector of the reverse process Gaussian distribution is calculated by the Bayes formula, as shown in formula (17): (17) Substitute formula (14) into formula (17), and use the deep reinforcement learning parameters to learn, as shown in formula (18): (18) According to formula (15), the relationship between and is further obtained by the reparameterization technique, as shown in formula (19): (19) wherein represents the noise predicted by the actor network of the intelligent agent; represents the variance adjustment coefficient; represents the random noise. According to the update rule of the reverse process in formula (19), after denoising at the

[0089] step, the is obtained.

[0090] Finally, the Softmax function is used to convert into the final decision probability distribution, as shown in formula (20): (20) wherein represents the probability vector of each action made under the environment state at the time slot; represents the dimension value corresponding to the action in the clarified action probability distribution of the intelligent agent at the time slot; and represent the index of the action; represents the natural constant; represents the summation. Through the above DDPM process, the probability distribution of the optimal decision

[0091] is derived, and the optimal decision probability distribution (action selection distribution) is output.

[0092] The construction of the multi-agent attention actor-critic model is further described below.

[0093] ​​In the present application, the decision-making process is described as a Markov Decision Process (MDP) process, which includes a state space, an action space and a reward function, represented by a tuple , wherein, is the state transition probability.

[0094] State space: The state is composed of a set of specific discrete variables, which can be perceived by the intelligent agent (cloud server and edge server) from the interaction in the environment and make decisions.

[0095] The state space includes the environmental state information required for decision-making. Specifically, the state of the cloud server includes the task queue length of the cloud server, the system utilization rate of the cloud server, the data transmission time from the cloud server to the edge server, the current available floating point computing power of the cloud server and the current backlog queue task, etc. as shown in equation (21): ; (21) wherein, represents the task queue length of the cloud server; represents the system utilization rate of the cloud server, represents the data transmission time from the cloud server to the edge server; represents the current available floating point computing power of the cloud server; represents the current backlog queue task of the cloud server.

[0096] The state of the edge server includes the task queue length of the edge server, the system utilization rate of the edge server, the data transmission time from the edge server to the user, the current available floating point computing power and the current backlog queue task, etc. as shown in equation (22): ; (22) wherein, represents the task queue length of the th edge server; represents the system utilization rate of the th edge server, represents the data transmission time from the th edge server to the th user; represents the current available floating point computing power of the th edge server; represents the current backlog queue task of the th edge server.

[0097] Then the state space, i.e. the state of all intelligent agents, is as shown in equation (23): ; (23) The calculation method of the data in the cloud server and edge server environment state is described in detail below.

[0098] Action space: the action space is defined as the set of all possible decisions that the agent can make. According to the current state, the agent will make a current action according to the probability distribution Make a current action As shown in equation (24): ; (24) Wherein, represents the action selected and executed by the agent at time ; represents the action that makes the function obtain the maximum value , , represents that the task is processed by the cloud server, represents that the task is unloaded to the 1st, 2nd,..., Ith edge server for processing.

[0099] Then the action space, that is, the set of actions performed by all agents, is shown in equation (25): ; (25) In the present application, the research is the cooperative work of multiple agents, therefore, the objective function is constructed based on the total system processing delay and the generated image quality, and the reward function is constructed with the final purpose of minimizing the objective function.

[0100] The total system processing delay and the generated image quality are described in detail below, and the calculation method of each state data mentioned in the state space part is also described here.

[0101] For the total system processing delay, it is described from two aspects of communication and calculation.

[0102] In the present application, there is communication between the cloud server (CS) and the edge server (ESs) and communication between the edge server (ESs) and the user (User), which are defined as the C2E communication model and the E2U communication model respectively.

[0103] For the C2E communication model, the maximum wireless transmission rate from the cloud server to the edge server is shown in equation (26): ; (26) Wherein, represents the maximum wireless transmission rate from the cloud server to the edge server; represents the channel bandwidth; represents the transmission power of the cloud server; represents the distance from the cloud server to the edge server Channel gain; This indicates the noise power at the receiving end.

[0104] For the E2U communication model, the maximum wireless transmission rate from the edge server to the user is shown in formula (27): ;(27) in, This indicates the maximum wireless transmission rate from the edge server to the user; Indicates channel bandwidth; Indicates the first The transmit power of each edge server; Indicates from the first The edge server to the first Channel gain for each user; This indicates the noise power at the receiving end.

[0105] It should be noted that since the propagation delay is in the nanosecond to microsecond range and the transmission delay is in the millisecond to second range, and the propagation delay is very small and negligible compared to the inference delay of the diffusion model, this invention only considers the transmission delay and queuing delay, and does not consider the propagation delay.

[0106] The following explains queuing delay and transmission delay.

[0107] Cloud servers consist of data centers and have multiple parallel outbound links for data processing; therefore, the computing queuing latency of cloud servers is... The probability that the service console in the cloud server is idle is shown in formula (28): ; (28) in, This indicates the probability that the service console in the cloud server is idle. Indicates the total number of service counters; This represents the factorial operation; This indicates the task arrival rate of the cloud server; Indicates the first Service rate of each service desk; This indicates the system utilization rate.

[0108] The system utilization rate is calculated as shown in formula (29): ; (29) The average queuing length of tasks on the cloud server is shown in formula (30): ; (30) Then the reasoning task The average waiting time is shown in formula (31): ; (31) Edge servers are typically single-server systems, so the computation queuing latency of edge servers follows an M / M / 1 model. The average queuing length of tasks is shown in formula (32). ; (32) in, Indicates the first Task queuing length for each edge server; Indicates the first System utilization of edge servers.

[0109] Then the reasoning task The average waiting time is shown in formula (33): ; (33) Regarding transmission delay: The data size corresponding to the image obtained in step t of the denoising process is shown in formula (34): ; (34) in, This indicates the first step of the noise reduction process. The data size of the image obtained from the step; Indicates the height of the image; Indicates the width of the image; The number of bits representing the image pixels; This represents the number of channels in the image. Each pixel contains an independent numerical value, representing different feature dimensions.

[0110] The calculation formula for the time required for data transmission from the cloud server to the edge server is shown in formula (35): ; (35) in, Representational reasoning task Intermediate results are transmitted from the cloud server to the edge server. Required transmission latency; This indicates the number of denoising steps performed by the cloud server; Indicates that the cloud server is executing The size of the intermediate results generated after the noise reduction step; This indicates the maximum wireless transmission rate from the cloud server to the edge server; This represents the total number of denoising steps.

[0111] when When the diffusion model denoising and image generation process is not performed on the cloud server, there is no data transmission from the cloud server to the edge server.

[0112] The formula for calculating the time required for data transmission from the edge server to the user is shown in formula (36): ; (36) in, Representational reasoning task The final result from the first The edge server to the first The transmission latency required by each user; This indicates the number of denoising steps performed by the edge server; Indicates the data size of the final generated image; This indicates the maximum wireless transmission rate from the edge server to the user.

[0113] when When the diffusion model denoising process for generating the image is not performed on the edge server, there is no data transmission from the edge server to the user. The final image is then generated by the cloud server and transmitted to the user, with the transmission delay shown in formula (37). ; (37) in, Representational reasoning task The results from the cloud server to the first The transmission latency required by each user; Indicates that the cloud server is executing The size of the intermediate results generated after the noise reduction step; This indicates the maximum wireless transmission rate from the cloud server to the user.

[0114] The above is an explanation of the communication aspect; the following explanation will focus on the computing aspect.

[0115] In the diffusion model network structure, the formula for calculating the number of floating-point operations in each layer is shown in formula (38): ; (38) in, Indicates the first The number of floating-point operations required for a single forward propagation of a network layer; Indicates the first The number of input channels in each convolutional layer; Indicates the first The number of output channels of each convolutional layer; Indicates the kernel size; Indicates the first The height of the layer input feature map; Indicates the first The width of the layer input feature map.

[0116] The computing latency includes cloud computing latency and edge computing latency.

[0117] The cloud computing latency refers to the time delay of the cloud server in the denoising step Next, the time delay generated in each step of the denoising process in the cloud server. The total time delay of the cloud server computing processing inference task is shown in equation (39): ; (39) Wherein, represents the computing processing time delay of the inference task on the cloud server; represents the number of denoising steps performed by the cloud server; represents the number of floating point operations required for one forward propagation of the layer network layer; represents the floating point computing capacity currently available to the cloud server.

[0118] When , it means that the diffusion model denoising image generation process is not calculated in the cloud server, so there is no cloud server computing time delay.

[0119] Wherein, represents the total amount of queued computing tasks in the cloud server at time slot, and the update rule is shown in equation (40): ; (40) Wherein, represents the total amount of queued computing tasks in the cloud server at time slot; represents the maximum value; the term represents the computing amount completed by the cloud server in the time slot lasting ; the term represents the computing amount of newly arrived tasks, which is added to the backlog tasks.

[0120] When and , the computing time delay of the cloud server satisfies equation (41): ; (41) Therefore, the total time delay of the cloud server is shown in equation (42): ; (42) Wherein, represents the total time delay of the cloud server; represents the task queuing time delay of the cloud server; represents the data transmission time delay of the cloud server; denotes the computation processing latency of the cloud server.

[0121] The edge computing latency refers to the computation processing latency of the edge server in the denoising step Next, the computation processing latency of the edge server in each step of the denoising process. The total latency of the edge server in computing and processing the inference task is shown in equation (43): ; (43) wherein, denotes the computation processing latency of the inference task on the edge server; denotes the number of denoising steps performed by the edge server; denotes the number of floating point operations required for one forward propagation of the layer network layer; denotes the floating point computation capability currently available to the edge server.

[0122] When , it means that the denoising image generation process of the diffusion model is not computed on the edge server, and thus there is no computation processing latency of the edge server.

[0123] wherein, denotes the total amount of queued computation tasks in the edge server at time slot , and the update rule is shown in equation (44): ; (44) wherein, denotes the total amount of queued computation tasks in the edge server at time slot ; denotes taking the maximum value; the term denotes the computation amount completed by the edge server in the time duration of at time slot ; the term denotes the computation amount of the newly arrived tasks, which is added to the backlog tasks.

[0124] Therefore, the total latency of the edge server is shown in equation (45): ; (45) wherein, denotes the total latency of the edge server; denotes the task queuing latency of the edge server; denotes the data transmission latency of the edge server; denotes the computation processing latency of the edge server.

[0125] As can be seen above, the total processing latency of the system includes the total latency of the cloud server and the total latency of the edge server, which is shown in equation (46): ; (46) wherein, denotes the total processing latency of the system for performing the inference task in the time slot ; (45) denotes the total processing latency of the cloud server; denotes the total processing latency of the edge server.

[0126] In some embodiments, the generated image quality is evaluated by three indicators: Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Frechet Inception Distance (FID).

[0127] Specifically, the Structural Similarity Index is used to measure the similarity of the generated image and the real image in brightness, contrast, structure, etc., as shown in formula (47): ; (47) wherein, denotes the Structural Similarity Index between the generated image and the real image ; (48) denotes the average value of the pixels of the generated image; denotes the average value of the pixels of the real image; denotes the standard deviation of the pixels of the generated image; denotes the standard deviation of the pixels of the real image; denotes the covariance of the generated image and the real image; , is a stable constant.

[0128] wherein, the real image is a pre-stored high-quality reference image or a high-quality image generated by a standard diffusion model full step.

[0129] The Peak Signal-to-Noise Ratio is used to quantify the pixel-level error between the generated image and the real image, as shown in formula (48): ; (48) wherein, denotes the Peak Signal-to-Noise Ratio between the generated image and the real image ; (49) denotes taking the maximum value; denotes the intensity value of the th pixel in the generated image; denotes the intensity value of the​ intensity value of a pixel; denotes the total number of pixels in the image.

[0130] The Frechet Inception Distance is used to measure the distribution difference between the generated image and the real image in the deep feature space, and the lower the distance value is, the more realistic the generated image is, as shown in equation (49): ; (49) wherein, denotes the Frechet Inception Distance between the generated image and the real image . denotes the mean vector of the generated image feature; denotes the mean vector of the real image feature; denotes the trace of a matrix; denotes the covariance matrix of the generated image feature; denotes the covariance matrix of the real image feature.

[0131] In the present application, the optimization target is to minimize the diffusion model processing task time while ensuring the quality of the generated image. The objective function of the optimization target is shown in equation (50): ; (50) wherein, denotes the minimum value; denotes the mathematical expectation; denotes the total processing delay of the system; denotes the task belongs to the task set .

[0132] Equation (50) satisfies the following constraint conditions at the same time, as shown in equations (51)-(58): ; (51) ; (52) ; (53) ; (54) ; (55) ; (56) ; (57) ; (58) wherein, .

[0133] Among them, formulas (51)~(53) define the minimum standard of setting the generated image related evaluation index to ensure the quality of the generated image. Among them, 、 、 respectively represent the minimum received value of the setting of the three indicators of structural similarity index, peak signal-to-noise ratio and Fréchet distance.

[0134] Formulas (54), (55) are used to ensure that the cloud server and the edge server cooperate to complete all denoising steps of the diffusion model, and require these steps to be integers.

[0135] Formula (56) is used to ensure that the system utilization rates of the cloud server and the edge server are both less than 1, that is, within a reasonable operating range.

[0136] Formulas (57), (58) are used to limit the value range of the computing resources of the cloud server and the edge server, wherein, and respectively represent the maximum floating-point computing capacity of the cloud server and the edge server.

[0137] Based on the above optimization objectives, objective functions and constraint conditions, the reward function of the multi-agent attention actor-critic model is constructed, as shown in formula (59): ; (59) Among them, represents the reward value obtained by all agents at time slot; and represent weight coefficients, the value range of which is (0, 1], and the sum is 1; represents a set of inference tasks, is one of the inference tasks; represents the number of edge servers, is the index of the edge server; represents the total latency of the cloud server; represents the total latency of the edge server; represents the normalized value of the structural similarity index of the generated image of the inference task ; represents the normalized value of the peak signal-to-noise ratio of the generated image of the inference task ; represents the normalized value of the Fréchet distance of the generated image of the inference task .

[0138] As Figure 4The algorithm architecture diagram of the multi-agent attention actor-critic model is shown. Noise sampling is performed in the high-dimensional state-action search space of the diffusion model, and the optimal action distribution is gradually approached through a multi-step denoising process. The multi-head attention mechanism is used in the critic network part, so that each agent can adaptively focus on the most relevant state and action of other agents according to global information, and the scalability problem of the critic network can be effectively solved.

[0139] In the present application, the system has (I+1) agents, including one cloud server and I edge servers. Each agent generates a corresponding action distribution according to its own local observation, and continuously adjusts the strategy in multiple rounds of interaction. The goal of the DMAAC algorithm is to enable all agents to learn a collaborative strategy to maximize the long-term expected discounted return, i.e., to optimize the decision quality and system overall performance simultaneously in a complex dynamic edge computing environment.

[0140] In some embodiments, in order to avoid the strategy converging to a non-optimal deterministic solution too early, the maximum entropy reinforcement learning framework, i.e., the Soft Actor-Critic algorithm, is adopted, and an entropy term is added to the policy gradient, as shown in formula (60): ; (60) wherein, denotes the gradient operator; denotes the function The gradient of the actor network parameter is the direction of parameter update; denotes expectation; denotes the performance objective function of the policy of the agent ; denotes the observed environment state of the agent ; denotes the experience replay buffer; denotes the action performed by the agent ; denotes the policy function; denotes the probability density of the action selected by the actor network in the state ; is a temperature parameter, which is a hyperparameter, used to balance the relative importance of the entropy term and the reward in the objective function; is the state-action value function estimated by the critic network of the agent , representing the expected cumulative return that can be obtained after performing the action in the state ; denotes the parameter of the critic network; ​​​​​denotes a state-dependent baseline function for reducing the variance of gradient estimates.

[0141] The value function learning objective is shown in equation (61): (61) where, denotes the target Q-value, a supervisory signal for training the critic network; denotes the immediate reward obtained after performing action in state ; denotes a discount factor that measures the current value of future rewards; denotes the action taken in the next state ; denotes the next state reached after performing action ; denotes the Q-value estimated by the target critic network, is the parameter of the target critic network; denotes the probability density of the action selected by the target policy network in the next state ; is the parameter of the target actor network.

[0142] To address the credit assignment and scalability issues in multi-agent environments, the present application introduces an attention mechanism in the critic network. The computational formula of the Q-value function is shown in equation (62): (62) where, denotes the final Q-value output of the critic network for agent ; denotes the local observation of agent ; denotes a two-layered multi-layer perceptron (MLP); denotes a one-layer MLP embedding function for encoding the state-action pair of agent itself; denotes the context information aggregated from other agents.

[0143] where the context information aggregation is shown in equation (63): (63) where, denotes the attention weight of agent on agent ; denotes agent embedding vectors of state-action pairs by embedding function encoding, shared matrix linear transformation, nonlinear function ( e.g., ReLU) to obtain.

[0144] where the attention weight is calculated as shown in equation (64): ; (64) where denotes an exponential function; and denotes the embedding representation of the agent and the agent . and denotes a trainable linear transformation matrix for generating the key and the query; denotes a transpose.

[0145] Due to parameter sharing, each critic network parameter is updated by minimizing the joint regression loss function of all critic networks. The joint regression loss function is shown in equation (65): ; (65) where denotes the total loss value of all agent critic network parameters ; denotes the index of the agent, ; denotes the mathematical expectation of the loss calculated on a batch of experience data randomly sampled from the experience replay buffer ; , respectively, denotes the state, action, reward, and next time slot state; denotes the predicted Q value of the critic network of the agent at time slot in state when performing action ; denotes the target Q value.

[0146] where the target Q value is calculated as shown in equation (66): ; (66) where denotes the target Q value of the agent ; denotes the reward obtained by the agent at time slot ; other physical quantities are consistent with equation (61), and thus are not described again. ​

[0147] After introducing attention, the policy gradient is shown in formula (67): (67) wherein, represents the observation set of all agents; represents the action set of all agents; represents the action set of all other agents except the agent ; and represents a multi-agent baseline function; other physical quantities are consistent with formula (60) and (62), and thus are not described herein.

[0148] Based on the above description, the multi-agent attention actor-critic model required by the application can be trained, and by using the model, the cloud-edge collaborative reasoning strategy can be generated by real-time acquisition of the environment state of the cloud server and the edge server.

[0149] In step S103, if it is determined in step S101 that the historical intermediate result cannot be reused, the number of denoising steps performed by the cloud server and the number of denoising steps performed by the edge server are determined according to the collaborative reasoning strategy obtained in step S102, the initial noise of the current text prompt word is denoised by the cloud server according to the corresponding denoising step, the current intermediate result is generated and stored in the blockchain, and then the current intermediate result is denoised by the edge server to generate the final image.

[0150] In step S104, if it is determined in step S101 that the historical intermediate result can be reused, the corresponding historical intermediate result is obtained, the number of denoising steps of the edge server is determined according to the collaborative reasoning strategy obtained in step S102, and the historical intermediate result is denoised by the edge server to generate the final image.

[0151] Corresponding to the above method, the application also provides a cloud-edge collaborative diffusion model reasoning system based on blockchain and reinforcement learning, which comprises a processor, a memory and a computer program / instruction stored on the memory, wherein the processor is used to execute the computer program / instruction, and when the computer program / instruction is executed, the system implements the steps of the foregoing method.

[0152] The embodiment of the application also provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the foregoing method are implemented. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0153] Those of ordinary skill in the art will appreciate that the various illustrative components, systems and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or both. The particular implementation is dependent on the specific application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, for example, the hardware can comprise an electronic circuit, an Application Specific Integrated Circuit (ASIC), a suitable firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the application are the program or code segments to perform a specific task. The program or code segments can be stored in a machine-readable medium, or transmitted by a carrier wave as data signals over a transmission medium or communication link.

[0154] It is to be understood that the application is not limited to the particular configurations and processes described herein and shown in the drawings, which can be varied in accordance with the particular needs of the application. For the sake of brevity, conventional techniques and methods related to making and using the application can not be described in detail herein. In the above embodiments, several specific steps are described and illustrated in order to provide a thorough understanding of the present application. However, the process of the present application can be practiced with less than all of the described and illustrated steps, or in a different order than that described and illustrated.

[0155] In the present application, features described and / or illustrated with respect to one embodiment can be used in the same or a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0156] The above description is only preferred embodiments of the present application, and is not intended to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A blockchain and reinforcement learning based federated diffusion model inference method, characterized in that, The method is executed in a blockchain network composed of a cloud server and at least one edge server as nodes, and the method comprises: obtaining a current text prompt word submitted by a user, calling a pre-trained semantic matching model through a smart contract pre-deployed on a blockchain to calculate the semantic similarity between the current text prompt word and historical text prompt words stored in the blockchain, and if the semantic similarity is greater than a preset threshold, it is determined that the corresponding historical intermediate result can be reused, otherwise, it cannot be reused; wherein the semantic matching model adopts a twin network architecture, and the semantic feature vectors of the current prompt word and the historical text prompt word are extracted through sub-networks sharing weights respectively, and the similarity between the two feature vectors is calculated, and the historical intermediate result refers to the diffusion model hidden variable obtained after the cloud server performs a part of denoising steps for the historical text prompt word; obtaining the current environment state of the cloud server and the edge server, and generating a collaborative reasoning strategy for the current reasoning task through a pre-trained multi-agent attention actor-critic model based on the diffusion model, the collaborative reasoning strategy including the denoising steps of the current reasoning task performed by the cloud server and the denoising steps performed by the edge server; wherein the multi-agent attention actor-critic model is constructed with the environment state of the cloud server and the edge server as the state space, and the denoising step distribution strategy of the cloud server and the edge server as the action space, generates an action probability distribution through a denoising diffusion process, and aggregates multi-agent information using an attention mechanism, and its reward function is composed of the weighted sum of the total system processing delay and the generated image quality, wherein the environment state includes task queue length, system utilization, data transmission time and computing power; if the historical intermediate result cannot be reused: according to the denoising steps of the cloud server and the denoising steps of the edge server obtained according to the collaborative reasoning strategy, the initial noise of the current text prompt word is first denoised by the cloud server according to the corresponding denoising steps, to generate a current intermediate result and store it to the blockchain, and then the edge server performs denoising on the current intermediate result to generate a final image; if the historical intermediate result can be reused, obtain the historical intermediate result from the blockchain, and perform denoising on the historical intermediate result by the edge server according to the denoising steps of the edge server obtained according to the collaborative reasoning strategy to generate a final image.

2. The blockchain and reinforcement learning based cloud-edge collaborative diffusion model inference method according to claim 1, characterized in that, The smart contract includes a node management contract, a prompt word management contract and a prompt word matching contract; wherein the node management contract is used to mark the node type and set the corresponding permissions and responsibilities when a new node joins the blockchain network; the prompt word management contract is used to record, deduplicate and verify the legality of the prompt word submitted by the user, and add a mark to the prompt word with existing historical reasoning data; The prompt word matching is approximately used to call the semantic matching model, perform the operation of calculating the semantic similarity between the current text prompt word and the historical text prompt word, and determine whether the corresponding historical intermediate result can be reused according to the result. 3.The blockchain and reinforcement learning based federated diffusion model inference method of claim 1, wherein, The semantic matching model includes a text encoder and a twin network, the text encoder includes a word segmenter and a Transformer module, and the method further includes extracting initial semantic feature vectors of the current prompt word and the historical text prompt word by using the text encoder, including: The current prompt word and the historical text prompt word are respectively decomposed into token sequences by the word segmenter, each token is embedded into a word vector to form a word vector sequence, and the word vector sequence is added to the learnable position encoding and then input into the Transformer module. The word vector sequence is encoded by the stacked multi-head attention layer and the feedforward neural network of the Transformer module, and the semantic feature vector of the current prompt word and the initial semantic feature vector of the historical text prompt word are output. The twin network includes two sub-networks sharing weights, each sub-network includes a convolutional network and a multi-layer perceptron, and the semantic similarity between the current text prompt word and the historical text prompt word is calculated, including:

4. The blockchain and reinforcement learning based federated diffusion model inference method of claim 3, wherein, The semantic feature vector of the current prompt word and the initial semantic feature vector of the historical text prompt word are respectively input into two sub-networks, in each sub-network, local semantic pattern features are extracted by the convolutional network, and then the local semantic pattern features are nonlinearly transformed and reduced in dimension by the multi-layer perceptron to obtain the final semantic feature vectors of the current prompt word and the historical text prompt word. The cosine similarity between the final semantic feature vectors of the current prompt word and the historical text prompt word is calculated, and the following formula is satisfied: The multi-agent attention actor-critic model generates an action probability distribution through a denoising diffusion process, including: ； wherein, denotes the current cue a semantic similarity between the current cue and a historical text cue; denotes a final semantic feature vector of the current cue; denotes a final semantic feature vector of the historical text cue; denotes a transpose; denotes an L2 norm.​ 5.The blockchain and reinforcement learning based federated diffusion model inference method of claim 1, wherein, An initial noise is sampled from a standard Gaussian distribution as an initial action probability distribution; Starting from the initial action probability distribution, the actor network in the multi-agent attention actor-critic model is iterated for a preset number of steps to denoise and gradually restore a clear action probability distribution, and the denoising steps satisfy the following formula: The clear action probability distribution is converted into a final action selection distribution by using a Softmax function, and the conversion process satisfies the following formula: ； wherein, represents the state of the environment observed by the agent at the time slot represents the state of the environment observed by the agent at the time slot represents the action probability distribution of the agent at the time slot represents the action probability distribution of the agent at the time slot represents the noise retention rate at the time slot represents the noise retention rate at the time slot represents the cumulative noise retention rate from the time slot to the time slot represents the cumulative noise retention rate from the time slot to the time slot represents the noise predicted by the critic network represents the state of the environment observed by the agent at the time slot represents the state of the environment observed by the agent at the time slot represents the state of the environment observed by the agent at the time slot represents the variance adjustment coefficient represents the random noise The total processing delay of the system includes the total delay of the cloud server and the total delay of the edge server, and the following formula is satisfied: ； wherein, represents the probability vector of taking each action at time slot under the environmental state; represents the dimension value in the clarified action probability distribution of the agent at time slot corresponding to action and represent the index of the action; represents a natural constant; represents summation.

6. The blockchain and reinforcement learning based federated diffusion model inference method of claim 1, wherein, The generated image quality is evaluated by three indicators of structural similarity index, peak signal-to-noise ratio and Frechet Inception Distance; ； wherein, represents the total processing delay of the system when performing an inference task in a time slot ; represents the total delay of the cloud server, including cloud computing processing delay, task queuing delay of the cloud server, data transmission delay of the cloud server transmitting intermediate results to the edge server; represents the total delay of the edge server, including edge computing processing delay, task queuing delay of the edge server, data transmission delay of the edge server transmitting the final image to the user.

7. The blockchain and reinforcement learning based federated diffusion model inference method of claim 6, wherein, The structural similarity index is used to measure the overall similarity of the generated image and the real image in multiple dimensions, and the multiple dimensions at least include brightness, contrast and structure; The peak signal-to-noise ratio is used to quantify the pixel-level error between the generated image and the real image; The Frechet Inception Distance is used to measure the distribution difference between the generated image and the real image in the deep feature space. The reward function satisfies the following formula:

8. The blockchain and reinforcement learning based federated diffusion model inference method of claim 7, wherein, ​ ； wherein, represents the reward value obtained by all agents in a time slot; and represents a weight coefficient; represents a set of inference tasks, is one of the inference tasks; represents a number of edge servers, is an index of the edge server; represents a total time delay of the cloud server; represents a total time delay of the edge server; represents a structure similarity index of a generated image of an inference task after normalization; represents a peak signal-to-noise ratio of a generated image of an inference task after normalization; represents a Frechet Inception Distance of a generated image of an inference task after normalization.​ 9.The blockchain and reinforcement learning based federated diffusion model inference method of claim 1, wherein, The method further comprises training the multi-agent attention actor-critic model through a maximum entropy reinforcement learning framework, and the training step comprises: updating each critic network parameter by minimizing a joint regression loss function of all agent critic networks; wherein each agent critic network utilizes an attention mechanism to dynamically calculate the relevance weight between its own and other agent environment state and action information, and aggregates the information of other agents according to the relevance weight to cooperatively estimate the action value function; wherein the joint regression loss function satisfies the following formula: ； in, Represents the network parameters of all intelligent agents and critics. The total loss value; Index representing the agent, ; This indicates the buffer from the experience replay. A batch of empirical data randomly sampled from the middle The expected value obtained by calculating the loss These represent the state, action, reward, and state of the next time slot, respectively. Indicates in Time-slotted intelligent agents Critics Network Execute in state Predicted Q-value under action; Indicates the target Q value; updating the actor network parameter by maximizing the weighted sum of the expected return and the policy entropy; wherein each agent actor network updates the policy parameter through a policy gradient method according to the value guidance provided by the critic network and the multi-agent baseline related to the policy of other agents. 10.A blockchain and reinforcement learning based cloud-edge collaborative diffusion model inference system, comprising a processor, a memory and computer programs / instructions stored on the memory, characterized in that, The processor is configured to execute the computer programs / instructions, and when the computer programs / instructions are executed, the system implements the steps of the method according to any one of claims 1 to 9.

Images