Edge collaborative reasoning optimization method for generative artificial intelligence service

Abstract

The invention provides a generative artificial intelligence service-oriented edge collaborative reasoning optimization method, which relates to the technical field of distributed computing, and comprises the following steps: a client sends a model request and text description to an edge server; the edge server generates an initial noise image according to the text description and the generative artificial intelligence model, and sends the generative artificial intelligence model and the initial noise image to the client; the client executes multiple rounds of reasoning processes according to the generative artificial intelligence model and the initial noise image to generate a client noise image; and the edge server continues to execute the remaining reasoning process based on the client noise image to generate a noise picture after reasoning is completed, and the noise picture after reasoning is completed is converted through the variational auto-encoder to obtain a generated picture. According to the invention, high-quality performance of service can be kept under the condition of limited computing resources and network bandwidth, and the stability and safety of user experience are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of distributed computing technology, and in particular to an edge collaborative reasoning optimization method for generative artificial intelligence services. Background Art

[0002] Edge computing and AIGC technology based on diffusion models have gradually become two key areas to promote the development of intelligent applications. Edge computing reduces dependence on central data centers by performing computing tasks at the edge of the network, effectively reduces data transmission delays, improves processing speed, and enhances user privacy protection. AIGC technology based on diffusion models uses its powerful data generation capabilities to generate high-quality images and texts by gradually adding and removing noise. Current AI-generated content (AIGC) services usually rely on centralized cloud computing processing. This dependence brings some problems: first, this approach leads to high network latency and large bandwidth usage, and second, a lot of energy is consumed in the process of processing user requests and completing user services. In addition, frequent data transmission between users and cloud centers increases the risk of privacy leakage. This frequency of data exchange may affect the availability and performance of services when network conditions are unstable or bandwidth is limited, and may also pose a potential threat to data privacy and security. During peak network usage hours, the overload of cloud centers further reduces service quality and affects user experience.

[0003] Therefore, how to optimize the computing process of generative artificial intelligence services becomes a technical problem that needs to be solved. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art or related technology, and discloses an edge collaborative reasoning optimization method for generative artificial intelligence services, which realizes the efficient execution of artificial intelligence generated content services on resource-constrained devices and improves the service quality of generative artificial intelligence services.

[0005] Glossary: ​​AI Generated Content (AIGC): refers to the content creation method that uses artificial intelligence algorithms to automatically generate, modify and enhance data. It includes areas such as image generation and natural language generation, and can automatically create novel content. Diffusion Models: A class of Markov chain generation models that generate data by gradually adding noise and then reversing this process. These models perform well in image and text generation tasks. Edge Computing: A distributed computing paradigm that transfers computing tasks from the central cloud to devices at the edge of the network to reduce latency and network burden and improve service response speed. Variational Autoencoder (VAE): A deep learning technique for generative models that generates new data instances by learning the latent representation of data.

[0006] The first aspect of the present invention discloses an edge collaborative reasoning optimization method for generative artificial intelligence services, including: S1, the client sends a model request and a text description to the edge server, so that the edge server calls the corresponding generative artificial intelligence model according to the model request; S2, the edge server generates an initial noise image according to the text description and the generative artificial intelligence model, and sends the generative artificial intelligence model and the initial noise image to the client; S3, the client performs multiple rounds of reasoning process according to the generative artificial intelligence model and the initial noise image to generate a client noise image, and sends the client noise image to the edge server; S4, the edge server continues to perform the remaining reasoning process based on the client noise image to generate a noise image that has completed reasoning. During this period, the multiple rounds of reasoning calculations involved in an image generation task are dynamically allocated to the client and the edge server; S5, the edge server converts the noise image that has completed reasoning through a variational autoencoder to obtain a generated image.

[0007] In this technical solution, the distributed reasoning optimization method combines edge computing with the diffusion model to achieve efficient execution of AI-generated content services on resource-constrained devices. This method allows devices in a wireless network to work together, maximizes the utilization efficiency of computing resources through distributed computing, and protects user privacy. By distributing the denoising steps of the diffusion model among devices, this method not only reduces dependence on the central cloud, but also reduces network energy consumption, and improves the response speed and quality of the service through close cooperation between edge computing nodes and user devices. This method is particularly suitable for application scenarios that require fast response and high computing efficiency, such as real-time image and video processing, and achieves efficient processing of content generation tasks while ensuring data privacy and security.

[0008] According to the edge collaborative reasoning optimization method for generative artificial intelligence services disclosed in the present invention, preferably, step S1 specifically includes:

[0009] The client sends a model request to the edge server and text description ; The edge server queries whether the model requested by the client is cached. If the model is not cached in the edge server, the model request is sent to the cloud so that the edge server receives the generative artificial intelligence model sent by the cloud and caches it.

[0010] According to the edge collaborative reasoning optimization method for generative artificial intelligence services disclosed in the present invention, preferably, step S2 specifically includes:

[0011] After determining that the edge server has cached the generative AI model, the edge server processes the text description to obtain the text encoding , and get the initial noise image based on the generative artificial intelligence model ;

[0012] Edge servers will generate AI models, text encoding And the initial noise image Send to the client;

[0013] The process of processing the text description to obtain the text encoding specifically includes:

[0014]

[0015] Among them, text represents the text vector, The text description Convert to the corresponding vector representation; It is a vector representation used to represent the location information of each text;

[0016] The text vector is processed through the Transformer model to obtain the feature representation of the text:

[0017]

[0018] Among them, z text Encode the text, () represents a series of Transformer layers, and text represents a text vector.

[0019] According to the edge collaborative reasoning optimization method for generative artificial intelligence services disclosed in the present invention, preferably, step S3 specifically includes:

[0020] Based on the generative artificial intelligence model, the client generates the initial noise image under the guidance of text encoding. conduct The reverse denoising process of the wheel, the denoising formula is:

[0021]

[0022] in, Indicates that the client is The noise image obtained in the round inference, is a predefined attenuation coefficient determined by the noise scheduler, represents the initial attenuation coefficient, represents the prediction noise of the client model at the current time step;

[0023] After t rounds of reasoning, the client noise image is obtained And send it to the edge server, which performs the remaining reasoning steps.

[0024] According to the edge collaborative reasoning optimization method for generative artificial intelligence services disclosed in the present invention, preferably, step S4 specifically includes:

[0025] The edge server sends the client noisy image and text encoding The remaining Wheel of Reasoning;

[0026] The edge server model is guided by text encoding. Perform the reverse denoising process, and the denoising formula is as follows:

[0027]

[0028] in, The edge server is in the inference step The The noise image obtained by step The edge server is in the inference step The The predefined attenuation coefficient of the step, is the attenuation coefficient of the t-th round of reverse denoising;

[0029] After the edge server completes all the remaining inference steps, the noisy image with completed inference is obtained. .

[0030] According to the edge collaborative reasoning optimization method for generative artificial intelligence services disclosed in the present invention, preferably, step S5 specifically includes:

[0031] The edge server sends the noise image Input to the variational autoencoder for encoding to obtain the final generated image :

[0032] Variational Autoencoder Pair The encoding formula is as follows:

[0033]

[0034] in It represents the decoder in the variational autoencoder, which is used to convert the noisy image back to the image space.

[0035] According to the edge collaborative reasoning optimization method for generative artificial intelligence services disclosed in the present invention, preferably, the dynamic allocation in step S4 specifically includes: dynamically adjusting the allocation of reasoning tasks according to the current network conditions and device performance; or determining the allocation of reasoning tasks according to the privacy of the reasoning tasks.

[0036] According to the edge collaborative reasoning optimization method for generative artificial intelligence services disclosed by the present invention, preferably, the generative artificial intelligence model is a diffusion model.

[0037] According to the edge collaborative reasoning optimization method for generative artificial intelligence services disclosed by the present invention, preferably, the diffusion model is specifically a Stable Diffusion model.

[0038] Practical application scenarios of the technical solution provided by the present invention: In view of the high computing demand and network resource consumption in traditional AIGC services, the present invention proposes a solution for collaborative reasoning of edge users. By distributing computing tasks between user devices (clients) and edge servers, this method effectively reduces dependence on the central cloud, reduces network latency and bandwidth usage, and improves service quality and user experience. This collaborative architecture enables reasoning calculations to be performed more efficiently in a distributed environment, thereby meeting the requirements of low latency. The present invention achieves an optimized distribution of computing loads by allowing user devices to perform some reasoning steps and completing the remaining steps on edge servers. This method not only effectively reduces the computing pressure of a single device, but also enhances the privacy protection capability of user data by reducing the amount of data transmitted, thereby efficiently providing AIGC services in a resource-constrained network environment. This load distribution method helps to maintain high-quality performance of services under limited computing resources and network bandwidth conditions, and significantly improves the stability and security of user experience.

[0039] The beneficial effects of the present invention include at least:

[0040] (1) By distributing computing tasks between user devices and edge servers, the latency and resource consumption of AIGC services are optimized while protecting user privacy and improving user experience. It reduces network burden by reducing dependence on the central cloud and can continuously optimize model performance by combining federated learning technology to achieve efficient, secure and user-friendly AIGC service delivery.

[0041] (2) By distributing the inference tasks of the diffusion model between user devices and edge servers, the computational burden of a single device can be reduced, thereby improving the efficiency and response speed of the overall service.

[0042] (3) By dynamically adjusting the allocation of inference tasks based on the current network conditions and device performance, the computing resource utilization efficiency of user devices is improved, achieving a more efficient content generation process.

[0043] (4) Reduce the waste of network resources between user devices and edge servers in AIGC services: Reduce overall energy consumption by balancing the computing load, ensuring that while meeting service quality requirements, the network communication requirements between devices are effectively reduced, thereby significantly improving the resource utilization efficiency of the system.

[0044] (5) By allowing the user device to perform part of the inference calculation, the need to transmit data to the central cloud is reduced, thereby enhancing the privacy security of the model in the edge computing environment. This method not only reduces network traffic, but also effectively protects data privacy by retaining sensitive data on the local device, while further improving the system's response speed and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of an edge collaborative reasoning optimization method for generative artificial intelligence services according to an embodiment of the present invention is shown.

[0046] Figure 2 A schematic diagram of an algorithm flow according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and therefore, the present invention is not limited to the limitations of the specific embodiments disclosed below.

[0048] like Figure 1 As shown, according to one embodiment of the present invention, an edge collaborative reasoning optimization method for generative artificial intelligence services is disclosed, including:

[0049] S1, request model and upload text description: The client sends a model request and text description to the edge server so that the edge server calls the corresponding generative artificial intelligence model according to the model request;

[0050] S2, generating an initial noisy image: the edge server generates an initial noisy image according to the text description and the generative artificial intelligence model, and sends the generative artificial intelligence model and the initial noisy image to the client;

[0051] S3, perform client reasoning: the client performs multiple rounds of reasoning according to the generative artificial intelligence model and the initial noise image, generates a client noise image, and sends the client noise image to the edge server;

[0052] S4, execute edge server reasoning: the edge server continues to perform the remaining reasoning process based on the client noise image to generate a noise image that has completed the reasoning. During this period, multiple rounds of reasoning calculations involved in an image generation task are dynamically allocated to the client and the edge server;

[0053] S5, image conversion: The edge server converts the noise image that has completed inference through a variational autoencoder to obtain a generated image.

[0054] like Figure 2 As shown, according to the edge collaborative reasoning optimization method for generative artificial intelligence services provided in the above embodiment, another embodiment of the present invention further discloses the working process of the above method in an actual application scenario:

[0055] (1) First, the user (client) sends a model request and text description to the edge server.

[0056] (2) After receiving the model request and text description from the user, the edge server first processes the user's text description to obtain a text encoding and generates an initial noise image. The edge server sends the model, text encoding, and initial noise image requested by the user to the user.

[0057] (3) After receiving the model, text encoding, and initial noisy image, the user follows the reverse denoising process of the conditional diffusion model. The model performs a partial reasoning process on the initial noisy image under the guidance of the text encoding to obtain the user noisy image. The user sends the user noisy image to the edge server for the remaining reasoning steps.

[0058] (4) After the edge server receives the user's noise image, the conditional diffusion model in the edge server performs the remaining reasoning process under the guidance of the text encoding to obtain the edge server noise image that completes the entire reasoning process.

[0059] (5) Since the edge server noise image is still in the latent space, it needs to be decoded by a VAE (variational autoencoder) to obtain the final generated image.

[0060] (6) The edge server sends the final generated image to the user, completing the user's AIGC service.

[0061] In step (1), the user sends a model request to the edge server and text description ;

[0062] In step (2), the edge server further queries whether the model requested by the user is cached. If the model is not cached, the user's model request is sent to the cloud so that the edge server can add the model to the cache after receiving the model sent by the cloud. If the model is cached, the edge server embeds the user's text description, and further processes it to obtain the text encoding to guide the subsequent reasoning process, and obtains the initial noise image through the noise predictor. ;

[0063] The process of embedding user text is shown in the following formula:

[0064]

[0065] in The text Convert to the corresponding vector representation; is a vector representation used to represent the location information of each text. After the text is embedded, it will be processed through a Transformer model to obtain the feature representation of the text (i.e., text encoding). The formula is as follows:

[0066]

[0067] in, Represents a series of Transformer layers, including Self-Attention layers and Feed-Forward Neural Network layers.

[0068] The edge server encodes the model and text requested by the user With the initial noise image Send to user.

[0069] In step (3), the user model further initializes the noise image under the guidance of the text encoding. conduct The reverse denoising process of the wheel, the denoising formula is:

[0070]

[0071] in, Indicates the user's reasoning step A step in The resulting noise image is is a predefined attenuation coefficient determined by the noise scheduler, which helps the model understand the amount of noise to remove at each step and recover a clearer image from the current noisy image. represents the initial attenuation coefficient, which is set to a value approximately equal to 0 in order to generate a completely noisy image, represents the prediction noise of the user model at the current time step, and is expressed by Subtract the predicted noise to get the noise image for that time step. The user will get the user noise image The data is sent to the edge server, which performs the remaining reasoning steps.

[0072] In step (4), the edge server further sends the user noise image and text encoding Input to the edge server model for the rest Reasoning step: The edge server model uses text encoding to guide the user's noisy image Perform the reverse denoising process, and the denoising formula is as follows:

[0073]

[0074] in, The edge server is in the inference step A step in The resulting noisy image.

[0075] In step (5), further, the edge server noise image Input to VAE for encoding to get the final generated image . VAE on edge server noisy images The encoding formula is as follows:

[0076]

[0077] in It represents the decoder in VAE, which converts the noisy image back to image space.

[0078] In summary, the present invention aims to optimize the latency and resource consumption of AIGC services by distributing computing tasks between user devices and edge servers, while protecting user privacy and improving user experience. It reduces the network burden by reducing the reliance on the central cloud, and continuously optimizes the model performance by combining federated learning technology to achieve efficient, secure and user-friendly AIGC service delivery. In the present invention, an edge server and user collaborative reasoning optimization method for diffusion model AIGC is proposed. The method distributes the reasoning tasks of the diffusion model between the user device and the edge server to reduce the computing burden of a single device and improve the efficiency and response speed of the overall service. Specifically, the method first generates an initial noisy image on the edge server using user requests and text prompts, and then the user device performs some reasoning steps to refine the image, and finally the edge server completes the remaining reasoning steps to generate the final image. In this process, the reasoning tasks between the user device and the edge server are dynamically allocated to balance the computing load and optimize the service delay. In addition, the method also reduces network resource consumption and improves the quality of experience (QoE) by performing efficient data and model transmission between the user device and the edge server. The approach focuses on optimizing the performance of AIGC services through distributed computing and task allocation.

[0079] All or part of the steps in the various methods of the above embodiments can be completed by controlling the relevant hardware through a program, and the program can be stored in a readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other readable medium that can be used to carry or store data.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An edge collaborative reasoning optimization method for generative artificial intelligence services, characterized in that: include: S1, the client sends a model request and a text description to the edge server, so that the edge server calls the corresponding generative artificial intelligence model according to the model request; S2, the edge server generates an initial noise image according to the text description and the generative artificial intelligence model, and sends the generative artificial intelligence model and the initial noise image to the client; S3, the client performs multiple rounds of reasoning according to the generative artificial intelligence model and the initial noise image to generate a client noise image, and sends the client noise image to the edge server; S4, the edge server continues to perform the remaining reasoning process based on the client noise image to generate a noise image that has completed the reasoning. During this period, multiple rounds of reasoning calculations involved in an image generation task are dynamically allocated to the client and the edge server; S5, the edge server converts the noise image that completes the inference through a variational autoencoder to obtain a generated image.

2. The edge collaborative reasoning optimization method for generative artificial intelligence services according to claim 1, characterized in that: The step S1 specifically includes: The client sends a model request and a text description to the edge server; The edge server queries whether the model requested by the client is cached. If the model is not cached in the edge server, the model request is sent to the cloud so that the edge server receives the generative artificial intelligence model sent by the cloud and caches it.

3. The edge collaborative reasoning optimization method for generative artificial intelligence services according to claim 1, characterized in that: The step S2 specifically includes: After determining that the edge server has cached the generative artificial intelligence model, the edge server processes the text description to obtain a text encoding , and obtain the initial noise image according to the generative artificial intelligence model ; The edge server encodes the generative artificial intelligence model and the text And the initial noise image Sending to the client; The process of processing the text description to obtain the text encoding specifically includes: Among them, text represents the text vector, The text description Convert to the corresponding vector representation; It is a vector representation used to represent the location information of each text; The text vector is processed through the Transformer model to obtain the feature representation of the text: Among them, z text Encode the text, () represents a series of Transformer layers, and text represents a text vector.

4. The edge collaborative reasoning optimization method for generative artificial intelligence services according to claim 3 is characterized in that: The step S3 specifically includes: The client generates an initial noise image based on the generative artificial intelligence model under the guidance of the text encoding. conduct The reverse denoising process of the wheel, the denoising formula is: in, Indicates that the client is The noise image obtained in the round inference, is a predefined attenuation coefficient determined by the noise scheduler, represents the initial attenuation coefficient, represents the prediction noise of the client model at the current time step; After t rounds of reasoning, the client noise image is obtained And send it to the edge server, which performs the remaining reasoning steps.

5. The edge collaborative reasoning optimization method for generative artificial intelligence services according to claim 4, characterized in that: The step S4 specifically includes: The edge server sends the client noise image and the text encoding The remaining Wheel of Reasoning; The edge server model is guided by text encoding. Perform the reverse denoising process, and the denoising formula is as follows: in, The edge server is in the inference step The The noise image obtained by step The edge server is in the inference step The The predefined attenuation coefficient of the step, is the attenuation coefficient of the t-th round of reverse denoising; After the edge server completes all remaining reasoning steps, the noise image of the completed reasoning is obtained. .

6. The edge collaborative reasoning optimization method for generative artificial intelligence services according to claim 5, characterized in that: The step S5 specifically includes: The edge server sends the noise image Input to the variational autoencoder for encoding to obtain the final generated image : Variational Autoencoder Pair The encoding formula is as follows: in It represents the decoder in the variational autoencoder, which is used to convert the noisy image back to the image space.

7. The edge collaborative reasoning optimization method for generative artificial intelligence services according to any one of claims 1 to 6, characterized in that: The dynamic allocation in step S4 specifically includes: Dynamically adjust the allocation of inference tasks based on current network conditions and device performance; Alternatively, the allocation of reasoning tasks is determined based on the privacy of the reasoning tasks.

8. The edge collaborative reasoning optimization method for generative artificial intelligence services according to any one of claims 1 to 6, characterized in that: The generative artificial intelligence model is a diffusion model.

9. The edge collaborative reasoning optimization method for generative artificial intelligence services according to claim 8, characterized in that: The diffusion model is specifically a Stable Diffusion model.

Images