A retrieval augmentation generation deployment method based on edge computing

Abstract

The application discloses a retrieval enhancement generation deployment method based on edge computing, which comprises the following steps: based on the container deployment published by the cloud, a customized model corresponding to the application service requirement information required for retrieval is deployed; wherein the container is obtained after the general large model and the corresponding application data are customized and compressed by the cloud; a few-sample or zero-sample learning method is used to fine-tune the deployed customized model, and knowledge base retrieval and updating are carried out based on the fine-tuned customized model and user interaction data; the local application service data knowledge base is updated based on the preprocessed cloud database, and the knowledge base data and related index are shared with other edge computing nodes in a point-to-point manner under the data compliance constraint; the application realizes dynamic data collection and processing based on the cloud and the edge computing node, ensures that the model service can utilize the latest and diversified data, and improves the response speed and output quality of the retrieval service.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a retrieval enhancement generation deployment method based on edge computing. BACKGROUND

[0002] In recent years, general large models have shown great application potential in various applications. General large models (such as GPT-4) have excellent performance in machine translation, chat and question answering, sentiment analysis, text generation and other fields due to their strong cross-task processing and context learning capabilities, and can even quickly adapt to customized service tasks after fine-tuning with few or zero samples. General model technology is mainly based on a probability generation model, so it also has limitations such as uncontrollable model output quality and difficulty in ensuring consistency. In order to overcome these limitations, based on retrieval enhancement generation technology, a vector index of data entries contained in an external reliable data source (in the same embedding space) is established, an embedding representation of a user query statement is generated, the most relevant data entries to the user query are found, and the user query statement is expanded using these data entries, thereby providing more rich context for the reasoning of the general large model and reducing the "model illusion" and consistency problems caused by uncontrollable model output quality.

[0003] General models combined with retrieval enhancement generation have shown great potential for customized services, but due to high computing resource requirements and scalability considerations, current deployment solutions mainly rely on the following solutions.

[0004] The first is a cloud deployment solution, such as cloud service providers that specialize in solving the computing resource requirements, cost, scalability and other issues required for general large model deployment. However, the cloud deployment solution also has two limitations. On the one hand, the quality of service of interactive applications enabled by general large models depends not only on the quality of the output content, but also on the response speed. For response speed, cloud-based general model deployment on high-performance servers has optimized the generation of reasoning from query to inference. In contrast, the fluctuation of the network environment for end users from the cloud has become the main bottleneck for response performance optimization. On the other hand, current retrieval enhancement generation technology mainly relies on static, offline external data sources / databases. These data provide rich context information for user dialogue queries, but the current solution cannot integrate real-time application data and user interaction data, and fails to fully utilize the potential value of real-time and interactive information for service quality improvement. The reason is that user data is difficult to collect centrally due to privacy protection compliance, and real-time data needs more effective distributed indexing mechanisms for management.

[0005] The second type is the distributed deployment solution. A distributed solution involves breaking down a large model into multiple smaller models (e.g., based on model parallelism, data parallelism, and pipeline parallelism) and deploying them on different servers. This effectively reduces the computing resource requirements of a single server. Combined with a retrieval database deployed on a dedicated server, this enables a general-purpose large model service based on retrieval enhancements. However, distributed deployment solutions primarily consider implementation in closed distributed computing environments, such as high-performance computing clusters or self-built supercomputing centers. The difference between this type of solution and cloud deployment is that the latter provides an abstraction of service outsourcing, while distributed deployment exposes more details and flexibility in deployment and management. The limitations of this type of solution are similar to those of cloud deployment solutions: it relies on static, offline external data sources / databases, and its service response performance is constrained by the user's network environment.

[0006] The third approach is model compression deployment, which reduces the computational and storage requirements of the model while maintaining its performance, and deploys it directly to the edge. Common model compression techniques include pruning, quantization, and knowledge distillation. Compared to the first two cloud-based solutions, this approach effectively reduces network latency. However, since the computing resources on the user's edge are not as abundant as those in the cloud or computing clusters, its performance in terms of model output quality and overall response performance may not be superior to the first two approaches. Furthermore, the retrieval enhancement generation of this approach currently still relies on API calls to existing search engines. While it can integrate local real-time application data and user interaction data, its effectiveness remains limited when local real-time and interaction data is scarce.

[0007] The fourth approach is containerized deployment, which packages the model and its dependencies into a single container, facilitating deployment in different environments. Containerized deployment improves efficiency and flexibility while reducing complexity. This approach is suitable for cross-computing environments, such as migrating models between different cloud data centers, but also for configuration within a single cloud or compute center cluster, or for migration from a cloud computing center to the user end. The limitation of this approach is that it is not optimized for real-time data updates and management in search enhancements, only providing convenience for model deployment and migration management.

[0008] In summary, the current deployment schemes for general models of retrieval enhancement still suffer from problems such as low response speed and lag in real-time data interaction.

[0009] Therefore, existing technologies still need improvement. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide a retrieval enhancement generation deployment method based on edge computing to address the shortcomings of existing technologies, thereby solving the problems of low response speed and lagging real-time data interaction in the deployment scheme of existing general models for retrieval enhancement generation.

[0011] The technical solution adopted by this invention to solve the technical problem is as follows:

[0012] In a first aspect, the present invention provides a retrieval enhancement generation and deployment method based on edge computing, comprising:

[0013] The container is a customized model corresponding to the application service requirements information for container deployment and retrieval published in the cloud; wherein, the container is a container customized and compressed by the cloud based on a general large model and corresponding application data.

[0014] The deployed custom model is fine-tuned using few-shot or zero-shot learning methods, and the knowledge base is retrieved and updated based on the fine-tuned custom model and user interaction data.

[0015] The local application service data knowledge base is updated based on the pre-processed cloud database, and the knowledge base data and related indexes are shared with other edge computing nodes in a peer-to-peer manner under data compliance constraints.

[0016] In one implementation, the customized model corresponding to the application service requirement information for container deployment and retrieval based on cloud-based deployment includes:

[0017] Collect the application service requirements information needed for the search, and register the service type with the cloud based on the collected requirements information;

[0018] The registration information is fed back to the cloud, and a customized container corresponding to the application service requirements information required for the search is obtained from the cloud.

[0019] The customized model is deployed based on the container, and model validation is performed.

[0020] In one implementation, the customized model is a model obtained by using knowledge distillation, model quantization, and model pruning techniques in the cloud to reduce the number of preset parameters, and by customizing and compressing the application data in the cloud.

[0021] In one implementation, the fine-tuning of the deployed custom model using a few-shot or zero-shot learning method includes, prior to:

[0022] Data for various interactive services is obtained from the cloud.

[0023] By combining locally stored application data and user preference data, a local application service data knowledge base is established.

[0024] Indexing the local application service data knowledge base using embedding models.

[0025] In an implementation, the fine-tuning of the deployed customized model using the few-shot or zero-shot learning method, and the knowledge base retrieval and updating based on the fine-tuned customized model and user interaction data, include:

[0026] Fine-tuning the customized model in the deployment container using the few-shot or zero-shot learning method with the data in the local application service data knowledge base, to obtain the fine-tuned customized model;

[0027] Creating a prompt word template required for service according to the application service requirement information required for retrieval;

[0028] Processing the user service dialogue and query based on the local application service data knowledge base, and filling the prompt word template through embedding model conversion and knowledge base retrieval;

[0029] Inference and output of results through the fine-tuned customized model, and transmission of the output results to the user end in a serialized manner;

[0030] Obtaining the user feedback data after differential privacy processing, and constructing an index based on the user feedback data, and updating the local application service data knowledge base.

[0031] In an implementation, the cloud database updates the local application service data knowledge base based on preprocessing, and shares the knowledge base data and related indexes with other edge computing nodes in a point-to-point manner under data compliance constraints, including:

[0032] Obtaining the preprocessed cloud database;

[0033] Updating the corresponding real-time index in the local application service data knowledge base according to the content index in the cloud database, and updating the corresponding knowledge base data;

[0034] Sharing the knowledge base data and related indexes with other edge computing nodes in a point-to-point manner under data compliance constraints, and performing data transmission and synchronization in a serialized manner.

[0035] In an implementation, the preprocessed cloud database is obtained by preprocessing and data format alignment of the collected multi-source heterogeneous data in the cloud, and establishing a corresponding content index; wherein the multi-source heterogeneous data includes multi-modal data and retrieval service interaction data.

[0036] In a second aspect, the present application provides a retrieval enhancement generation and deployment device based on edge computing, comprising:

[0037] a model deployment module configured to deploy a customized model corresponding to application service requirement information based on a container published by a cloud;

[0038] a retrieval and update module configured to fine-tune the deployed customized model by using a few-shot or zero-shot learning method, and perform knowledge base retrieval and update based on the fine-tuned customized model and user interaction data;

[0039] a data synchronization module configured to update a local application service data knowledge base based on a preprocessed cloud database, and share knowledge base data and related indexes with other edge computing nodes in a point-to-point manner under data compliance constraints.

[0040] In a third aspect, the present application provides a terminal comprising a processor and a memory, wherein the memory stores a retrieval enhancement generation and deployment program based on edge computing, and the retrieval enhancement generation and deployment program based on edge computing is configured to implement operations of the retrieval enhancement generation and deployment method based on edge computing when executed by the processor.

[0041] In a fourth aspect, the present application further provides a medium, which is a computer readable storage medium, and the medium stores a retrieval enhancement generation and deployment program based on edge computing, and the retrieval enhancement generation and deployment program based on edge computing is configured to implement operations of the retrieval enhancement generation and deployment method based on edge computing when executed by a processor.

[0042] The present application has the following effects by using the above technical solutions:

[0043] The present application deploys a customized model corresponding to application service requirement information based on a container published by a cloud, fine-tunes the deployed customized model by using a few-shot or zero-shot learning method, performs knowledge base retrieval and update based on the fine-tuned customized model and user interaction data, and updates a local application service data knowledge base based on a preprocessed cloud database. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from the structures shown in the drawings without creative labor.

[0045] Figure 1 is a flowchart of the retrieval enhancement generation deployment method based on edge computing in the present application.

[0046] Figure 2 is a deployment and service framework for general large model retrieval enhancement generation in an implementation manner of the present application.

[0047] Figure 3 is a schematic diagram of a machine learning model customization process driven by a large model provider in an implementation manner of the present application.

[0048] Figure 4 is a schematic diagram of a double feedback mode of an interactive service based on a general large model in an implementation manner of the present application.

[0049] Figure 5 is a functional schematic diagram of a terminal in an implementation manner of the present application.

[0050] The purposes, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0051] In order to make the purposes, technical solutions and advantages of the present application clearer and more explicit, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0052] Exemplary method

[0053] The current deployment scheme mainly relies on the following solutions: the first is a cloud deployment scheme, the service response performance of this scheme is subject to the network environment on the user side. The second is a distributed deployment scheme, the limitation of this scheme is similar to the cloud deployment scheme, which depends on static and offline external data sources / databases, and its service response performance is subject to the network environment on the user side. The third is a model compression deployment scheme, the retrieval enhancement generation of this scheme currently still relies on the interface call of the existing search engine, which can integrate local real-time application data and user interaction data, but when the local real-time and interaction data are limited, its effect is still very limited. The fourth is a containerized deployment scheme, the limitation of this scheme is that it does not optimize the real-time data update and management for retrieval enhancement generation, but only provides the convenience of model deployment and migration management.

[0054] In summary, the current deployment scheme of the general model of retrieval enhancement generation still has the problems of low response speed and lag of real-time data interaction.

[0055] To solve the above technical problems, the embodiment of the present application provides a retrieval enhancement generation deployment method based on edge computing. The method deploys a customized model corresponding to the application service requirement information required for retrieval based on the container published by the cloud, fine-tunes the deployed customized model using a few-shot or zero-shot learning method, performs knowledge base retrieval and update based on the fine-tuned customized model and user interaction data, and updates the local application service data knowledge base based on the preprocessed cloud database. The embodiment of the present application realizes dynamic data collection and processing based on the cloud and edge computing nodes, ensures that the model service can use the latest and diversified data, and improves the response speed and output quality of the retrieval service.

[0056] As shown in Figure 1 , the embodiment of the present application provides a retrieval enhancement generation deployment method based on edge computing, which includes the following steps:

[0057] Step S100, deploying a customized model corresponding to the application service requirement information required for retrieval based on the container published by the cloud; wherein the container is a container obtained by customizing and compressing the general large model and the corresponding application data by the cloud.

[0058] In the embodiment, a general large model deployment service framework integrating retrieval enhancement generation technology is provided (as shown in Figure 2 ), which aims to optimize the retrieval enhancement generation performance and improve the output quality and response speed of the general large model in different customized services such as user dialogue, query and generation by using the heterogeneous computing power, network and real-time data resources of the edge computing environment.

[0059] The core idea in the embodiment is to combine containerization and knowledge distillation technology, fine-tune the general large model in the cloud, package different customized models into containers and deploy the containers to high-performance edge intelligent servers, to provide fast response, customized end-side / near-end-side inference or dialogue service for services with different application requirements. At the same time, real-time data collection is performed from multiple perception data sources and user interaction service processes combined with differential privacy technology, and a distributed database is established around the collected data from each source in the compliant data management domain, to provide sufficient, efficient and real-time context for retrieval enhancement generation and improve the output quality of the model service.

[0060] The detailed technical scheme in the embodiment includes three core stages (as shown in Figure 3 ). Among them, the second and third stages are based on the double feedback mode specific to the interactive service of the general large model (as shown in Figure 4(As shown). In the dual-feedback model, there are two feedback loops: an inner loop and an outer loop. The inner loop exists between the user and the model. The user provides prompts or responses to the model, which then generates a response based on the user's input and previous interaction data, adding the response to its context window. The outer loop exists between the model service and the knowledge base. The model leverages the relevant context provided by the knowledge base to provide more stable and higher-quality output, while simultaneously recording the model's interaction history with the user in the knowledge base, thus constructing richer contextual data.

[0061] Specifically, in one implementation of this embodiment, step S100 includes the following steps:

[0062] Step S101: Collect the application service requirement information for the search, and register the interactive service type with the cloud based on the collected requirement information;

[0063] Step S102: Feed back the registration information to the cloud and obtain the customized container published by the cloud that corresponds to the application service requirement information required for the search.

[0064] Step S103: Deploy the customized model based on the container and perform model verification.

[0065] In this embodiment, the first stage involves the construction of a cloud database, compression of a general cloud model, and deployment of a customized model on edge computing nodes. The customized model is a model obtained by using knowledge distillation, model quantization, and model pruning techniques in the cloud to reduce the number of preset parameters and by customizing and compressing the application data in the cloud. After compression, the customized model is packaged into a container, and the cloud deploys the container to various edge computing nodes.

[0066] The following combination Figure 3 The interaction process between the cloud, edge computing nodes, and the user terminal describes the first stage of the edge computing-based retrieval enhancement generation and deployment method.

[0067] In this phase, the first step is to deploy a general-purpose large model (such as...) in the cloud. Figure 3 Step 1.1 in the previous section states that this general-purpose model possesses broad application potential and powerful cross-task processing capabilities, enabling it to provide users with generative interactive services. Then, an index database (such as...) is built in the cloud. Figure 3 Step 1.2 in the process involves storing a large number of data entries using forward indexing, reverse indexing, or block indexing techniques based on embedded models to support retrieval enhancement generation techniques. To ensure efficient operation on edge computing nodes, the general-purpose large model is quantized and compressed in the cloud (e.g., ...). Figure 3Step 1.6 in FIG. 1, for example, by using techniques such as knowledge distillation, model quantization, and model pruning to appropriately reduce the number of parameters, and by using relevant application data in the cloud database for customization, the compressed customized model is packaged into a container (such as Figure 3 Step 1.7 in FIG. 1, to prepare for deployment to various edge computing nodes.

[0068] At the same time, the edge computing nodes start collecting their required application service requirements (such as Figure 3 Step 1.3 in FIG. 1, and interact with the cloud for service type registration (such as Figure 3 Step 1.4 in FIG. 1, which helps the edge nodes to clearly define the types and requirements of their services. The edge nodes feed back the registration information to the cloud (such as Figure 3 Step 1.5 in FIG. 1, and the cloud issues corresponding containers to the edge computing nodes (such as Figure 3 Step 1.8 in FIG. 1). After receiving the containers, the edge computing nodes start deploying the general models (such as Figure 3 Step 1.9 in FIG. 1, and perform model verification (such as Figure 3 Step 1.10 in FIG. 1), to ensure the correct operation of the models on the edge nodes. Finally, the edge nodes and the cloud confirm the deployment and service information (such as Figure 3 Step 1.11 in FIG. 1), to ensure that the entire deployment process is error-free and all steps are successfully completed.

[0069] In this embodiment, containerization and knowledge distillation are combined to achieve efficient deployment and flexible management of general models in edge computing environments. Unlike existing containerization deployment solutions that mainly focus on centralized computing environments, this embodiment realizes orchestration optimization in conjunction with the service types of different edge computing nodes and the distribution of knowledge bases during container distribution.

[0070] As shown in Figure 1 In an implementation manner of the embodiment of the present application, the retrieval enhancement generation and deployment method based on edge computing further includes the following steps:

[0071] Step S200, fine-tuning the deployed customized model using a few-shot or zero-shot learning method, and performing knowledge base retrieval and updating based on the fine-tuned customized model and user interaction data.

[0072] Specifically, in an implementation manner of the embodiment, the following steps are included before step S200:

[0073] Step S201a, obtaining data for various interactive services from the cloud;

[0074] Step S201b, establishing a local application service data knowledge base in combination with locally stored application data and user preference data;

[0075] Step S201c: Use the embedding model to create an index for the local application service data knowledge base.

[0076] In one implementation of this embodiment, step S200 includes the following steps:

[0077] Step S201: Using data from the local application service data knowledge base, fine-tune the customized model in the deployment container using a few-shot or zero-shot learning method to obtain the fine-tuned customized model.

[0078] Step S202: Create a prompt word template for the service based on the application service requirements information retrieved;

[0079] Step S203: Process user service dialogues and queries based on the local application service data knowledge base, and populate the prompt word template by embedding model transformation and knowledge base retrieval.

[0080] Step S204: Perform inference using the fine-tuned customized model and output the results, then transmit the output results to the user terminal in a serialized manner;

[0081] Step S205: Obtain user feedback data after differential privacy processing, and build an index based on the user feedback data to update the local application service data knowledge base.

[0082] In this embodiment, the second stage involves the process of fine-tuning the customized model, providing retrieval services based on the fine-tuned model, and updating the model based on service feedback.

[0083] The following combination Figure 3 The interaction process between the cloud, edge computing nodes, and the user terminal describes the second stage of the edge computing-based retrieval enhancement generation and deployment method.

[0084] At this stage, for various interactive services, each edge computing node (such as edge servers deployed in a community or building unit in a smart home scenario) subscribes to the cloud for relevant data of the type of service it serves (such as... Figure 3 Step 2.4) combines relevant application data and user preference data stored locally on each edge computing node to establish a local application service data knowledge base (such as...). Figure 3 Step 2.3 in the process, and using the embedding model to build an index for the local application service data knowledge base (e.g., Figure 3 Step 2.6 in the previous section). Next, each edge computing node uses data from its local application service data knowledge base to fine-tune the general model (i.e., the customized model) in the edge computing node deployment container using a few-shot or zero-shot learning method (e.g., ...). Figure 3Step 2.7 in the process is to optimize the service quality of the model.

[0085] Each edge computing node, after deploying its tuned services (such as...) Figure 3 Step 2.8) involves creating the required prompt word templates for the service based on service needs (e.g., ...). Figure 3 (Step 2.9 in the previous section) These templates will help customize the model to generate more accurate responses. For user service dialogues and queries, processing can be done locally on each edge computing node (e.g., ...). Figure 3 Step 2.10 in the process involves transformation via embedded model (e.g.) Figure 3 Step 2.11) and knowledge base retrieval (e.g.) Figure 3 In step 2.12), populate the prompt word template to generate a response (such as...). Figure 3 (Step 2.13 in the text).

[0086] Customize the model to perform inference and output results (e.g.) Figure 3 In step 2.14), the output is serialized and transmitted to the user terminal (e.g., ...). Figure 3 Step 2.15). User feedback data is backed up (e.g., ...). Figure 3 Step 2.16 in the document, and undergoes compliance review and authorization (e.g. Figure 3 Step 2.17 in the document. Differential privacy processing (e.g., ...) is applied to user feedback data. Figure 3 Step 2.18) and collect data to edge computing nodes (such as...) Figure 3 Step 2.19 in the text involves indexing and constructing the real-time feedback data at the edge computing node (e.g., ...). Figure 1 Step 2.20 in the document, and update the local application service data knowledge base (e.g., ...). Figure 3 Step 2.21) ensures that the data in the local application service data knowledge base is always up-to-date.

[0087] This embodiment differs from existing search enhancement generation technologies based solely on the cloud or user mobile devices. By integrating the heterogeneous computing power, network resources, and local data of edge computing nodes in the edge computing environment, it can provide faster service response, customized service output, and protection of user data privacy compared to cloud-based solutions.

[0088] like Figure 3 As shown, in one implementation of this invention, the edge computing-based retrieval enhancement generation deployment method further includes the following steps:

[0089] Step S300: Update the local application service data knowledge base based on the pre-processed cloud database, and share the knowledge base data and related indexes with other edge computing nodes in a point-to-point manner under data compliance constraints.

[0090] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0091] Step S301: Obtain the preprocessed cloud database;

[0092] Step S302: Update the corresponding real-time index in the local application service data knowledge base according to the content index in the cloud database, and update the corresponding knowledge base data.

[0093] Step S303: Under data compliance constraints, knowledge base data and related indexes are shared with other edge computing nodes in a point-to-point manner, and data transmission and synchronization are performed through serialization.

[0094] In this embodiment, in the third stage, the cloud preprocesses the collected multi-source heterogeneous data to obtain a preprocessed cloud database. Then, based on the dual feedback mode of the retrieval enhancement generation technology of the general large model, the local knowledge base of each edge computing node is updated. The preprocessed cloud database is the database obtained by the cloud after preprocessing and aligning the data format of the collected multi-source heterogeneous data, and then establishing corresponding content indexes. The multi-source heterogeneous data includes: multiple modal data and retrieval service interaction data.

[0095] The following combination Figure 3 The interaction process between the cloud, edge computing nodes, and the user terminal describes the third stage of the edge computing-based retrieval enhancement generation and deployment method.

[0096] During this phase, in addition to edge computing nodes updating their local application service data knowledge base using user interaction data, the cloud data center simultaneously collects multimodal data from different external data sources (such as...). Figure 3 Step 3.1) involves data in multiple modalities, including images, videos, text, and numerical values, as well as service interaction data authorized by the user. After collecting this multimodal data in the cloud, it is added to the cloud database according to the corresponding application service categories to establish a unified embedding model representation (e.g., ...). Figure 3 (Step 3.2 in the text).

[0097] Meanwhile, after obtaining compliance review and authorization of user data (such as...), the cloud... Figure 3 Step 3.1) involves performing differential privacy processing on user data (e.g., ...). Figure 3 Step 3.2) and collect data to the cloud (e.g., ...). Figure 3 Step 3.3). The cloud performs preprocessing on the collected multi-source heterogeneous data (e.g., ...). Figure 3 Step 3.4) and data format alignment (e.g. Figure 3Step 3.5 in FIG. 3, to establish relevant content index (e.g. Figure 3 Step 3.6 in FIG. 3, to facilitate subsequent indexing and retrieval.

[0098] Then, according to the service type subscribed by each edge computing node in the second stage, the cloud data center pushes the updated relevant data to the local application service data knowledge base of each edge computing node (e.g. Figure 3 Step 3.7 in FIG. 3, updates the corresponding real-time index in the local application service data knowledge base of each edge computing node (e.g. Figure 3 Step 3.8 in FIG. 3, and updates its local application service data knowledge base (e.g. Figure 5 Step 3.9 in FIG. 3). During this process, the knowledge base data and related indexes can also be shared between different edge computing nodes in a point-to-point manner under data compliance constraints (e.g. Figure 5 Step 3.10 in FIG. 3, and the knowledge base data can be transmitted and synchronized through serialization (e.g. ​ Step 3.11 in FIG. 3).

[0099] Through the above steps, the cloud and edge computing nodes achieve dynamic data collection and processing, ensuring that the model service can utilize the latest and diversified data to improve the response speed and output quality of the service.

[0100] This embodiment is based on the double feedback mode of retrieval enhancement generation technology of general large models, fully utilizes the data distributed in edge computing nodes, interactive service data and third-party data source data, uses embedding models to establish real-time indexes for the collected new data, and improves the real-time and reliability of interactive service context information. In contrast, existing solutions mainly rely on static, offline retrieval data sets or knowledge bases that have been constructed, and the data generated during user interaction is only used to optimize the quality of the current interactive service, without fully utilizing these real-time information and third-party update data.

[0101] In another implementation manner of the embodiment, for the three stages of the above edge computing-based retrieval enhancement generation deployment method, any one of the following variations can be made to obtain the corresponding variation scheme:

[0102] 1) Dynamic model selection and switching: Introduce a dynamic model selection and switching mechanism on the edge computing node to cope with different application scenarios and user needs. By monitoring the current system state and user request type, the most suitable model version or architecture is intelligently selected and switched. For example, in the case of high query density, a lightweight model optimized for performance can be selected, while in the case of high precision generation, a full-featured large model is switched to. This can improve the flexibility and adaptability of the system, ensuring optimal performance under diverse service demands.

[0103] 2) Hybrid Cloud-Edge Deployment: Adopting a hybrid cloud-edge deployment scheme, combining the advantages of cloud and edge computing nodes to optimize resource utilization and improve system performance. Complex calculations and data processing are performed in the cloud, including large-scale data training and model updates, while real-time inference and user interaction tasks are handled on edge nodes. In this way, the computing load can be balanced, network latency can be reduced, efficient resource management and fast response can be achieved, which is particularly suitable for application scenarios that require high real-time performance and low latency.

[0104] 3) Federated Learning and Collaborative Training: Utilize federated learning techniques to enable collaborative training between edge computing nodes, sharing model parameters or updates without centralized data storage. Each edge computing node trains the model locally and then shares updates through secure aggregation protocols, improving overall model performance while protecting data privacy. This not only enhances data privacy and security, reduces data transmission costs, but also enables cross-node collaborative optimization, particularly suitable for application fields such as medical and financial fields where data is scattered and sensitive.

[0105] 4) Multi-modal Data Fusion: Integrate multiple modal data such as text, images, audio, etc. on edge computing nodes to improve the diversity and accuracy of generation and retrieval. By comprehensively analyzing data from different perception sources, the system can better understand user needs and provide more intelligent services. For example, in smart home applications, voice commands, video surveillance, and sensor data can be combined to achieve more intelligent home control and automation. This multi-modal data fusion method can enhance the robustness and adaptability of the system, supporting more complex application scenarios.

[0106] 5) Real-time Adaptive Optimization: Implement real-time adaptive optimization on edge computing nodes, dynamically adjusting model inference and data processing strategies based on current network conditions, computing resources, and user needs. The system can monitor and evaluate environmental changes in real time, automatically adjusting resource allocation and computational complexity to ensure optimal response speed and service quality. For example, when network conditions are poor, reduce the complexity of model inference to ensure service continuity and user experience. Through this real-time adaptive optimization, the system's real-time performance and flexibility can be improved to meet application needs in various complex and dynamic environments.

[0107] 6) Intelligent caching and prefetching strategy: Implement intelligent caching and prefetching strategy at the edge computing node, predict user behavior and historical data in advance to cache or prefetch data or models that may be needed. Based on user's browsing history and usage habits, the system can intelligently predict user's needs and pre-load relevant resources at the right time, thereby reducing response time and improving system efficiency. For example, in e-commerce platform applications, relevant product information and recommendation models can be pre-fetched based on user's browsing history and purchase behavior to provide faster and personalized user experience. This intelligent caching and prefetching strategy can significantly improve system performance and user satisfaction.

[0108] The embodiment achieves the following technical effects through the above technical solutions:

[0109] The embodiment deploys a customized model corresponding to the application service requirement information required by retrieval based on a container published by the cloud, fine-tunes the deployed customized model using a few-shot or zero-shot learning method, and can perform knowledge base retrieval and updating based on the fine-tuned customized model and user interaction data, and update the local application service data knowledge base based on the preprocessed cloud database. The embodiment realizes dynamic data collection and processing based on the cloud and edge computing nodes, ensures that the model service can use the latest and diversified data, and improves the response speed and output quality of the retrieval service.

[0110] Exemplary device

[0111] Based on the above embodiment, the application further provides a retrieval enhancement generation and deployment device based on edge computing, comprising:

[0112] A model deployment module is configured to deploy a customized model corresponding to application service requirement information required by retrieval based on a container published by the cloud. The container is obtained by customizing and compressing a general large model and corresponding application data based on the cloud.

[0113] A retrieval and updating module is configured to fine-tune the deployed customized model using a few-shot or zero-shot learning method, and perform knowledge base retrieval and updating based on the fine-tuned customized model and user interaction data.

[0114] A data synchronization module is configured to update a local application service data knowledge base based on a preprocessed cloud database, and share knowledge base data and related indexes with other edge computing nodes in a point-to-point manner under data compliance constraints.

[0115] The embodiment achieves the following technical effects through the above technical solutions:

[0116] The embodiment based on the cloud releases a customized model corresponding to the application service requirement information required by container deployment and retrieval, and adopts a few-sample or zero-sample learning method to fine-tune the deployed customized model, and can perform knowledge base retrieval and updating based on the fine-tuned customized model and user interaction data, and update the local application service data knowledge base based on the preprocessed cloud database. The embodiment based on the cloud and the edge computing node realizes dynamic data collection and processing, ensures that the model service can use the latest and diversified data, and improves the response speed and output quality of the retrieval service.

[0117] Based on the above embodiment, a terminal is also provided, and a principle block diagram thereof can be as shown in ​

[0118] The terminal includes a processor, a memory, an interface, a display screen and a communication module connected through a system bus. The processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a storage medium and an internal memory. The storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the storage medium to run. The interface is used to connect external devices. The display screen is used to display corresponding information. The communication module is used to communicate with a cloud server or other devices.

[0119] The computer program is executed by the processor to implement the operations of the retrieval enhancement generation and deployment method based on edge computing.

[0120] Those skilled in the art can understand that, ​ The principle block diagram shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific terminal can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0121] In one embodiment, a terminal is provided, which includes a processor and a memory. The memory stores a retrieval enhancement generation and deployment program based on edge computing. The retrieval enhancement generation and deployment program based on edge computing is executed by the processor to implement the operations of the retrieval enhancement generation and deployment method based on edge computing as above.

[0122] In one embodiment, a storage medium is provided, which stores a retrieval enhancement generation and deployment program based on edge computing. The retrieval enhancement generation and deployment program based on edge computing is executed by the processor to implement the operations of the retrieval enhancement generation and deployment method based on edge computing as above.

[0123] ​Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments of the present application can include non-volatile and volatile memory.

[0124] In summary, the present application provides a retrieval enhancement generation deployment method based on edge computing, which comprises: deploying a customized model corresponding to the application service requirement information required for retrieval based on a container published by a cloud; wherein the container is obtained after the cloud customizes and compresses a general large model and corresponding application data; fine-tuning the deployed customized model using a few-shot or zero-shot learning method, and performing knowledge base retrieval and updating based on the fine-tuned customized model and user interaction data; updating the local application service data knowledge base based on the preprocessed cloud database, and sharing the knowledge base data and related index with other edge computing nodes in a point-to-point manner under data compliance constraints; the present application realizes dynamic data collection and processing based on the cloud and edge computing nodes, ensures that the model service can use the latest and diversified data, and improves the response speed and output quality of the retrieval service.

[0125] It should be understood that the application of the present application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes should belong to the protection scope of the claims of the present application.

Claims

1. An edge-computing-based retrieval augmentation generation deployment method, characterized by, The method comprises the following steps: a container is deployed and a customized model corresponding to the application service requirement information to be retrieved is retrieved based on the cloud; wherein the container is obtained by customizing and compressing a general large model and corresponding application data in the cloud; a few-shot or zero-shot learning method is used to fine-tune the deployed customized model, and a knowledge base is retrieved and updated based on the fine-tuned customized model and user interaction data; a local application service data knowledge base is updated based on a preprocessed cloud database, and knowledge base data and related indexes are shared with other edge computing nodes in a point-to-point manner under data compliance constraints; the method of fine-tuning the deployed customized model using a few-shot or zero-shot learning method and updating the knowledge base based on the fine-tuned customized model and user interaction data comprises: using the data in the local application service data knowledge base, a few-shot or zero-shot learning method is used to fine-tune the customized model in the deployed container to obtain the fine-tuned customized model; a prompt word template required for creating a service based on the retrieved application service requirement information is created; the local application service data knowledge base is used to process user service dialogues and queries, and the prompt word template is filled in through embedded model conversion and knowledge base retrieval; the fine-tuned customized model is used for reasoning and outputting results, and the output results are transmitted to the user end in a serialized manner; differential privacy processed user feedback data is obtained, and the local application service data knowledge base is updated based on the index construction of the user feedback data. 2.The edge-computing-based retrieval augmentation generation deployment method according to claim 1, wherein, the method of deploying and retrieving a customized model corresponding to the application service requirement information to be retrieved based on the cloud comprises: collecting the application service requirement information to be retrieved, and registering interactive service types with the cloud based on the collected requirement information; feedback the registration information to the cloud, and obtain the customized container corresponding to the application service requirement information to be retrieved published by the cloud; deploy the customized model based on the container, and perform model verification. 3.The edge-computing-based retrieval augmentation generation deployment method according to claim 1, wherein, The customized model is obtained by reducing the preset parameter quantity of the cloud using knowledge distillation, model quantization and model pruning techniques, and customizing and compressing the application data in the cloud. 4.The edge-computing-based retrieval augmentation generation deployment method according to claim 1, wherein, the method of fine-tuning the deployed customized model using a few-shot or zero-shot learning method comprises: obtaining data for various interactive services from the cloud; establishing a local application service data knowledge base by combining locally stored application data and user preference data; using an embedded model to index the local application service data knowledge base.

5. The edge-computing-based retrieval augmentation generation deployment method according to claim 1, characterized in that, the method of updating the local application service data knowledge base based on the preprocessed cloud database and sharing the knowledge base data and related indexes with other edge computing nodes in a point-to-point manner under data compliance constraints comprises: obtaining the preprocessed cloud database; updating the corresponding real-time index in the local application service data knowledge base according to the content index in the cloud database, and updating the corresponding knowledge base data; The knowledge base data and related index are shared with other edge computing nodes in a point-to-point manner under data compliance constraints, and data transmission and synchronization are performed in a serialized manner. 6.The edge-computing-based retrieval augmentation generation deployment method according to claim 1, wherein, The pre-processed cloud database is obtained by pre-processing and data format alignment of collected multi-source heterogeneous data, and establishing a corresponding content index; the multi-source heterogeneous data includes multi-modal data and retrieval service interaction data.

7. An edge-computing-based retrieval enhancement generation deployment apparatus for implementing the edge-computing-based retrieval enhancement generation deployment method according to any one of claims 1-6, characterized by The method comprises the following steps: The model deployment module is used to deploy and retrieve the customized model corresponding to the application service requirement information based on the cloud-published container; the container is obtained by customizing and compressing the general large model and corresponding application data in the cloud; The retrieval and updating module is used to fine-tune the deployed customized model by using a few-shot or zero-shot learning method, and to perform knowledge base retrieval and updating based on the fine-tuned customized model and user interaction data; The data synchronization module is used to update the local application service data knowledge base based on the pre-processed cloud database, and to share the knowledge base data and related index with other edge computing nodes in a point-to-point manner under data compliance constraints.

8. A terminal, characterized by comprising: The method comprises the following steps: The processor and the memory, the memory stores the edge computing-based retrieval enhancement generation deployment program, the edge computing-based retrieval enhancement generation deployment program is executed by the processor to realize the operation of the edge computing-based retrieval enhancement generation deployment method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores the edge computing-based retrieval enhancement generation deployment program, the edge computing-based retrieval enhancement generation deployment program is executed by the processor to realize the operation of the edge computing-based retrieval enhancement generation deployment method in any one of claims 1-6.

Images