System and method for optimizing few-sample visual model based on end-cloud collaboration

Abstract

The invention relates to a few-sample visual model optimization system and method based on end-cloud collaboration. The system comprises edge equipment and a cloud server. The edge device collects an environment image of the current deployment environment and sends the environment image to the cloud server; the cloud server is configured to extract an image sample from a pre-training data set after receiving the environment image, and fuse the image sample with the environment image based on a block-level mixing mode to generate an environment perception training sample; adjusting and optimizing the backbone network based on an environment perception training sample and an optimization strategy including mixed perception contrast learning and a multi-task auxiliary target to generate a lightweight adjusting and optimizing module; sending the generated tuning module to the edge device; and the edge device loads the tuning module to perform a few-sample reasoning task. The system provided by the invention has the advantages of low training and communication overhead, no need of extra annotation information, remarkable tuning effect and the like, and can be widely applied to edge intelligent vision application with frequent environment change and limited samples.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, and in particular to a few-shot visual model optimization system and method based on edge-cloud collaboration. Background Technology

[0002] Currently, the deployment of visual models on edge devices has become a mainstream trend, especially in practical applications such as intelligent security and industrial inspection. To address the high cost and difficulty in acquiring data annotation, few-shot learning techniques have been widely adopted to improve the generalization ability of visual models under data-scarce conditions. However, pre-trained visual models often experience significant performance degradation in new deployment environments, particularly when there are large differences in background information, lighting conditions, or interference, making it difficult for the robustness of the visual models to meet practical needs. For example, in intelligent surveillance scenarios, visual models need to classify monitored targets, but complex background elements in the deployment environment (such as office equipment and greenery) can easily distract the model, causing key features to be interfered with by noise and reducing recognition accuracy.

[0003] Existing solutions largely focus on interference suppression or attention mechanism optimization at the model feature layer, improving the environment offset problem by reducing the impact of background noise. While these methods can alleviate performance degradation to some extent, they typically lead to high inference overhead and struggle to achieve real-time adaptation to dynamic environments, ultimately limiting further improvements in visual model performance. Therefore, research has shifted towards enabling visual models to adapt to the environment after deployment through lightweight fine-tuning strategies. However, existing fine-tuning methods generally suffer from two bottlenecks: firstly, they rely on a large amount of labeled data; secondly, they require thorough environment modeling before deployment, which is difficult to achieve in real-world scenarios where edge devices have limited computing resources and the deployment environment is constantly changing.

[0004] For example, CN120542505A discloses an optimization method and system for an edge vision AI neural network model. The method includes: performing hierarchical adaptive analysis of visual features on the input image to obtain the visual feature importance distribution, and constructing an initial neural network model adapted to edge devices based on the visual feature importance distribution; performing parameter optimization on the initial neural network model to balance energy consumption and accuracy, and obtaining edge device optimization parameters; training the initial neural network model to obtain a trained edge vision model; performing model pruning for visual semantic perception on the trained edge vision model to obtain a target network structure; and performing deployment optimization based on hardware characteristics to obtain a vision model that runs efficiently on edge devices. This enables the model to adapt to the heterogeneous characteristics of different edge computing platforms, improving the feasibility of applying visual AI technology on diverse edge devices.

[0005] For example, CN119294451A discloses a visual model operator optimization method and device for domestically produced hardware, applied to visual models. The method includes: extending the visual model to multiple position encoding methods based on the relative position encoding and conditional position encoding of the relative position encoding module; performing position encoding on the visual model based on the multiple position encoding methods to obtain the structurally optimized target visual model corresponding to each position encoding method; inputting a preset target image into a target visual model; performing operator abstraction on the target visual model to obtain the corresponding abstract target visual model; obtaining the correspondence between neural network operators and domestically produced hardware operators provided by domestic hardware; and implementing the neural network operators on domestic hardware based on the correspondence.

[0006] This invention provides a few-shot visual model optimization system and method based on edge-cloud collaboration. By performing lightweight fine-tuning on the model, it can actively adapt to the current environment after deployment, thereby improving classification performance.

[0007] Furthermore, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making this invention, but due to space limitations, not all details and contents were listed in detail. However, this does not mean that the present invention does not possess the features of these prior art. On the contrary, the present invention already possesses all the features of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0008] Current mainstream methods mainly focus on suppressing interference in the model's feature layers or optimizing its attention mechanism, aiming to reduce background noise to cope with the impact of environmental changes. While these techniques can partially alleviate the performance degradation of visual models, they are often accompanied by high inference costs and limited real-time response capabilities to dynamic scenes, thus restricting continuous performance improvement. Therefore, research has gradually shifted towards exploring lightweight fine-tuning strategies, hoping to endow visual models with the ability to autonomously adapt to new environments after deployment. However, existing fine-tuning schemes still face two major challenges: first, they typically require a large number of labeled image samples for training; second, they rely on complete modeling of the target environment before deployment, which is difficult to achieve under real-world conditions where edge computing device resources are scarce and application scenarios frequently change.

[0009] The purpose of this invention is to provide an efficient, low-cost, and fast-execution system and method for optimizing few-sample visual models, which achieves environmental perception and visual model adaptation through an edge-cloud collaborative mechanism.

[0010] To address the shortcomings of existing technologies, this invention provides a few-shot visual model optimization system based on edge-cloud collaboration. The system includes an edge device and a cloud server. The edge device acquires environmental images of the current deployment environment and sends them to the cloud server. The cloud server is configured to: upon receiving the environmental images, extract image samples from a pre-training dataset and fuse them with the environmental images using a block-level fusion method to generate environmental perception training samples; optimize the backbone network based on the environmental perception training samples and an optimization strategy that includes hybrid perception contrastive learning and multi-task auxiliary objectives to generate a lightweight optimization module; send the generated optimization module to the edge device; and load the optimization module onto the edge device to perform few-shot inference tasks.

[0011] This invention presents a cloud-edge collaborative few-shot visual model optimization system that effectively addresses the technical shortcomings of existing methods in terms of inference cost, real-time response capability, and reliance on labeled data and prior modeling. To address the high inference overhead caused by the complex interference suppression or attention mechanism optimization at the feature layer in mainstream methods, this invention shifts the main computational burden to the cloud server. Edge devices only need to perform lightweight tuning module loading and inference, significantly reducing local computing resource consumption and improving deployment feasibility and response efficiency under resource-constrained conditions. Simultaneously, to address the issue of insufficient model adaptability in dynamic scenes, the system dynamically generates environmental perception training samples on the cloud server by real-time acquisition of environmental images of the current deployment environment, enabling rapid capture and modeling of new environmental features and enhancing the model's robustness and real-time adaptability in changing scenarios. More importantly, this invention employs a block-level hybrid approach to fuse pre-training samples with real-world images, requiring only a small number of unlabeled or weakly labeled samples to construct effective training data. This overcomes the dependence of traditional fine-tuning methods on large amounts of labeled data. Furthermore, it eliminates the need for complete modeling of the target environment before deployment, achieving on-demand model updates through online perception and incremental optimization, significantly reducing reliance on prior knowledge. Overall, this invention balances computational efficiency and deployment flexibility while ensuring the model's continuous adaptability, providing a feasible path for the long-term stable operation of edge vision systems in complex and ever-changing environments.

[0012] According to a preferred embodiment, the steps of generating environment-aware training samples by the cloud server include: after receiving an environment image, randomly selecting image samples and their corresponding labels from a pre-training dataset to form a subset of cloud data; for each image sample, processing it based on a random data augmentation method and generating two different views; randomly cropping the environment image to obtain a cropped environment image; mixing one of the views with the environment information in the cropped environment image; and using the generated environment-aware hybrid view as an environment-aware training sample.

[0013] By randomly sampling pre-training samples and generating dual views, combined with cropping and block-level blending of environmental images, this approach effectively integrates prior knowledge and real-world scene features, enhancing the diversity and representativeness of environmental perception. The random augmentation and blending strategies enhance model robustness without additional annotation, supporting efficient fine-tuning with few samples, reducing dependence on labeled data and complete environment modeling, while ensuring the quality and speed of training sample generation. According to a preferred implementation, the steps for constructing the optimization strategy on the cloud server include: constructing positive and negative sample pairs in contrastive learning; calculating the vector representations of positive and negative sample pairs in the latent space based on the positive and negative sample pairs, the backbone network, and the projector; calculating the mixed-sensory contrastive loss of the image sample based on the vector representations obtained from two different perspectives of the same image sample in the latent space; calculating the overall mixed-sensory contrastive loss based on the mixed-sensory contrastive loss of several image samples; and calculating the multi-task-assisted mixed-sensory contrastive loss based on the overall mixed-sensory contrastive loss when learning multiple tasks.

[0014] This optimization strategy constructs a hybrid perception contrastive learning mechanism, generating positive sample pairs using different views of the same image and constructing negative sample pairs across sample views. This strengthens the model's ability to discriminate environmental changes, effectively mitigates background noise interference, and improves feature robustness. Compared to traditional attention optimization methods, this strategy is performed in the cloud, avoiding increased computational burden at the edge and reducing inference costs. Furthermore, contrastive learning does not rely on densely labeled data. Combined with multi-task auxiliary objectives, it can achieve cross-task knowledge transfer using a small number of samples, alleviating the dependence on large amounts of labeled data and pre-deployment environmental modeling, and significantly improving the model's adaptability and generalization performance in dynamic scenarios.

[0015] According to a preferred embodiment, the steps for an edge device to perform a few-shot inference task include: fusing the tuning module with the local backbone network to form a final adapted visual model.

[0016] This invention integrates the tuning module with the local backbone network, introducing lightweight parameter updates only at the edge devices. This avoids retraining the entire visual model and high-cost feature reconstruction, significantly reducing inference latency, improving response efficiency in dynamic scenes, and effectively alleviating the high computational cost problem caused by complex attention mechanism optimization in traditional methods. This integration method does not require modification of the original backbone structure, maintaining the stability of the visual model. At the same time, the tuning module generated by the cloud server injects environmental adaptive knowledge, enabling the visual model to quickly adapt to new environments with a small number of samples. This breaks through the dependence of traditional fine-tuning on a large amount of labeled data and prior environment modeling, achieving efficient, flexible, and low-resource-consumption continuous optimization after deployment.

[0017] According to a preferred embodiment, the optimization strategy of the optimization module includes: the contrastive learning part aggregates similar samples at low temperature and improves the separation of dissimilar samples by temperature to achieve feature space optimization; and the original knowledge of the visual model is maintained based on the multi-task auxiliary target.

[0018] The optimization strategy of this module enhances feature aggregation between similar samples through low temperature and promotes separation of dissimilar samples through high temperature, effectively optimizing the feature space distribution, improving the model's ability to discriminate background noise, and alleviating the high inference cost caused by traditional feature layer interference suppression. Simultaneously, based on multi-task auxiliary objectives, it preserves the original knowledge of the backbone network, avoiding knowledge forgetting during fine-tuning and ensuring the generalization ability of the visual model when adapting to new environments with limited samples. This optimization strategy does not require a large amount of labeled data or complete environment modeling; it can achieve accurate and rapid model optimization with only a small number of samples, significantly enhancing the adaptability and deployment flexibility of the visual model in edge scenarios.

[0019] According to a preferred embodiment, the tuning strategy of the tuning module further includes: incrementally adjusting the parameters of key layers in the backbone network by inserting a set of lightweight low-rank matrices without modifying the original visual model structure.

[0020] This invention integrates the tuning module with the existing visual model via parameter injection, without modifying the network structure or increasing the inference computation burden. Edge devices only need to load this tuning module to complete the visual model adaptation, enabling the visual model to quickly acquire environmental perception capabilities in new environments, enhancing the recognition and focusing of foreground targets, while effectively suppressing background noise interference and improving recognition accuracy and inference efficiency under conditions of limited samples.

[0021] According to a preferred embodiment, the calculation formula for generating the environment-aware hybrid view by the cloud server is as follows: ;in, This represents an environment-aware hybrid view. A binary mask representing random sampling. Indicates from image samples Views built using random data augmentation This represents environmental information, specifically the cropped environmental image. This indicates element-wise multiplication.

[0022] This computational method enhances the view and environment images through mask fusion, eliminating the need for complex structural modifications or dense annotations and significantly reducing reliance on prior modeling. It injects environmental context while preserving foreground semantics, improving the visual model's robustness to interference, avoiding the high inference costs of traditional attention mechanisms, and supporting rapid adaptation to dynamic scenes with limited samples, balancing efficiency and generalization.

[0023] According to a preferred embodiment, the formula for the cloud server to calculate the vector representation of positive and negative sample pairs in the latent space is as follows: ;in, This indicates a single-layer fully connected network. Indicates the backbone network. Represents image samples view Environmental Information The resulting environment-aware blended view. Represents image samples Another view, and They represent and Vector representation in the latent space.

[0024] This computational approach extracts environmental perception features jointly through a backbone network and a projection head, achieving implicit suppression of background noise without increasing the inference burden on edge devices. It utilizes a hybrid environmental perception view and multi-view samples to construct positive and negative pairs, avoiding reliance on large amounts of labeled data and complete environment modeling. This enhances the visual model's adaptability in dynamic scenes and effectively overcomes the high cost and response lag issues of traditional methods.

[0025] According to a preferred embodiment, the formula for calculating the hybrid sensing contrast loss of image samples by the cloud server is as follows: ; in, Represents cosine similarity. Indicates the temperature coefficient of positive sample pairs. The temperature coefficient representing the negative sample pair. This represents the vector representation of a positive sample in the latent space. and They represent and Vector representation in the latent space for Positive samples, Forming positive sample pairs This represents the vector representation of a negative sample in the latent space.

[0026] This hybrid sensing loss calculation uses contrastive learning to bring positive sample pairs closer together and push away negative sample pairs, enhancing the model's ability to discriminate foreground targets and effectively suppressing background interference. The dual temperature coefficient design improves feature decoupling capabilities and avoids the high inference overhead caused by complex attention mechanisms. It requires no extensive labeling or environmental priors, relying only on a small number of samples for optimization on cloud servers, supporting low-power, high-efficiency adaptation to dynamic scenarios on edge devices, and overcoming the dual limitations of traditional methods in terms of data and modeling.

[0027] The present invention provides a method for optimizing a few-shot visual model based on edge-cloud collaboration from a second aspect. The method includes: an edge device acquiring environmental images of the current deployment environment and sending them to a cloud server; upon receiving the environmental images, the cloud server extracting image samples from a pre-training dataset and fusing them with the environmental images using a block-level fusion method to generate environmental perception training samples; the cloud server optimizing the backbone network based on the environmental perception training samples and an optimization strategy including hybrid perception contrastive learning and multi-task auxiliary objectives, generating a lightweight optimization module, and sending the generated optimization module to the edge device; and the edge device loading the optimization module to perform a few-shot inference task.

[0028] This method utilizes an edge-cloud collaborative architecture to move the complex optimization process to the cloud server. Edge devices only need to collect environmental images and load a lightweight tuning module, significantly reducing local computational burden and avoiding the high inference costs associated with adjustments to feature layers or attention mechanisms in traditional methods. The cloud server constructs environmentally aware samples using a block-level hybrid strategy, eliminating the need for extensive labeled data or complete environment modeling, thus overcoming the dependence of existing fine-tuning schemes on data and priors. The tuning module can be quickly deployed after generation, improving model adaptation speed and deployment convenience, effectively supporting real-time response and continuous performance optimization in dynamic scenarios, and meeting the practical needs of edge devices with limited resources and diverse scenarios. Attached Figure Description

[0029] Figure 1 This is a structural framework diagram of the few-shot visual model optimization system based on edge-cloud collaboration provided by the present invention; Figure 2 The visual model provided by this invention is based on dog images generated by traditional few-shot learning methods; Figure 3 The visual model provided by this invention is a dog image generated based on the few-shot learning method of this invention; Figure 4 The visual model provided by this invention is a cat image generated based on a traditional few-shot learning method; Figure 5 The visual model provided by this invention is a cat image generated based on the few-shot learning method of this invention; Figure 6 The visual model provided by this invention is based on squirrel images generated by traditional few-shot learning methods; Figure 7 The visual model provided by this invention is a squirrel image generated based on the few-shot learning method of this invention; Figure 8 This is a schematic diagram of the module connections of the edge-cloud collaborative few-shot visual model optimization system provided by the present invention; Figure 9This is a schematic diagram of the operation logic of the edge-cloud collaborative few-shot visual model optimization system provided by the present invention; Figure 10 This is a flowchart illustrating the few-shot visual model optimization method based on edge-cloud collaboration provided by the present invention.

[0030] List of reference numerals 100: Edge device; 110: Image acquisition module; 120: Embedded processor; 130: Communication module; 140: Local storage module; 200: Cloud server; 210: Cloud computing module; 220: Data storage module; 230: Communication interface. Detailed Implementation

[0031] The following is a detailed explanation with reference to the accompanying drawings.

[0032] This invention provides explanations and clarifications for some terms and concepts.

[0033] In embodiments of the present invention, edge device 100 refers to a terminal device deployed at the network edge with certain computing and storage capabilities, including but not limited to smart cameras, embedded processors 120, industrial inspection equipment, and mobile terminals. Edge device 100 is typically used to process collected sensory data locally to reduce communication latency and bandwidth overhead, thereby adapting to application scenarios with high requirements for real-time performance, power consumption, or connection stability. In this invention, edge device 100 undertakes core functions such as visual model inference, environmental image acquisition, and adaptation parameter loading.

[0034] The visual model used in this invention refers to a deep neural network model for processing image or video data, particularly a neural network structure based on the Vision Transformer (ViT) and its variants. This type of visual model achieves global modeling and semantic understanding of the target object by dividing the input image into multiple non-overlapping image patches and utilizing a self-attention mechanism to model long-distance dependencies between image regions. In some embodiments, the visual model includes, but is not limited to, ViT, DeiT, and Swing Transformer, whose excellent feature transfer capabilities and structural versatility make them suitable for the rapid adaptive optimization scheme proposed in this invention under few-shot environments.

[0035] During the operation of the system of this invention, the edge device 100 acquires and uploads an image that does not contain foreground targets; this image is called a foreground-free image. The foreground-free image mainly contains background information or interfering elements in the deployment environment, such as walls, tables, chairs, floors, or fixed devices in the scene, but does not contain objects to be classified. The foreground-free image is used to characterize the background feature distribution of the deployment environment, facilitating the visual model to learn how to ignore task-irrelevant environmental backgrounds during the adaptation process, thereby improving the ability to focus on foreground targets and the accuracy of recognition.

[0036] Cloud server 200 refers to a server cluster or data center with remote computing and data management capabilities, typically connected and interacting with edge device 100 via the Internet. In embodiments of this invention, cloud server 200 has the ability to load copies of the visual model, access pre-trained image datasets, and perform parameter optimization tasks. Upon receiving a foreground-free image from edge device 100, cloud server 200 generates model adaptation parameters suitable for the deployment environment through environment-aware data augmentation and hybrid contrastive learning mechanisms, and returns these parameters to edge device 100 via the network. The cloud server 200 of this invention can effectively reduce the computational burden on the edge side and improve the adaptability and inference performance of the visual model in new environments.

[0037] In this invention, the view (such as a "view constructed from random data augmentation" or an "environment-aware hybrid view") is a derived visual form formed after specific technical processing. The view is obtained based on the original data through operations such as data augmentation and formula calculation, and has a clear functional orientation (used for environment-aware hybrid computation).

[0038] An image refers to raw visual material that has not undergone complex derivative processing. For example, a cropped image is the raw data obtained after performing a basic "cropping" operation on environmental information. Images are one of the input materials in formula calculations.

[0039] Current mainstream visual model optimization methods mainly focus on interference suppression or improvement of attention mechanisms at the feature layer. Their core objective is to enhance the robustness of visual models in complex environments by reducing background noise. While these techniques can mitigate performance degradation caused by environmental changes to some extent, the complex feature processing mechanisms they rely on often incur significant computational overhead. Furthermore, limited by the static nature of fixed parameters, they struggle to achieve low-latency real-time responses in dynamic scenarios, forming a bottleneck for performance improvement. To address this limitation, research has gradually shifted towards lightweight fine-tuning strategies, attempting to endow visual models with environmental adaptability after deployment. However, existing solutions still face key constraints: on the one hand, their performance optimization effectiveness is highly dependent on the continuous supply of large-scale labeled data; on the other hand, a complete model of the target environment needs to be completed before deployment. This dual dependence makes improving the adaptability of visual models significantly challenging in practical applications where edge devices have limited computing resources and application scenarios frequently switch.

[0040] To address the shortcomings of existing technologies, this invention provides a few-shot visual model optimization system and method based on edge-cloud collaboration. This invention also provides a cloud server 200 for constructing a few-shot visual model tuning module.

[0041] Example 1 The edge-cloud collaborative few-shot visual model optimization system of the present invention includes an edge device 100 and a cloud server 200. The edge device 100 and the cloud server 200 constitute an overall edge-cloud collaborative architecture.

[0042] like Figure 8 As shown, the edge device 100 mainly includes an image acquisition module 110, an embedded processor 120, a communication module 130, and a local storage module 140. For example... Figure 9 As shown, the image acquisition module 110 is used to acquire environmental images of the current deployment environment. The embedded processor 120 integrates a visual Transformer inference unit and supports loading tuning modules to adjust the behavior of the visual model, such as... Figure 9 As shown. The communication module 130 is used to transmit environmental images uplink to the cloud server 200 and receive the tuning module returned from the cloud server 200, such as... Figure 9 As shown. The local storage module 140 is used to cache environmental images, intermediate states of visual model inference, and adaptation parameter files.

[0043] like Figure 8 As shown, the cloud server 200 mainly consists of a cloud computing module 210, a data storage module 220, and a communication interface 230. The cloud computing module 210 runs a GPU or TPU accelerator and is responsible for performing rapid visual model fine-tuning operations based on uploaded environment images, such as... Figure 9As shown. The data storage module 220 is used to store the pre-trained dataset and the corresponding visual model files, such as... Figure 9 As shown. The communication interface 230 is used to receive environmental images from the edge device 100 and send the fine-tuned optimization module to the edge device 100.

[0044] The aforementioned physical modules establish an end-to-end communication path via a stable data link (such as a 4G / 5G or Wi-Fi network). During system operation, edge device 100 uploads an unlabeled environmental image to cloud server 200. Upon receiving the image, cloud server 200 extracts image samples from the pre-training dataset, performs environmental awareness data augmentation and fine-tuning operations, and sends the generated optimization module (low-rank adaptation module) to edge device 100. Edge device 100 loads this optimization module and performs few-shot inference tasks.

[0045] In the system of this invention, the coordinated operation between various physical hardware units ensures the visual model's ability to quickly adapt to the environment.

[0046] First, the edge device 100 acquires a foreground-free environment image in the current deployment environment through the image acquisition module 110. For example... Figure 1 As shown in step ①, the edge device 100 uploads an environmental image to the cloud server 200. Specifically, the edge device 100 uploads the environmental image to the cloud server 200 via the communication module 130. This environmental image serves as the environmental information basis for subsequent visual model adaptation and is the input source for the entire optimization process.

[0047] After receiving the environmental image, such as Figure 1 In step ② of the cloud-side model fine-tuning process, the cloud computing module 210 mixes the environmental image with image samples from the pre-training dataset in a block-level mixing manner. By constructing an environment-aware mask (binary mask), the cloud computing module 210 inserts background image blocks from the received environmental image into the image samples to form an environment-aware mixed view for fine-tuning, i.e., the environment-aware training sample. In this process, the system does not rely on any manually labeled information and can complete data augmentation and visual model adjustment solely based on the environmental image itself.

[0048] After fine-tuning, the cloud computing module 210 generates a low-rank adaptation module, or tuning module, for adaptation. This module is only a few MB in size and is then transmitted to the edge device 100 via the communication interface 230. The edge device 100 loads this tuning module into the backbone network, enabling the pre-trained backbone network to adapt to the current deployment environment. Throughout this process, the data path is clear, the amount of communication data is small, and computational operations are limited to the cloud computing module 210, ensuring efficient deployment of the system on resource-constrained devices.

[0049] This invention presents a few-shot visual model optimization method based on edge-cloud collaboration. Addressing the common performance degradation problem of few-shot visual models in real-world deployment environments, it proposes an optimization mechanism that is label-independent, resource-efficient, and adapts quickly. For example... Figure 1 and Figure 10 In summary, the processing flow can be divided into four stages: The first stage involves uploading environmental images.

[0050] S100: Edge device 100 collects environmental images of the current deployment environment and sends them to cloud server 200, such as... Figure 10 As shown.

[0051] Specifically, edge device 100 acquires an environmental image of the current deployment environment. And send it to the cloud server 200. For example... Figure 1 As shown in step ①, the environmental image of the current forest environment collected by the edge device 100 is uploaded to the cloud server 200.

[0052] The second phase involves fine-tuning the visual model on the cloud server 200.

[0053] S200: Upon receiving the environmental image, the cloud server 200 extracts image samples from the pre-training dataset and fuses them with the environmental image using a block-level fusion method to generate environmental perception training samples. Based on the environmental perception training samples and an optimization strategy incorporating hybrid perception contrastive learning and multi-task auxiliary objectives, the cloud server 200 optimizes the backbone network, generating lightweight optimization modules, such as... Figure 10 As shown. Here, the block-level fusion method is the core technical means by which the cloud server 200 fuses the image samples of the pre-training dataset with the environmental image during the generation of environmental perception training samples. Its preferred implementation is achieved through the calculation formula in step S211, and finally generates an environmental perception fusion view for backbone network optimization.

[0054] like Figure 1 As shown in step ②, the fine-tuning of the visual model of the cloud server 200 includes two steps: constructing environmental perception training samples and multi-task assisted hybrid contrastive learning.

[0055] S210: Construct environmental perception training samples.

[0056] This step aims to address the problem of insufficient samples on terminal devices, which limits the adaptability of visual models to new environments. Since deployed devices often lack sufficient local training samples, making it difficult for visual models to effectively learn and generalize to environmental changes, this invention fully utilizes the large-scale basic dataset already available on the cloud server 200, combined with environmental images uploaded by the user terminal (edge ​​device 100). We will construct environmental perception training samples that integrate environmental information to enhance our environmental perception capabilities.

[0057] S211: Construct an environment-aware hybrid view.

[0058] Cloud server 200 uses pre-trained datasets for pre-training visual models (e.g., ImageNet-1K, which includes image information and corresponding label information) Randomly select a subset of image samples and their corresponding labels to form a cloud data subset. ,like Figure 1 The cloud data subset is shown in the image.

[0059] For each image sample , Two different views are generated based on random data augmentation methods. Preferably, the random data augmentation methods include random cropping, color dithering, and Gaussian blurring. The two different views generated here are: and . Figure 1 This demonstrates two views obtained from the same panda image in a subset of cloud data, using different random data augmentation methods.

[0060] The cloud computing module 210 in the cloud server 200 uploads environmental images. Random cropping is performed to obtain the cropped environmental image. Here, random cropping refers to randomly selecting a sub-region of a certain location and size within the environmental image for cropping.

[0061] The cloud computing module 210 in the cloud server 200 blends any one of the views with the environmental information in the cropped environment image (hereinafter referred to as...). (For example), an environment-aware blended view is generated. The generated environment-aware blended view is as follows: Figure 1 The image marked in the image is the result of block-level blending.

[0062] Preferably, the cloud server 200 generates an environment-aware hybrid view in a block-level blending manner, and the calculation formula for this process is as follows: .

[0063] In the above formula, This represents an environment-aware hybrid view. Indicates from image samples Views built using random data augmentation This represents environmental information, specifically the cropped environmental image. This indicates element-wise multiplication. A binary mask representing random sampling.

[0064] S220: Perform multi-task assisted hybrid perception contrastive learning.

[0065] Existing few-shot learning and model transfer methods typically focus on adjustments in high-level semantic feature spaces to improve the generalization ability of visual models on new tasks, new categories, or new domains. Specifically, these methods mostly employ strategies such as feature alignment, prototype networks, or meta-learning to enhance the visual model's ability to represent category information by optimizing the distance structure or inter-class distribution in the semantic space. However, these methods primarily focus on modeling task semantics, often assuming that background information in the input image is irrelevant or negligible, thus lacking the ability to model and adapt to environmental factors.

[0066] This assumption does not hold true in real-world deployment scenarios such as edge intelligence. In typical edge applications such as industrial quality inspection, autonomous driving, and remote monitoring, the operating environment after the visual model is deployed often exhibits high stability and repeatability, displaying clear scene structure and distribution characteristics even when the background changes. However, existing methods generally assume that the environment is dynamic, unpredictable, and even unmodelable. Therefore, their training phase is entirely based on the source dataset, without considering the visual priors of the deployment environment. This leads to a lack of perceptual ability in visual models when faced with new background information during actual reasoning, resulting in problems such as semantic confusion and blurred target boundaries.

[0067] More importantly, the adaptation mechanisms of these methods are mostly limited to the ends of the network structure, such as classifier layers, projection heads, or shallow alignment modules, while the feature extraction process of the backbone network itself usually remains unchanged. This "shallow tuning" approach cannot fundamentally improve the visual model's perception strategy in new environments. Especially under settings with strong background interference and a small number of image samples, the visual model remains highly sensitive to irrelevant regions (such as background, lighting, and texture changes), leading to unstable performance and difficulty in adapting to input distribution shifts under real deployment conditions.

[0068] Therefore, existing technologies lack a mechanism for systematically introducing environmental information from the training phase at the methodological level, and their structural design also fails to achieve environmental adaptation and optimization at the backbone network level. These shortcomings limit the generalization ability and deployment practicality of few-shot visual models in real-world scenarios, especially in edge computing environments where computing resources are limited and training opportunities are scarce. Once visual models are removed from the source data distribution, their performance is prone to degradation, and they lack the ability to adapt to individual environments.

[0069] This invention proposes a more structurally robust solution to the aforementioned problems: it not only actively incorporates environmental images of the current deployment environment during the training phase, but also constructs an optimization strategy that incorporates hybrid perceptual contrastive learning and multi-task auxiliary objectives during the optimization process to drive the backbone network for environmental adaptive tuning. The core objective of this method is to guide the visual model to suppress task-irrelevant environmental interference while maintaining its original semantic modeling capabilities, thereby achieving more robust feature extraction and better few-shot inference performance.

[0070] This optimization strategy, by introducing a combination of primary task loss and hybrid perception contrastive loss, reduces the interference of environmental information from the previously constructed environment-aware samples on the core features of the visual model while maintaining stable performance of the original task. Through multi-task constraints, the visual model can achieve a dynamic balance between the original task objective and the environment discrimination objective, thereby effectively decoupling task features and environmental features. The design of this multi-task-assisted hybrid perception contrastive learning module will be detailed below, including its loss structure, positive-negative pair generation method, optimization objective, and its role in the training process.

[0071] S221: Constructing positive and negative sample pairs in contrastive learning.

[0072] Positive sample pairs: This refers to image samples originating from the same original image but augmented using different data augmentation methods. For example, one data augmentation method could be mixed with the environment image, while another method is not mixed with the environment image.

[0073] Negative sample pairs: It is a subset of data from the cloud. Image samples of different images, all of which are mixed with environmental images.

[0074] like Figure 1 As shown, different data-augmented views derived from the same panda image are used as positive sample pairs, while panda views and sports car views (negative samples) that mix environmental image information are used as negative sample pairs to learn discriminative feature representations.

[0075] This invention constructs positive and negative sample pairs in such a way that, during the contrastive learning process, the visual model maximizes the feature similarity between different views of the same image sample in the latent space, while minimizing the feature consistency between image samples that are mixed with environmental information but are semantically unrelated, thereby improving the environmental robustness and discriminative ability of the visual model.

[0076] S222: Calculate the vector representation of positive and negative sample pairs in the latent space based on positive and negative sample pairs, backbone network, and projection head function.

[0077] like Figure 1As shown, positive and negative sample pairs are input into the pre-trained backbone network. and projector head .

[0078] Here, the backbone network is the core of the deep neural network, typically a pre-trained deep convolutional neural network (such as ResNet, VGG, etc.), used to extract high-level semantic features from inputs (such as images, text). The projector is a small neural network (usually composed of 2-3 fully connected layers) connected after the backbone network, used to map the features extracted by the backbone network to a latent space suitable for contrastive learning.

[0079] Therefore, the formula for calculating the vector representation of positive and negative sample pairs in the latent space using the projection head in the cloud server 200 is as follows: ; .

[0080] In the above formula, This represents a single-layer fully connected network for the projection head. Indicates the backbone network. Represents image samples view Environmental Information The resulting environment-aware blended view. Represents image samples Another view, and They represent and Vector representation in the latent space.

[0081] S223: Calculate the hybrid sensing contrast loss of image samples based on vector representations in the latent space obtained from two different perspectives of the same image sample.

[0082] Specifically, in obtaining the latent representations of the same image sample from two different perspectives and Then, the mixed-sensory contrast loss of the image samples is calculated.

[0083] The formula for calculating the hybrid sensing contrast loss of image samples is: .

[0084] In the above formula, This represents the hybrid sensing contrast loss for an image sample. Represents cosine similarity. Indicates the temperature coefficient of positive sample pairs. The temperature coefficient representing the negative sample pair. This represents the vector representation of a positive sample in the latent space. and They represent and Vector representation in the latent space for Positive samples, Forming positive sample pairs This represents the vector representation of a negative sample in the latent space.

[0085] S224: Calculate the overall mixed-sensory contrast loss based on the mixed-sensory contrast loss of several image samples.

[0086] When there are N image samples in a batch, the overall mixed sensing contrast loss is calculated by averaging the loss of each image sample. .

[0087] Overall Mixed Sensing Contrast Loss The calculation formula is: .

[0088] In the above formula, N represents the batch size, that is, the total number of image samples contained in a batch. This represents the index position of the image sample in the batch, and iterates through the samples from 1 to N. =1 indicates the first image sample. =2 represents the second image sample, ..., =N represents the last image sample. This represents the mixed-sensory contrast loss of an image sample.

[0089] S225: When learning multiple tasks, calculate the multi-task-assisted mixed-sensory contrast loss based on the overall mixed-sensory contrast loss.

[0090] Multi-task learning enables visual models to learn multiple tasks simultaneously to capture more general and discriminative features. For example, multiple tasks may include tasks targeting contrastive loss and tasks targeting pre-trained task-specific losses.

[0091] To preserve the feature extraction capabilities of the backbone network and prevent overfitting to environmental information, this architecture incorporates a pre-training task for the backbone network during its contrastive learning process. For different backbone network pre-training tasks, a task-specific loss function is employed. Examples include cross-entropy loss for supervised tasks, and DINO and iBOT losses for self-supervised tasks.

[0092] The complete formula for calculating the multi-task assisted hybrid sensing contrast loss is as follows: , In the above formula, This represents the overall mixed-perception contrast loss. and task-specific loss function Same weight. This part is as follows: Figure 1 The multi-task comparison loss is shown in the figure.

[0093] Cloud server 200 uses multi-task assisted hybrid perception to compare loss This is used to train and update the lightweight low-rank adapter module, which is the tuning module of the visual model in the cloud server 200.

[0094] As described above, the optimization strategy of the optimization module of the present invention includes: the contrastive learning part aggregates similar samples at low temperature and improves the separation of dissimilar samples by temperature to achieve feature space optimization; and the original knowledge of the visual model is maintained based on the multi-task auxiliary target.

[0095] S300: The cloud server 200 sends the generated tuning module to the edge device 100, such as... Figure 10 As shown.

[0096] like Figure 1 As shown in step ③, the cloud server 200 transmits the tuning module to the edge device 100. Specifically, after the visual model training is completed, the cloud server 200 packages the optimized tuning module, i.e., the lightweight low-rank adapter module, and sends it to the edge device 100.

[0097] Edge device 100 does not require structural modifications or complex configurations; it only needs to load and integrate the optimization module into the existing visual model to enhance the visual model's environmental adaptability.

[0098] Existing technologies often rely on introducing additional structures or operations during the inference phase to enhance the visual model's ability to focus on foreground targets when dealing with environmental interference. For example, some methods explicitly extract foreground regions through image cropping, thus introducing a preprocessing step during inference to improve discriminativity; others introduce specific attention modules during inference to attempt to recapture information about target regions from features output by the backbone network. While these methods improve visual model performance to some extent, their modifications to the inference process are significant—additional attention modules, region selectors, or structural reconstruction operations significantly increase the computational overhead and memory consumption of the visual model during inference. This is unacceptable for resource-constrained edge devices 100, easily leading to deployment difficulties, increased response latency, and even inference failure.

[0099] In contrast, the optimization module proposed in this invention enables the visual model to acquire explicit environmental perception capabilities during training through a hybrid enhancement and contrastive learning mechanism for environmental images. The optimization module has already learned and solidified the concept of "ignoring irrelevant backgrounds and focusing on key foregrounds" during the training phase, and the relevant parameters are efficiently transferred to the lightweight optimization module through fine-tuning of the model parameters. Therefore, during deployment, the optimization module is loaded as static parameters, requiring no modification to the visual model's ontology structure or inference process, and does not increase operator complexity or memory access during inference. In other words, this invention effectively avoids the resource-intensive requirements of existing methods during the inference phase through a strategy of "active adaptation during training and zero overhead during inference," exhibiting high engineering practicality and edge deployment friendliness.

[0100] Furthermore, because this invention introduces a contrastive learning mechanism that combines environmental image blending enhancement with semantic preservation during the visual model training phase, the visual model has learned how to extract feature representations highly relevant to the task objective from background interference. The parameter adjustment capabilities inherent in the tuning module enable the visual model to "perceive and adjust the differences in the distribution of features in the current deployment environment," thereby more accurately locating the foreground target region, enhancing key semantic expressions, and reducing the impact of background or irrelevant regions on classification / recognition decisions. Therefore, the loaded visual model naturally possesses environmental adaptability and foreground attention capabilities, achieving robust performance improvements without explicitly constructing saliency maps, attention maps, or pruning operations. This advantage not only demonstrates the "zero-intrusion" nature of this invention during the inference phase but also fundamentally enhances the generalization and deployment efficiency of the visual model.

[0101] As described above, this invention significantly improves the foreground recognition capability of visual models in complex environments without increasing inference computation costs or changing the structure of the visual model by moving the environment adaptation task to the training stage and deploying it in the form of lightweight parameter modules. This is significantly different from existing technical approaches and has remarkable technological progress and innovation.

[0102] S400: Edge device 100 loads a tuning module for few-shot inference tasks.

[0103] Edge device 100 downloads the tuning module sent by cloud server 200 and integrates it with the existing local backbone network to form the final adapted visual model, which is used to perform few-shot inference tasks such as 1-shot and 5-shot. Figure 1 As shown in step ④, device-side few-shot inference is achieved: when a bird is in the previously collected forest environment (environmental image), the fused visual model can more accurately identify its category.

[0104] Specifically, the tuning module is implemented based on LoRA (Low-Rank Adaptation) technology. Without modifying the original visual model structure, it incrementally adjusts the parameters of key layers in the backbone network by inserting a set of lightweight low-rank matrices. This fusion process does not require changes to the visual model structure or additional inference computation. The edge device 100 only needs to load the tuning module and perform parameter injection operations to enable the visual model to perceive and adapt to the deployment environment, thereby effectively focusing on foreground targets and suppressing background interference, achieving efficient and accurate few-shot recognition inference.

[0105] Figures 2 to 7 The paper intuitively demonstrates the significant advantages of the proposed environment-aware optimization strategy in few-shot reasoning tasks. Figures 2 to 7 In the diagram, the color of the squares indicates the level of attention the backbone network pays to different areas. The more red the square is, the higher the level of attention; the more blue it is, the lower the level of attention.

[0106] Figure 2 The image shows a dog image generated by a visual model based on a traditional few-shot learning method. Figure 3 The image shows a dog image formed by a visual model based on the few-shot learning method of the present invention.

[0107] by Figure 2 and Figure 3 For example, traditional few-shot learning methods lack an environmental adaptation mechanism for the backbone network, resulting in attention maps (see...). Figure 2 In the image, squares on the target object tend to be blue, while squares that tend to be red are often more prominent on the target object (e.g., ...). Figure 2 The dog in the image is scattered between itself and the background, making it difficult to focus attention on the target area and affecting the accuracy of reasoning. After environmental perception optimization using the optimization module proposed in this invention, the attention map generated by the visual model (see...) Figure 3 In the model, the red-biased squares can significantly focus on the dog itself, effectively suppressing background interference and improving the perception and reasoning performance of the visual model.

[0108] Figure 4 The image shows a cat image generated by a visual model based on a traditional few-shot learning method. Figure 5 The image shown is a cat image generated by a visual model based on the few-shot learning method of the present invention.

[0109] Similarly, Figure 4 and Figure 5 This demonstrates the differences in performance of visual models when processing images of cats in new environments. Figure 4In the image, the reddish squares are mainly scattered in the background, while the squares on the target object (cat) are bluish, indicating that the attention area is relatively scattered, with some focus on the background bushes. After environmental perception optimization using the optimization module proposed in this invention, the visual model generated... Figure 5 In the image, the reddish squares are distributed on the head of the target object (cat), and most of the squares (including the bluedish squares) are clustered on the head and body of the target object (cat). This indicates that attention is more focused on the cat's head and body, which significantly improves the accuracy of target perception.

[0110] Figure 6 The image shows a squirrel formed by a visual model based on a traditional few-shot learning method. Figure 7 The image shows a squirrel formed by a visual model based on the few-shot learning method of the present invention.

[0111] exist Figure 6 and Figure 7 Similar phenomena can also be observed in [the context of this]. Figure 6 In the image, the reddish squares are mainly distributed in the squirrel and background areas, indicating that attention not only covers the squirrel but also extends to the tree trunk and background areas. After environmental perception optimization using the optimization module proposed in this invention, the visual model generated... Figure 7 In the image, the reddish squares are mainly distributed on the squirrel's body, indicating that attention is highly focused on the squirrel's body, especially the head and forelimb areas, reflecting the visual model's stronger ability to identify key targets in the environment.

[0112] In summary, the environmental perception optimization strategy proposed in this invention can effectively guide the visual model to focus on key target areas in different scenarios, significantly improving the robustness and accuracy of few-shot inference.

[0113] This invention enables rapid adaptation of visual models in a completely label-free environment, and is particularly suitable for typical edge vision systems such as mobile terminals, industrial cameras, and smart security systems. It significantly enhances the generalization ability and inference performance of visual models while improving deployment flexibility.

[0114] Example 2 This embodiment is based on an edge-cloud collaborative few-shot visual model optimization system, including an edge device 100 and a cloud server 200. The edge device 100 and the cloud server 200 constitute the overall edge-cloud collaborative architecture.

[0115] In the edge device 100, the image acquisition module 110 is preferably a camera component (camera) with image sensing circuitry, used to acquire environmental images of the deployment environment. The camera component is circuitically connected to the embedded processor 120.

[0116] The embedded processor 120 is preferably a processor or dedicated integrated chip that includes a CPU, an NPU, and an AI acceleration unit. The embedded processor 120 integrates a vision Transformer inference unit and supports loading tuning modules to adjust the behavior of the vision model.

[0117] The communication module 130 is preferably a first network communication module, used to transmit environmental images uplink to the cloud server 200 and receive the tuning module returned from the cloud server 200. The first network communication module can specifically be an Ethernet interface circuit or a wireless communication module, wherein the wireless communication module includes a Wi-Fi communication module, a 4G / 5G communication module, etc. The embedded processor 120 is connected to the first network communication module circuit.

[0118] The local storage module 140 is preferably a first storage chip, used to cache environmental images, intermediate states of visual model inference, and adaptation parameter files. The embedded processor 120 is circuitically connected to the first storage chip.

[0119] In the cloud server 200, the physical hardware of the cloud computing module 210 is a computing chip, including a central processing unit (CPU), a graphics processing unit (GPU) or a tensor processor (TPU), as well as supporting memory and high-speed interconnect circuits, used to perform rapid visual model fine-tuning operations based on uploaded environment images.

[0120] The data storage module 220 is preferably a second storage chip, used to store the pre-trained dataset and the corresponding visual model file. The computing chip is circuitically connected to the second storage chip.

[0121] The communication interface 230 is preferably a second network communication module, used to receive environmental images from the edge device 100 and send the fine-tuned optimization module to the edge device 100. The second network communication module can specifically be an Ethernet interface circuit or a wireless communication module, wherein the wireless communication module includes a Wi-Fi communication module, a 4G / 5G communication module, etc. The computing chip is connected to the second network communication module circuit.

[0122] Specifically, the camera component sends the raw environmental image to the embedded processor 120 in the form of a digital signal. After receiving the raw environmental image from the camera component, the embedded processor 120 temporarily stores it in the local storage module 140, and performs preliminary processing by its internal CPU or AI acceleration unit to form an environmental image that can be read by the computing chip, preparing for subsequent inference and transmission.

[0123] After the visual Transformer inference unit loads the initial model, if the embedded processor 120 determines that model optimization is needed, the embedded processor 120 sends the pre-processed environmental image to the computing chip in the cloud server 200 via the first network communication module. The embedded processor 120 also sends the environmental image, the intermediate state of the visual model inference, and the adaptation parameter file to the first storage chip.

[0124] After receiving the environmental image, the intermediate state of the visual model inference, and the adaptation parameter file, the first storage chip performs caching.

[0125] Upon receiving the environmental image, the second network communication module sends the image to the computing chip. After receiving the environmental image, the computing chip extracts image samples from the pre-training dataset and fuses them with the environmental image using a block-level fusion method to generate environmental perception training samples. Based on the environmental perception training samples and an optimization strategy incorporating hybrid perception contrastive learning and multi-task auxiliary objectives, the computing chip optimizes the backbone network, generating a lightweight optimization module.

[0126] The computing chip sends the tuning module to the second network communication module. Upon receiving the tuning module, the second network communication module sends it to the first network communication module. The computing chip also sends the pre-trained dataset and the corresponding visual model file to the second storage chip for storage.

[0127] After receiving the tuning module from the second network communication module, the first network communication module sends the tuning module to the embedded processor 120.

[0128] After receiving the tuning module from the first network communication module, the embedded processor 120 loads the tuning module and integrates it with the local backbone network to form the final adapted visual model for few-shot inference tasks.

[0129] It should be noted that the specific embodiments described above are exemplary. Those skilled in the art can devise various solutions inspired by the disclosure of this invention, and these solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents. This specification contains multiple inventive concepts; phrases such as "preferredly" or "according to a preferred embodiment" indicate that the corresponding paragraph discloses an independent concept. The applicant reserves the right to file divisional applications based on each inventive concept.

Claims

1. A few-shot visual model optimization system based on edge-cloud collaboration, characterized in that, The system includes an edge device (100) and a cloud server (200). The edge device (100) collects environmental images of the current deployment environment and sends them to the cloud server (200). The cloud server (200) is configured as follows: After receiving the environmental image, image samples are extracted from the pre-training dataset and fused with the environmental image based on a block-level mixing method to generate environmental perception training samples. The backbone network is tuned based on the environmental perception training samples and an optimization strategy that includes hybrid perception contrastive learning and multi-task auxiliary objectives, generating a lightweight tuning module. Send the generated tuning module to the edge device (100); The edge device (100) loads a tuning module to perform few-shot inference tasks.

2. The system according to claim 1, characterized in that, The steps of generating environment-aware training samples by the cloud server (200) include: After receiving environmental images, image samples and their corresponding labels are randomly extracted from the pre-trained dataset to form a subset of cloud data; For each image sample, two different views are generated based on a random data augmentation method; The environmental image is randomly cropped to obtain the cropped environmental image; Blend one of the views with environmental information from the cropped environment image; The generated environment-aware hybrid view is used as an environment-aware training sample.

3. The system according to claim 1 or 2, characterized in that, The steps for constructing the optimization strategy on the cloud server (200) include: Constructing positive and negative sample pairs in contrastive learning; The vector representation of the positive and negative sample pairs in the latent space is calculated based on the positive and negative sample pairs, the backbone network, and the projection head. The mixed-sensory contrast loss of the image sample is calculated based on the vector representations in the latent space obtained from two different perspectives of the same image sample. The overall mixed-sensory contrast loss is calculated based on the mixed-sensory contrast loss of several image samples. When learning multiple tasks, multi-task-assisted mixed-sensing contrast loss is calculated based on the overall mixed-sensing contrast loss.

4. The system according to any one of claims 1 to 3, characterized in that, The steps for the edge device (100) to perform few-shot inference tasks include: The optimization module is then integrated with the local backbone network to form the final adapted visual model.

5. The system according to any one of claims 1 to 4, characterized in that, The optimization strategies of the optimization module include: The contrastive learning part aggregates similar samples at low temperatures and improves the separation of dissimilar samples by temperature to achieve feature space optimization. The goal is to maintain the original knowledge of the visual model based on multi-task assistance.

6. The system according to any one of claims 1 to 5, characterized in that, The optimization strategy of the optimization module also includes: Without modifying the original visual model structure, the parameters of key layers in the backbone network are incrementally adjusted by inserting a set of lightweight low-rank matrices.

7. The system according to any one of claims 1 to 6, characterized in that, The calculation formula for the cloud server (200) to generate the environment-aware hybrid view is as follows: ； in, This represents an environment-aware hybrid view. A binary mask representing random sampling. Indicates from image samples Views built using random data augmentation This represents environmental information, specifically the cropped environmental image. This indicates element-wise multiplication.

8. The system according to any one of claims 1 to 7, characterized in that, The cloud server (200) calculates the vector representation of positive and negative sample pairs in the latent space using the following formula: ； in, This indicates a single-layer fully connected network. Indicates the backbone network. Represents image samples view Environmental Information The resulting environment-aware blended view. Represents image samples Another view, and They represent and Vector representation in the latent space.

9. The system according to any one of claims 1 to 8, characterized in that, The formula used by the cloud server (200) to calculate the hybrid sensing contrast loss of the image samples is as follows: ； in, Represents cosine similarity. Indicates the temperature coefficient of positive sample pairs. The temperature coefficient representing the negative sample pair. and They represent and Vector representation in the latent space for Positive samples, Forming positive sample pairs This represents the vector representation of a negative sample in the latent space.

10. A method for optimizing few-shot visual models based on edge-cloud collaboration, characterized in that, The method includes: The edge device (100) collects environmental images of the current deployment environment and sends them to the cloud server (200); After receiving the environmental image, the cloud server (200) extracts image samples from the pre-training dataset and fuses them with the environmental image based on a block-level mixing method to generate environmental perception training samples. The cloud server (200) optimizes the backbone network based on the environmental perception training samples and the optimization strategy including hybrid perception contrastive learning and multi-task auxiliary objectives, generates a lightweight optimization module, and sends the generated optimization module to the edge device (100). The edge device (100) loads a tuning module to perform few-shot inference tasks.

Images