End side AI model OTA updating optimization method and system based on end-cloud collaboration

Abstract

The invention discloses an end side AI model OTA updating optimization method and system based on end-cloud collaboration. The method comprises the following steps that: end-side equipment collects and preprocesses multi-modal data, carries out reasoning to obtain a reasoning result, and carries out lightweight pre-judgment to generate a pre-judgment result; collecting double-mark feedback information of the user on the reasoning result and the pre-judgment result, then associating to form a feedback data set, and performing priority ranking to obtain a feedback priority; screening feedback data according to the feedback priority, and uploading the feedback data to a cloud server after hierarchical encryption; after processing by the cloud server, receiving an OTA update package pushed by the cloud server in a differentiated manner, verifying after updating, and rolling back to the model before updating when the update is abnormal; and continuously executing the above steps, and uploading the update state and the verification result to the cloud server. The problems that an existing end-side AI model is low in updating efficiency, poor in hardware adaptation, incapable of responding to personalized requirements of users, insufficient in end-cloud collaboration and weak in data transmission safety can be solved.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence technology, edge-cloud collaboration technology and OTA update technology, and in particular relates to an edge-side AI model OTA update optimization method and system based on edge-cloud collaboration. Background Technology

[0002] With the rapid development of artificial intelligence technology, the deployment scenarios of AI models are gradually extending from the cloud to the edge. Edge AI deploys trained models directly on terminal devices (such as mobile phones, industrial sensors, smart cameras, smart wearable devices, etc.), and can complete key processes such as data collection and inference calculation locally on the device without the need for high-frequency data interaction with the cloud. It has advantages such as low transmission latency, high data privacy, and saving network bandwidth, and is widely used in scenarios such as smart terminals, industrial equipment, and IoT nodes.

[0003] However, in actual deployment and operation, edge AI models generally suffer from performance stagnation and insufficient dynamic adaptation capabilities. Edge AI models often rely on historical datasets in the cloud for training, making it difficult to match dynamically changing parameters such as lighting, temperature, and noise in the real-world environment. This can easily lead to decreased inference accuracy and increased false positive rates. Furthermore, these models cannot respond to personalized user habits and customized needs, making it difficult to perform targeted optimizations based on actual user feedback. Over long-term operation, they are prone to deviating from real-world application scenarios.

[0004] Currently, updates to existing edge AI models mostly rely on manual updates or full OTA (Over-The-Air) pushes. Traditional manual updates require technicians to replace model files on-site, which is time-consuming, labor-intensive, and inefficient, and cannot meet the update needs of large-scale terminal devices. While full OTA updates achieve automated pushes, they require the transmission of complete model files, resulting in large update packages that consume significant network bandwidth and edge device storage resources. On edge terminals and mobile devices with limited bandwidth, storage, and computing resources, this can easily lead to slow updates, device lag, and abnormally high power consumption, severely impacting normal device operation.

[0005] Existing edge-cloud collaboration solutions have significant shortcomings in their collaboration mechanisms. Edge-side inference results and user feedback cannot effectively feed back into cloud training, resulting in a lack of real-world data support for cloud model training and a disconnect between optimization directions and actual edge-side needs. Furthermore, these solutions generally lack adaptation mechanisms for the heterogeneity of edge device hardware, failing to differentiate processing based on the computing power and storage conditions of different devices. This often leads to hardware incompatibility and excessively high runtime loads in updated models.

[0006] Furthermore, the lack of reasonable filtering mechanisms and privacy protection measures during the data upload process from the device to the cloud leads to increased bandwidth consumption due to the large amount of redundant data uploaded, and also poses a risk of privacy information leakage. Meanwhile, after model training is completed in the cloud, the lack of verification and compatibility testing of update packages can easily result in update failures, device malfunctions, and other problems, making it difficult to guarantee the stability and reliability of edge AI applications. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for OTA update and optimization of edge AI models based on edge-cloud collaboration, so as to solve the above problems.

[0008] To achieve the above objectives, in a first aspect of the present invention, a method for OTA update and optimization of edge AI models based on edge-cloud collaboration is provided, comprising: The edge device collects and preprocesses multimodal data through the edge AI model, infers inference results and inference logs, and performs lightweight prediction based on the device operation data and the inference logs to generate prediction results. The edge device collects user feedback information with dual tags for the inference result and the prediction result, associates the feedback information with the corresponding inference result, inference log and prediction result to form a feedback dataset, and sorts the feedback data in the feedback dataset by priority to obtain the feedback priority; The edge device filters the feedback data according to the feedback priority, and uploads it to the cloud server after hierarchical encryption. The edge device receives OTA update packages pushed by the cloud server in a differentiated manner, completes update package verification, download and model replacement, performs multi-dimensional verification of the model after the update, and rolls back to the model before the update in case of an anomaly. The edge device continues to perform the above steps and uploads the updated status and verification results to the cloud server.

[0009] Furthermore, in the aforementioned edge-cloud collaborative OTA update optimization method for edge AI models, the multimodal data includes on-site environmental data, user operation data, and equipment operation data; the preprocessing includes multimodal data separation, data cleaning, noise reduction, normalization, and format conversion; the prediction results include no need for update, incremental update recommendation, full update recommendation, and hardware compatibility risk warning.

[0010] Furthermore, in the aforementioned edge-cloud collaborative OTA update and optimization method for edge AI models, user feedback methods include click markers, voice feedback, and text input feedback. The process of collecting user feedback information on the dual-labeled reasoning results and prediction results also includes validating the feedback information and filtering out invalid feedback. The feedback priority is ranked according to at least one of the following factors: feedback frequency, reasonableness of correction, severity of reasoning error, and scope of influence of prediction deviation, and the feedback data is divided into three levels: high, medium and low.

[0011] Furthermore, in the aforementioned edge-cloud collaborative OTA update optimization method for edge AI models, the hierarchical encryption adopts a mechanism combining symmetric encryption and differential privacy; high-priority feedback data is encrypted using the AES256 symmetric encryption algorithm with added differential privacy noise, while medium or low-priority feedback data is encrypted using the conventional AES symmetric encryption algorithm; the edge device uses lossless compression algorithm, breakpoint resume mechanism, and dynamic rate adjustment to upload the feedback data to the cloud server.

[0012] In a second aspect of the invention, a method for OTA update and optimization of edge AI models based on edge-cloud collaboration is also proposed, executed by a cloud server, comprising: The cloud server receives the hierarchically encrypted feedback dataset uploaded by the end device and performs hierarchical decryption to obtain the original feedback dataset; valid feedback data is selected from the original feedback dataset, and multimodal data fusion is performed to generate a training dataset; multiple versions of optimized models are trained based on the initial AI model and the training dataset, and the adaptability pre-verification of the multiple versions of optimized models is performed to remove optimized models that fail the verification. The cloud server performs lightweight processing on the optimized model that has passed the verification, generates OTA update packages according to the prediction results and feedback priorities uploaded by the terminal devices, and encrypts and verifies the OTA update packages. The cloud server pushes the OTA update package to the corresponding end devices in a differentiated manner; The cloud server receives the update status and verification results returned by the terminal device, manages the model version, and supports version rollback, incremental iteration and batch updates.

[0013] Furthermore, in the aforementioned edge-cloud collaborative OTA update and optimization method for edge AI models, the multimodal data fusion includes cross-modal data alignment, feature fusion, and sample balancing; the multi-version optimized model is obtained by fine-tuning the parameters of the initial AI model through a transfer learning algorithm; and the adaptability pre-verification includes verifying the model's operational compatibility, computing power consumption, and storage usage.

[0014] Furthermore, in the aforementioned edge-cloud collaborative OTA update and optimization method for edge AI models, the lightweight processing includes model pruning and quantization. The OTA update package is encrypted using an asymmetric encryption algorithm, has an expiration date, and includes a verification code and a hardware compatibility report. The tiered generation of OTA update packages specifically includes: generating incremental update packages for end-side devices that are predicted to require incremental updates and have high feedback priority; generating full update packages for end-side devices that are predicted to require full updates and have high feedback priority; and temporarily not generating OTA update packages for end-side devices that are predicted to require no updates but have low-priority feedback.

[0015] In a third aspect of the invention, an edge AI model OTA update and optimization system based on edge-cloud collaboration is also proposed, comprising an edge device and a cloud server, which communicate with each other via a wireless network; the edge device is used to execute the edge AI model OTA update and optimization method based on edge-cloud collaboration described in the first aspect; the cloud server is used to execute the edge AI model OTA update and optimization method based on edge-cloud collaboration described in the second aspect.

[0016] Furthermore, in the aforementioned edge-cloud collaborative edge AI model OTA update and optimization system, the edge device includes a data acquisition module, an inference module, a lightweight prediction module, a user feedback module, a data encryption upload module, an OTA update module, a verification module, and a storage module.

[0017] Furthermore, in the aforementioned edge-cloud collaborative edge AI model OTA update and optimization system, the cloud server includes a data receiving and decryption module, a data processing module, a model training module, an adaptability pre-verification module, an update package generation module, an OTA push module, a version management module, and a storage module.

[0018] Compared with the prior art, the present invention has at least the following technical effects: This invention achieves multimodal data acquisition, inference, and lightweight prediction on the edge through the collaborative operation of edge devices and cloud servers. It can predict model update needs and hardware compatibility in advance, solving the problems of fixed performance and insufficient dynamic adaptation capabilities in existing edge AI models. By collecting user feedback with dual tags, prioritizing it, and tiered encrypted uploading, it can accurately capture users' personalized needs, improving the effectiveness of feedback data while reducing bandwidth consumption and privacy leakage risks.

[0019] Furthermore, the cloud-based system completes multimodal fusion training, multi-version model optimization, and compatibility pre-verification based on encrypted feedback data. This ensures that model optimization aligns with real-world scenarios and device hardware conditions on the edge, preventing model incompatibility and excessive workload. By generating OTA update packages in a tiered manner and performing push, verification, replacement, and multi-dimensional validation, along with an automatic rollback mechanism for anomalies, the update package size is significantly reduced, update efficiency is improved, and the stable and reliable operation of edge devices is guaranteed. This comprehensive closed-loop iteration and version management mechanism enables continuous optimization of edge AI models, improving overall inference accuracy, scenario adaptability, update efficiency, and operational security. Attached Figure Description

[0020] Figure 1 This is a flowchart of an OTA update and optimization method for edge AI models based on edge-cloud collaboration in one embodiment of the present invention; Figure 2 This is a flowchart of an edge-side AI model OTA update and optimization method based on edge-cloud collaboration in another embodiment of the present invention; Figure 3 This is a schematic diagram of the overall process of collaboration between the end-side device and the cloud server in one embodiment of the present invention; Figure 4 This is a schematic diagram of the workflow of the lightweight prediction module of the end-side device in the end-side AI model OTA update and optimization system based on end-cloud collaboration in one embodiment of the present invention. Figure 5 This is an execution logic diagram of the edge AI model OTA update and optimization system based on edge-cloud collaboration on the cloud server in one embodiment of the present invention; Figure 6 This is a detailed flowchart of the closed-loop iteration between the edge device and the cloud server in the OTA update and optimization of the edge AI model based on edge-cloud collaboration in one embodiment of the present invention. Detailed Implementation

[0021] The following will describe in more detail, with reference to the schematic diagrams, a method and system for OTA update and optimization of edge AI models based on edge-cloud collaboration, according to the present invention. Preferred embodiments of the invention are shown. It should be understood that those skilled in the art can modify the invention described herein while still achieving its advantageous effects. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0022] For clarity, not all features of the actual embodiments are described. In the following description, well-known functions and structures are not detailed in detail, as they would obscure the invention with unnecessary detail. It should be understood that in the development of any actual embodiment, numerous implementation details must be made to achieve the developer's specific objectives, such as changes from one embodiment to another according to limitations related to the system or business. Furthermore, it should be understood that such development work may be complex and time-consuming, but is merely routine work for those skilled in the art.

[0023] The invention is described more specifically by way of example in the following paragraphs with reference to the accompanying drawings. The advantages and features of the invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0024] Based on the teachings of this specification, those skilled in the art can form new technical solutions through cross-combination of different implementation methods without creating technical contradictions. Such variations should all be considered to fall within the protection scope of this invention.

[0025] Example 1 This embodiment discloses an OTA update optimization method for edge AI models based on edge-cloud collaboration. This method can solve the problems of low update efficiency, poor hardware compatibility, inability to respond to personalized user needs, insufficient edge-cloud collaboration, and weak data transmission security in existing edge AI models. (Refer to...) Figure 1 and Figure 3 This method is executed by the end-side device, and the specific steps are as follows: S101: The edge device collects and preprocesses multimodal data through the edge AI model, infers inference results and inference logs, performs lightweight prediction based on the device operation data and the inference logs, and generates prediction results; S102: The terminal device collects the user's dual-labeled feedback information on the inference result and the prediction result, associates the feedback information with the corresponding inference result, inference log, and prediction result to form a feedback dataset, and sorts the feedback data in the feedback dataset by priority to obtain the feedback priority; S103: The end-side device filters the feedback data according to the feedback priority, and uploads it to the cloud server after hierarchical encryption; S104: The edge device receives the OTA update package pushed by the cloud server, completes the update package verification, download and model replacement, performs multi-dimensional verification of the model after the update, and rolls back to the model before the update in case of an anomaly. S105: The terminal device continues to execute the above steps and uploads the updated status and verification results to the cloud server.

[0026] In this embodiment, the edge devices include smart terminals, industrial edge devices, IoT terminals, and smart wearable devices.

[0027] For step S101, after the terminal device starts running, it first deploys and loads a preset initial AI model as the terminal AI model. In this embodiment, the terminal AI model includes an image recognition model, a speech recognition model, a natural language processing model, and a behavior analysis model. It can be flexibly deployed according to the specific application scenario.

[0028] Once the device enters operational mode, it collects multimodal data from the scene in real time through the edge AI model. This multimodal data includes environmental data, user operation data, and device operation data. In this embodiment, the environmental data includes physical parameters of the scene where the edge device is located (such as light intensity, temperature, and humidity), as well as scene images and audio data. User operation data includes user commands and interaction data issued to the edge AI application. Device operation data includes the edge device's own computing power utilization, remaining storage space, memory usage, power consumption rate, and other operational status information.

[0029] After acquiring multimodal data, the edge device performs hierarchical preprocessing on the data. The preprocessing includes sequentially performing multimodal data separation, data cleaning, noise reduction, normalization, and format conversion to remove redundant information and abnormal interference data from the collected data, unify the data format and numerical range, so that the data is accurate and effective and meets the inference input requirements of the edge AI model.

[0030] After preprocessing, the initial AI model performs on-site inference based on the standardized data and outputs the corresponding inference results. Simultaneously, the edge device records all relevant parameters throughout the inference process, including inference time, inference accuracy, data type, corresponding environmental parameters, and device operating load information, forming an inference log.

[0031] In addition, the edge device has a built-in hardware compatibility prediction module. Based on the collected device operation data and inference log data, it comprehensively judges whether the current model needs to be updated, the bandwidth and storage resources required for the update, and whether the updated model can adapt to the current device hardware conditions, ultimately generating a prediction result. In this embodiment, the specific prediction results include four categories: no update required, incremental update recommended, full update recommended, and hardware compatibility risk warning. After the prediction result is generated, it is stored synchronously with the inference results and inference log data, providing a direct basis for the selection of subsequent update strategies and avoiding invalid updates and hardware incompatibility issues from the source.

[0032] In step S102, after completing the inference and lightweight prediction in step S101, the edge device simultaneously displays the generated inference and prediction results to the user. A visual user feedback entry point is also provided on the interactive interface, offering a convenient feedback channel for the user. The user can simultaneously mark both the inference and prediction results using any of the following feedback methods: click, voice feedback, or text input feedback. Specifically, the inference result can be marked as correct, incorrect, or ambiguous; the prediction result can be marked as reasonable, unreasonable, or indicating a confirmed adaptation risk.

[0033] While completing the labeling, users can also submit corresponding correction information for incorrect reasoning, fuzzy reasoning, and unreasonable predictions. The correction information includes the correct reasoning results manually entered by the user, the adjustment opinions on the reasoning results, and the correction suggestions for the prediction results, thereby clarifying the reasoning defects and prediction biases of the edge AI model.

[0034] During the process of collecting user feedback information on the dual marking of the inference result and the prediction result, the edge device will perform validity verification on the feedback information, filter out invalid feedback data such as malicious marking, meaningless marking, and repeated random filling, and ensure that the feedback information used for subsequent model optimization is true and reliable.

[0035] In addition, to enhance user engagement, the device is equipped with a feedback incentive mechanism to guide users to provide more standardized and comprehensive tagging feedback.

[0036] Furthermore, the edge device will associate and integrate the effective feedback information with the corresponding inference results, inference logs, and prediction results to form a feedback dataset.

[0037] After constructing the feedback dataset, the edge device prioritizes the feedback data within the dataset to obtain feedback priorities. In this embodiment, the feedback priorities are ranked according to at least one of the following factors: feedback frequency, reasonableness of correction, severity of inference error, and scope of influence of prediction deviation, thus dividing the feedback data into high, medium, and low levels.

[0038] In this embodiment, frequently occurring erroneous reasoning and serious prediction deviations with a wide impact are given high-priority feedback; infrequent erroneous reasoning and minor prediction deviations with a small impact are given medium-priority feedback; and ambiguous reasoning results and supplementary suggestions for reasonable predictions are given low-priority feedback.

[0039] By prioritizing the data, the cloud can use high-priority feedback data during subsequent model optimization and training, making the model optimization direction more aligned with real-world scenarios and user needs.

[0040] For step S103, the edge device first performs priority filtering on the formed feedback dataset based on the feedback priority sorting result of step S102. Specifically, the device prioritizes uploading high-priority and medium-priority feedback data, while low-priority feedback data is temporarily stored in the edge's local storage space and uploaded uniformly when the batch upload conditions are met later, avoiding resource waste caused by frequent transmission of scattered small data.

[0041] After filtering, the edge device performs hierarchical encryption on the filtered feedback data, employing a dual encryption mechanism combining symmetric encryption and differential privacy to ensure data security and user privacy. High-priority feedback data is encrypted using the AES-256 symmetric encryption algorithm, with differential privacy noise added to prevent user privacy leaks; medium-priority or low-priority feedback data is encrypted using the conventional AES symmetric encryption algorithm, improving processing efficiency while maintaining security. After encryption, the edge device uploads the encrypted feedback dataset to the cloud server via wireless networks such as 5G, WiFi, and NBIoT. The cloud server receives, parses, and processes the data uploaded by the edge device through a model training platform, data storage platform, and multimodal adaptation platform.

[0042] To further conserve bandwidth resources, the edge device compresses the feedback dataset before uploading. This embodiment uses the LZ77 lossless compression algorithm to compress the data volume and reduce data transmission. Simultaneously, the edge device employs a breakpoint resume mechanism. If a network interruption occurs during transmission, the upload can resume from the point of interruption after the network is restored, without retransmitting all data, thus avoiding data loss and duplicate transmissions. Furthermore, the device dynamically adjusts the upload rate based on the current network status, increasing the upload speed to improve efficiency when bandwidth is sufficient and decreasing the upload rate when bandwidth is limited to avoid affecting the normal operation of edge AI applications and user experience.

[0043] In step S104, after receiving the OTA update notification from the cloud server, the terminal device, based on its own network status, the user-preset update strategy, and the prediction result generated in step S101, autonomously chooses to execute the update immediately or delay the update.

[0044] Before the update begins, the device performs a dual check on the OTA update package, verifying both its integrity and compatibility. Specifically, it verifies the completeness of the update package using a verification code and compares the list of compatible devices within the update package with the hardware compatibility report to confirm that the device's hardware matches the update package. If the verification fails or the device is incompatible, the device rejects the update, notifies the user, and reports the verification failure to the cloud server. If the verification passes and the device is compatible, the update package download begins. During the download process, the device employs a resume and incremental download mechanism. If the network is unexpectedly interrupted, the download can continue after the connection is restored, without needing to re-download the entire update package. Furthermore, the entire process is performed in the background, prioritizing the normal operation of the AI ​​application on the device and minimizing disruption to normal user experience.

[0045] After the update package is downloaded, the edge device automatically replaces the model files, achieving OTA update of the edge AI model. Following the update, the edge device automatically starts the optimized AI model, collects multimodal data from the field, and performs multi-dimensional inference tests to verify the model's inference accuracy, operational stability, hardware compatibility, and low-power performance. During testing, the edge device synchronously records test data and collects secondary user feedback on the updated model. The update results, test data, and user feedback are uploaded to the cloud server. The cloud server then records the update status of each edge device and generates an update log.

[0046] If any anomalies are detected in the model during testing, such as a significant decrease in inference accuracy, lag, excessive computing power consumption, or rapid battery drain, the edge device will automatically trigger a rollback mechanism to restore the model to its initial version before the update, preventing any impact on the normal use of edge AI applications. Simultaneously, the edge device will report the anomaly information to the cloud server. The cloud server will then optimize the model and adjust the parameters based on the anomaly information. Once optimization is complete, it will push a compatible update package back to the device to ensure both the effectiveness of the model update and the safety of device operation.

[0047] For step S105, after completing a single model update and verification, the end device will not stop running, but will repeatedly execute all processes from S101 to S104, and continue to enter the next optimization cycle.

[0048] During continuous iteration, edge devices constantly collect inference results, multimodal data, user operation data, and newly generated user-labeled feedback information, and synchronously upload this data to the cloud server. This provides realistic, dynamic, and scenario-appropriate data for the continuous optimization and training of the cloud model. Simultaneously, edge devices report the update status, verification results, and user feedback from each round of use to the cloud server in real time. This enables the cloud to continuously optimize model parameters, adjust adaptation strategies, and improve version management based on global data from multiple devices and scenarios, achieving dynamic iterative upgrades of the edge AI model.

[0049] Example 2 This embodiment provides an edge-side AI model OTA update and optimization method based on edge-cloud collaboration. This method is executed by a cloud server and works in conjunction with the edge-side device process in Embodiment 1 to jointly complete the closed-loop optimization and update of the model. (Refer to...) Figure 2 and Figure 3 This method is executed by a cloud server, and the specific steps are as follows: S201: The cloud server receives the hierarchically encrypted feedback dataset uploaded by the end device and performs hierarchical decryption to obtain the original feedback dataset; it selects valid feedback data from the original feedback dataset, performs multimodal data fusion to generate a training dataset, trains a multi-version optimized model based on the initial AI model and the training dataset, and performs adaptability pre-verification on the multi-version optimized model to remove the optimized models that fail the verification. S202: The cloud server performs lightweight processing on the optimized model that has passed the verification, generates OTA update packages according to the prediction results and feedback priority uploaded by the terminal device, and encrypts and verifies the OTA update packages. S203: The cloud server pushes the OTA update package to the corresponding end device in a differentiated manner; S204: The cloud server receives the update status and verification results returned by the terminal device, manages the model version, and supports version rollback, incremental iteration and batch updates to maintain closed-loop iteration.

[0050] In step S201, the cloud server receives the feedback dataset uploaded by one or more end-devices, which has undergone hierarchical encryption as described in step S103 of Embodiment 1. First, the encrypted data is decrypted hierarchically according to its encryption level, sequentially restoring the original feedback data of high priority, medium priority, and low priority, thus obtaining the original feedback dataset suitable for model training. Subsequently, the cloud server performs secondary filtering and preprocessing on the original feedback dataset, selecting valid feedback data. Valid feedback data here refers to user-labeled feedback data and corresponding inference and prediction data that have passed validity verification. Redundant and abnormal data are then removed to ensure the reliability of subsequent training data.

[0051] After data filtering, the cloud server performs multimodal data fusion on the valid feedback data, specifically including cross-modal data alignment, feature fusion, and sample balancing. Environmental data, user behavior data, and inference logs from different devices are uniformly standardized and feature-enhanced to generate a training dataset for the multimodal model. This multimodal data fusion effectively improves the diversity and completeness of the training dataset, avoids overfitting during model training, and enhances the model's generalization ability and adaptability to multiple scenarios.

[0052] Furthermore, the cloud server, through its built-in model training platform, uses a pre-set initial AI model as a foundation and the training dataset of the aforementioned multimodal models for model update and optimization training. During training, transfer learning algorithms are employed to fine-tune the parameters of the initial AI model, ensuring training effectiveness while reducing cloud computing power consumption and shortening the overall training cycle. Simultaneously, training loss functions (such as cross-entropy loss function) and accuracy evaluation metrics (such as accuracy, recall, F1 score, etc.) are set during training to monitor loss values ​​and accuracy metrics in real time. In this embodiment, training stops when the loss value drops to a preset threshold and the accuracy metrics stabilize, resulting in multiple versions of optimized models optimized for different hardware and scenarios. During this process, the cloud can further aggregate feedback data from multiple edge devices to achieve global data-driven model optimization, making the model more suitable for the on-site environment and user needs of various edge devices.

[0053] After training, the cloud server performs pre-validation of the optimized model for each version. This pre-validation includes verifying model compatibility, computational power consumption, and storage usage. Specifically, it verifies the model's compatibility, computational power consumption, and storage usage on the corresponding edge device by combining the hardware parameters uploaded by the edge device, the remaining computational power, storage capacity, and prediction results. In this embodiment, model versions that exceed the device's capacity in terms of computational power consumption, exceed the remaining storage space, or are incompatible with the model are directly removed by the cloud server, retaining only the optimized model that can stably adapt to the hardware limitations of the corresponding edge device. Simultaneously, the cloud server fully records all relevant parameters during the model training process, including training time, training accuracy, and changes in loss value, forming a training log for subsequent model optimization and problem troubleshooting.

[0054] In step S202, the cloud server performs lightweighting processing on the optimized model that passed the adaptability pre-verification in step S201, based on the hardware parameters (e.g., computing power, storage capacity, memory size) of the corresponding edge device. This lightweighting processing includes model pruning and quantization. Specifically, model pruning removes redundant parameters from the model that have a minor impact on inference accuracy, while quantization converts the model parameters from a high-precision format (e.g., 32-bit) to a low-precision format (e.g., 8-bit). This significantly reduces the model size and computing power consumption while maintaining essentially no decrease in inference accuracy, enabling the optimized model to adapt to the hardware limitations of the edge device.

[0055] After completing the lightweighting of the model, the cloud server generates OTA update packages in a tiered manner based on the prediction results and feedback priorities uploaded by the edge devices.

[0056] In this embodiment, the hierarchical generation of OTA update packages specifically includes: generating incremental update packages for end-devices whose prediction result is a suggested incremental update and whose feedback priority is high; generating full update packages for end-devices whose prediction result is a suggested full update and whose feedback priority is high; and not generating OTA update packages for end-devices whose prediction result is no need for updates but have low-priority feedback, but only temporarily storing the optimized model parameters in the cloud, waiting for batch updates to be performed when the conditions are met later.

[0057] After the update package is generated, the cloud server encrypts and verifies its integrity. The cloud server uses an asymmetric encryption algorithm (such as RSA) to encrypt the OTA update package to ensure its security during network transmission and prevent data theft or tampering. Simultaneously, an expiration date is set for the OTA update package to prevent expired updates from being used unauthorizedly.

[0058] In this embodiment, the generated OTA update package contains an optimized edge AI model that matches the device hardware, model update instructions, and update verification information. The update verification information in the OTA update package includes a checksum, a list of compatible devices, and a hardware compatibility report. These are used to verify the integrity of the update package when received by the edge device, confirm device compatibility, and prevent the update package from being used on incompatible edge devices, which could lead to update failure or malfunction.

[0059] In step S203, after completing the hierarchical generation, encryption, and verification of the OTA update package, the cloud server accurately delivers the OTA update package matching the device to the target device based on the hardware type of the end device, the actual application scenario, and the prediction results reported by the end device. During the push process, the cloud strictly matches the device hardware parameters, application scenario requirements, prediction result type, and feedback priority, ensuring that each end device only receives update packages that are adapted to its own operating conditions and meet current optimization needs, avoiding the push of incompatible, unnecessary, or resource-overloaded update content to the device.

[0060] In this embodiment, the cloud pushes incremental update packages and full update packages to the corresponding edge devices according to pre-set push rules. Specifically, for devices that are determined to require incremental updates and have high feedback priority, only incremental update packages are sent; for devices that require full updates and have high-priority feedback, full update packages that have undergone adaptation verification are sent; for devices that are predicted not to need updates or only have low-priority feedback, no update packages are pushed in this round, and only status monitoring and data synchronization are maintained. Through this differentiated push method, the bandwidth consumption caused by invalid data transmission can be significantly reduced, alleviating the storage and computing power pressure on edge devices.

[0061] In step S204, the cloud server receives and parses the update status, verification results, user feedback, and update anomaly information returned by the end-side devices, and comprehensively understands the status information of each end-side device, such as whether the model update is successful, the stability of operation, the accuracy of inference, and whether automatic rollback is triggered.

[0062] In addition, the cloud server uses a built-in model version management module to record and categorize all generated optimized model versions, model update history, optimization records, and adaptation status for each edge device, forming a version archive. This module has version management capabilities, supporting model version rollback, incremental iteration, and batch updates.

[0063] Specifically, when a device reports an update anomaly or model incompatibility, the problematic version can be quickly located and a rollback mechanism can be initiated. Based on continuously collected feedback data from the device, the model can be continuously optimized and iterated. Simultaneously, devices can be grouped according to application scenarios and hardware specifications to enable batch update pushes, effectively reducing the operational risks associated with large-scale centralized updates.

[0064] Furthermore, after completing version management, the cloud server will combine the current update status and verification results with the new round of on-site inference data, multimodal data, and user-labeled feedback information uploaded by the edge device, and re-enter the process of steps S201-S203. This will cooperate with the process of steps S101-S104 of the edge device to form a closed-loop iterative mechanism of edge prediction - user feedback - cloud optimization - OTA update - iterative verification. This will enable long-term, dynamic, and adaptive iterative upgrades of the edge AI model, continuously improving inference accuracy and hardware compatibility.

[0065] Example 3 Reference Figure 4-6 This embodiment discloses an edge-side AI model OTA update and optimization system based on edge-cloud collaboration, including an edge device and a cloud server, which are connected via a wireless network. The edge device is used to execute the method in Embodiment 1 above, and the cloud server is used to execute the method in Embodiment 2 above.

[0066] The edge device includes a data acquisition module, an inference module, a lightweight prediction module, a user feedback module, a data encryption upload module, an OTA update module, a verification module, and a storage module.

[0067] The data acquisition module is used to collect multimodal data from the field. After performing hierarchical preprocessing on the data, it sends it to the inference module and the lightweight prediction module. The multimodal data includes field environmental data, user operation data, and equipment operation data.

[0068] The inference module is equipped with an initial AI model. It receives preprocessed data from the data acquisition module, performs on-site inference operations, obtains inference results, records inference logs, and sends the inference results and inference logs to the lightweight prediction module and the user feedback module.

[0069] The lightweight prediction module uses device operation data and inference logs to predict model update needs, resource requirements, and hardware compatibility, generating prediction results which are then sent to the user feedback module and the data encryption upload module. These prediction results include suggestions such as no update required, incremental update recommended, full update recommended, and hardware compatibility risk warnings.

[0070] The user feedback module is used to display reasoning results and prediction results, provide an entry point for user-marked feedback, collect user-marked feedback information, prioritize and validate the feedback data, and send the feedback dataset to the data encryption and upload module.

[0071] The data encryption upload module is used to filter and encrypt the feedback dataset in a hierarchical manner, and upload the encrypted and compressed feedback dataset to the cloud server via wireless network.

[0072] The OTA update module is used to receive differentiated OTA update packages pushed by the cloud server, verify and download the update packages, and complete the replacement and update of the AI ​​model on the device side.

[0073] The verification module is used to perform multi-dimensional inference tests on the optimized AI model after the model update is completed, verify the model performance, collect secondary feedback from users, and send the update results, test data and secondary feedback to the cloud server. If model abnormalities are found, the model rollback operation is triggered.

[0074] The storage module is used to store the initial AI model, the optimized AI model, the feedback dataset, the inference log data, the update package and the update log, and temporarily store low-priority feedback data.

[0075] The cloud server includes a data receiving and decryption module, a data processing module, a model training module, a multimodal adaptation module, an adaptability pre-verification module, an update package generation module, an OTA push module, a version management module, and a storage module.

[0076] The data receiving and decryption module is used to receive encrypted feedback datasets uploaded by the receiving end device, perform hierarchical decryption and decompression processing on them, and obtain the original feedback datasets.

[0077] The data processing module is used to perform secondary filtering, preprocessing, and multimodal data fusion processing on the original feedback dataset to generate a multimodal model training dataset.

[0078] The model training module is used to update and optimize the model based on the initial AI model, using a multimodal model training dataset to obtain multiple versions of optimized edge AI models, and to record training logs.

[0079] The multimodal adaptation module is used to perform scenario- and hardware-specific optimizations on the optimization model for different end-device hardware parameters and application scenarios, generating multiple versions of the adaptation model.

[0080] The adaptability pre-verification module is used to perform adaptability pre-verification on multi-version optimized models, eliminate models that do not meet the adaptability standards, return non-compliant models to the multimodal adaptation module for re-optimization, and push compliant models to the update package generation module to ensure model compatibility with edge device hardware.

[0081] The update package generation module is used to perform targeted lightweight processing on multi-version optimization models that meet the compatibility standards. Based on the prediction results and feedback priorities, it generates differentiated OTA update packages in stages and performs encryption and verification processing on the update packages.

[0082] The OTA push module is used to push OTA update packages to the corresponding end devices based on the hardware type, application scenario, and prediction results of the end devices, and to receive update results, test data, and secondary feedback from the end devices, and record update logs.

[0083] The version management module is used to classify and manage model versions, record model update history and adaptation status, and support model version rollback, incremental iteration and batch updates.

[0084] The storage module is used to store the feedback dataset, multimodal model training dataset, initial AI model, multi-version optimized AI model, training log, update log and model version information uploaded by each end device.

[0085] In summary, this system, through the collaborative operation of edge devices and cloud servers, enables multimodal data acquisition, inference, and lightweight prediction on the edge. It can proactively anticipate model update needs and hardware compatibility, addressing the issues of fixed performance and insufficient dynamic adaptation capabilities in existing edge AI models. By collecting user feedback with dual tags, prioritizing it, and tiered encrypted uploading, it can accurately capture users' personalized needs, improving the effectiveness of feedback data while reducing bandwidth consumption and privacy risks.

[0086] Furthermore, the cloud-based system completes multimodal fusion training, multi-version model optimization, and compatibility pre-verification based on encrypted feedback data. This ensures that model optimization aligns with real-world scenarios and device hardware conditions on the edge, preventing model incompatibility and excessive workload. By generating OTA update packages in a tiered manner and performing push, verification, replacement, and multi-dimensional validation, along with an automatic rollback mechanism for anomalies, the update package size is significantly reduced, update efficiency is improved, and the stable and reliable operation of edge devices is guaranteed. This comprehensive closed-loop iteration and version management mechanism enables continuous optimization of edge AI models, improving overall inference accuracy, scenario adaptability, update efficiency, and operational security.

[0087] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A method for OTA update and optimization of edge AI models based on edge-cloud collaboration, characterized in that, include: The edge device collects and preprocesses multimodal data through the edge AI model, infers inference results and inference logs, and performs lightweight prediction based on the device operation data and the inference logs to generate prediction results. The edge device collects user feedback information with dual tags for the inference result and the prediction result, associates the feedback information with the corresponding inference result, inference log and prediction result to form a feedback dataset, and sorts the feedback data in the feedback dataset by priority to obtain the feedback priority; The edge device filters the feedback data according to the feedback priority, and uploads it to the cloud server after hierarchical encryption. The edge device receives OTA update packages pushed by the cloud server in a differentiated manner, completes update package verification, download and model replacement, performs multi-dimensional verification of the model after the update, and rolls back to the model before the update in case of an anomaly. The edge device continues to perform the above steps and uploads the updated status and verification results to the cloud server.

2. The method according to claim 1, characterized in that, The multimodal data includes on-site environmental data, user operation data, and equipment operation data; the preprocessing includes multimodal data separation, data cleaning, noise reduction, normalization, and format conversion; the prediction results include no need to update, incremental update recommendation, full update recommendation, and hardware compatibility risk warning.

3. The method according to claim 1, characterized in that, User feedback methods include click markers, voice feedback, and text input feedback; The process of collecting user feedback information on the dual-labeled reasoning results and prediction results also includes validating the feedback information and filtering out invalid feedback. The feedback priority is ranked according to at least one of the following factors: feedback frequency, reasonableness of correction, severity of reasoning error, and scope of influence of prediction deviation, and the feedback data is divided into three levels: high, medium and low.

4. The method according to claim 1, characterized in that, The hierarchical encryption adopts a mechanism that combines symmetric encryption and differential privacy; high-priority feedback data is encrypted using the AES256 symmetric encryption algorithm and differential privacy noise is added, while medium or low-priority feedback data is encrypted using the conventional AES symmetric encryption algorithm; the end-side device uses lossless compression algorithm, breakpoint resume mechanism and dynamic rate adjustment to upload the feedback data to the cloud server.

5. A method for OTA update and optimization of edge AI models based on edge-cloud collaboration, characterized in that, include: The cloud server receives the hierarchically encrypted feedback dataset uploaded by the end device and performs hierarchical decryption to obtain the original feedback dataset. Valid feedback data is selected from the original feedback dataset, and multimodal data fusion is performed to generate a training dataset. Multiple versions of optimized models are trained based on the initial AI model and the training dataset. Adaptability pre-verification is performed on the multiple versions of optimized models, and optimized models that fail the verification are removed. The cloud server performs lightweight processing on the optimized model that has passed the verification, generates OTA update packages according to the prediction results and feedback priorities uploaded by the terminal devices, and encrypts and verifies the OTA update packages. The cloud server pushes the OTA update package to the corresponding end devices in a differentiated manner; The cloud server receives the update status and verification results returned by the terminal device, manages the model version, and supports version rollback, incremental iteration and batch updates.

6. The method according to claim 5, characterized in that, The multimodal data fusion includes cross-modal data alignment, feature fusion, and sample equalization; the multi-version optimized model is obtained by fine-tuning the parameters of the initial AI model through transfer learning algorithms; the adaptability pre-verification includes verifying the model's operational compatibility, computing power consumption, and storage usage.

7. The method according to claim 5, characterized in that, The lightweighting process includes model pruning and quantization; The OTA update package is encrypted using an asymmetric encryption algorithm, has an expiration date, and includes a verification code and a hardware compatibility report. The tiered generation of OTA update packages specifically includes: generating incremental update packages for end-side devices that are predicted to require incremental updates and have high feedback priority; generating full update packages for end-side devices that are predicted to require full updates and have high feedback priority; and temporarily not generating OTA update packages for end-side devices that are predicted to require no updates but have low-priority feedback.

8. An edge-side AI model OTA update and optimization system based on edge-cloud collaboration, characterized in that, The device includes an end-side device and a cloud server, which communicate with each other via a wireless network; the end-side device is used to perform the method according to any one of claims 1-4; the cloud server is used to perform the method according to any one of claims 5-7.

9. The system according to claim 8, characterized in that, The edge device includes a data acquisition module, an inference module, a lightweight prediction module, a user feedback module, a data encryption upload module, an OTA update module, a verification module, and a storage module.

10. The system according to claim 8, characterized in that, The cloud server includes a data receiving and decryption module, a data processing module, a model training module, an adaptability pre-verification module, an update package generation module, an OTA push module, a version management module, and a storage module.

Images