Multimodal ai model continuous learning and management method, system, device, and medium
By inserting a lightweight adaptation parameter module into the smart park AI model for incremental training, and combining knowledge distillation and historical sample memory, a version graph is constructed and a secure canary release is performed. This solves the catastrophic forgetting and coupling problems of smart park AI models in data drift and new scene recognition, and achieves efficient and interpretable model iteration and management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI YIBANG INTELLIGENT TECH CO LTD
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-09
AI Technical Summary
Existing AI models for smart parks suffer from problems such as catastrophic forgetting, high computational costs, inaccurate version management, and high risks when facing data distribution drift and the need to identify new scenarios. In particular, it is difficult to maintain model coordination and balance in the incremental updates of multimodal models.
We employ a multimodal AI model for continuous learning and management. We use a lightweight adaptive parameter module for incremental training, combined with knowledge distillation loss and a historical key sample memory bank, to construct a version graph of a directed acyclic graph. We then use a secure canary release pipeline for gradual deployment and automatic rollback.
It enables high-frequency, low-risk AI model iteration, improves model performance and stability, increases update efficiency by more than 10 times, and improves model accuracy by 5.8% quarter-on-quarter. It avoids catastrophic forgetting and model coupling problems, and supports flexible combination and traceable version management.
Smart Images
Figure CN122174920A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence operation and maintenance and smart park management, and in particular to a method, system, device and medium for continuous learning and management of multimodal AI models. Background Technology
[0002] In smart parks, artificial intelligence models are widely used for various tasks such as video analysis, voice interaction, and time series prediction. These models typically adopt an "offline training, one-time deployment, long-term operation" model. However, the park's business and environment are constantly changing, causing online models to face data distribution drift and the need to recognize new scenarios, resulting in model performance degradation over time.
[0003] To address this issue, the current mainstream approach is to manually collect new data, retrain the entire model, and then completely replace and deploy it. This approach has significant drawbacks: First, when the model is fine-tuned on new data, it overwrites previously learned knowledge, leading to "catastrophic forgetting." Second, full retraining is computationally expensive and time-consuming, failing to meet the needs of rapid business iteration. Third, traditional model version management often involves overall version snapshots, making it impossible to accurately track which specific sub-capabilities of the model were improved by a single update. Finally, directly replacing and deploying a new model entirely carries high risks, lacking incremental verification and safe rollback mechanisms, potentially leading to global business failures.
[0004] Meanwhile, to overcome the drawbacks of full replacement, the industry tries to adopt incremental update methods. However, for multimodal fusion models such as vision and speech, the various modules are interdependent. If only a single modality is updated locally, it is easy to disrupt the overall coordination and balance of the model, leading to unstable or degraded performance. Existing methods are difficult to achieve safe modular incremental evolution.
[0005] Therefore, there is an urgent need for a solution that can support the safe, efficient, traceable, and flexibly combinable continuous learning and version management of AI models in smart parks, while avoiding performance degradation, especially catastrophic forgetting. Summary of the Invention
[0006] The technical problem to be solved by the present invention is to provide a method, system, device and medium for continuous learning and management of multimodal AI models, in view of the above-mentioned defects of the prior art.
[0007] To achieve the above objectives, in a first aspect, the present invention provides a method for continuous learning and management of multimodal AI models, the method comprising the following steps: S1, in response to a model update trigger event, retrieves the incremental data corresponding to that trigger event.
[0008] S2, based on the incremental data, the continuous learning engine module is used to incrementally train the multimodal AI model currently deployed in the production environment; the incremental training includes: inserting a lightweight adaptation parameter module corresponding to the triggering event into the backbone network of the multimodal AI model, freezing the original parameters of the backbone network, and training only the lightweight adaptation parameter module to generate a new functional unit associated with the triggering event; the lightweight adaptation parameter module is a pluggable adapter or a low-rank adaptive module.
[0009] S3, register the newly added functional unit as a capability node in the version graph; the version graph is a directed acyclic graph structure, wherein nodes represent capability units of the model, and directed edges represent the dependencies between capability units.
[0010] S4, deploy the new version model containing the newly added functional units online through the secure canary release pipeline; the secure canary release pipeline is configured to: verify the functionality and performance of the new version model in an isolated environment, and gradually expand its service scope in the production environment according to a preset strategy, while performing automatic rollback when functional abnormalities or performance degradation are detected.
[0011] In the multimodal AI model continuous learning and management method of the present invention, the incremental training further includes: When training the lightweight adaptation parameter module, a knowledge distillation loss term is introduced, using the model currently deployed in the production environment as the teacher model to constrain the output distribution of the new version model; The knowledge distillation loss term is the KL divergence loss.
[0012] In the multimodal AI model continuous learning and management method of the present invention, the incremental training further includes: Samples are extracted from the historical key sample memory and mixed with the incremental data for training to enhance the model's generalization ability to historical tasks. The historical key sample memory bank stores representative difficult samples from each historical task whose model prediction uncertainty is higher than a preset threshold or whose classification confidence is lower than a preset threshold.
[0013] In the multimodal AI model continuous learning and management method of the present invention, the capability node includes the following attribute information: node unique identifier, capability semantic description, functional or performance gain index, and a list of parent nodes it depends on. The model version running in the production environment corresponds to a snapshot of the parameter set defined by one or more capability nodes in the version map.
[0014] In the multimodal AI model continuous learning and management method of the present invention, the secure grayscale release pipeline specifically includes the following sub-steps: S41, Deploy the new version model in a sandbox environment, process the shadow traffic replicated in the production environment, and compare and test it with the production environment; S42, if the comparison test results meet the preset standards, then the production traffic is gradually switched to the new version model according to the preset gray-scale strategy; the gray-scale strategy includes at least one of the following: expanding the effective geographical area, user group ratio, or business scenario type in stages; S43. During the canary release process, the online functional and performance metrics of the new version model are monitored in real time. If any monitored metric deteriorates beyond a preset threshold, a rollback operation is automatically triggered to switch the online service back to the previous stable model version.
[0015] In the multimodal AI model continuous learning and management method of the present invention, the rollback operation is based on the version graph, and rolls the model back to the previous stable combination of capability nodes that does not contain the newly added capability nodes, or switches to the model version composed of the historical capability node combination specified by the user.
[0016] Secondly, the present invention also provides a multimodal AI model continuous learning and management system for implementing the method described above, comprising: The continuous learning engine module is configured to respond to model update trigger events and perform incremental training on multimodal AI models based on a dynamic parameter isolation mechanism. The version graph management module is configured to register, store, and manage capability nodes and their dependencies in the form of a directed acyclic graph. The secure canary release and control module is configured to perform shadow traffic comparison tests of the new version model, multi-stage gradual canary releases, and monitoring-based automatic rollback operations.
[0017] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multimodal AI model continuous learning and management method as described above.
[0018] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the multimodal AI model continuous learning and management method as described above.
[0019] The present invention has the following beneficial effects: This invention constructs a closed-loop continuous evolution mechanism for multimodal AI systems by decoupling the model update process into three stages: incremental training, capability registration, and secure deployment. At the training level, a dynamic parameter isolation mechanism is employed, training only lightweight adaptation modules and freezing the backbone network. This fundamentally avoids the overwriting of historical knowledge by new task learning, effectively solving the problems of catastrophic forgetting and coupling between different modal models. At the management level, functional units are modularized in the form of capability nodes, and version dependencies are organized through a directed acyclic graph structure, achieving traceability, interpretability, and composability of model evolution. At the deployment level, a secure canary release pipeline is used to verify new functions and performance in an isolated environment before gradually expanding the service scope, and automatic rollback is supported for anomalies, significantly reducing the business impact risk of model deployment on the production environment. In summary, this method enables high-frequency, low-risk, and interpretable continuous iteration of AI models, making it particularly suitable for complex scenarios with dynamically changing business needs, such as smart parks. After actual deployment and testing in the park, the old task indicators fluctuated by less than 1%. The average monthly update frequency of the park's AI model increased from 0.5 times to 4.2 times, improving update efficiency by more than 10 times. The average accuracy of the model increased by 5.8% quarter-on-quarter. Attached Figure Description
[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of a multimodal AI model continuous learning and management method provided in an embodiment of the present invention.
[0021] Figure 2 This is a schematic diagram of the structure of the multimodal AI model continuous learning and management system provided in an embodiment of the present invention.
[0022] Figure 3 This is a fragment of the spectral data from Example 1.
[0023] Figure 4 This is a fragment of the spectral data from Example 2.
[0024] Figure 5 This is a fragment of the spectral data from Example 3. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] The embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. It should be understood that the embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the invention.
[0027] This invention aims to provide a method for continuous learning and version management of multimodal AI models for park governance. By constructing a three-in-one technical system of elastic continuous learning mechanism, fine-grained version map, and secure release pipeline, it can achieve incremental enhancement of model capabilities, accurate traceability of version changes, and zero-risk verification of new model launches while avoiding catastrophic forgetting.
[0028] like Figure 1 As shown, this embodiment of the invention provides a method for continuous learning and management of a multimodal AI model, including the following steps: S1, in response to a model update trigger event, retrieves the incremental data corresponding to that trigger event.
[0029] Smart parks typically deploy numerous AI models, such as video analytics models for personnel recognition and behavior detection, speech understanding models for visitor voice interaction, time-series prediction models for energy consumption and load forecasting, and multimodal fusion models for security incident identification and equipment diagnostics. When new scenarios or data distribution shifts, such as the introduction of new visitor types, new equipment, or seasonal changes altering energy consumption patterns, model update trigger events are generated. After a model update event is triggered, corresponding incremental data needs to be acquired for subsequent incremental training of the model.
[0030] S2, based on the incremental data, the continuous learning engine module is used to perform incremental training on the multimodal AI model currently deployed in the production environment; the incremental training includes: inserting a lightweight adaptation parameter module corresponding to the triggering event into the backbone network of the multimodal AI model, freezing the original parameters of the backbone network, and training only the lightweight adaptation parameter module based on the dynamic parameter isolation mechanism to generate a new functional unit associated with the triggering event.
[0031] In this embodiment of the invention, the lightweight adaptation parameter module is a pluggable adapter or a low-rank adaptive module, and the number of parameters in the lightweight adaptation parameter module is less than 5% of the number of parameters in the production environment model. Pluggable adapters and low-rank adaptive modules are typical examples of efficient parameter fine-tuning techniques, introducing only a very small number of trainable parameters, significantly reducing training computation overhead and storage costs; the standardized and pluggable module structure facilitates reuse or removal across different tasks, enhancing the flexibility and scalability of the system architecture; and physical isolation from the backbone network parameters avoids the problem of overall imbalance easily caused by local updates.
[0032] In this embodiment of the invention, an independent, lightweight adaptive parameter module is inserted into the backbone network of the multimodal model for each new task. For example, when it is necessary to enhance the model's nighttime visual recognition capability, the corresponding pluggable adapter is inserted and trained only in the network layer of the visual branch; when it is necessary to add dialect recognition capability, another independent low-rank adaptive module is inserted only in the speech branch. During the training of these new modules, the parameters of the original model's backbone network and the adapter parameters of other modalities are frozen. This method strictly restricts updates to a specific, small subset of parameters, physically blocking the path where an update in one place affects the whole, thus avoiding catastrophic forgetting. Updating visual capabilities does not disturb the parameters of the speech branch, and vice versa. The parameters of the core fusion layer of the multimodal model are also typically protected, thereby ensuring the stability of the decision logic that comprehensively judges different modal features and avoiding the disruption of the overall multimodal inference balance due to single-modal updates.
[0033] In some embodiments of the present invention, the incremental training further includes: when training the lightweight adaptation parameter module, introducing a knowledge distillation loss term, using the model currently deployed in the production environment as the teacher model, and constraining the output distribution of the new version model; the knowledge distillation loss term is KL divergence loss.
[0034] In incremental training, a knowledge distillation loss term is introduced, using the mature output distribution of the teacher model for historical tasks as a soft label to guide the new model to maintain consistent response to inputs from old tasks while learning new capabilities. This effectively alleviates performance fluctuations in old tasks caused by training data distribution bias or insufficient samples, and further strengthens defenses against catastrophic forgetting. KL divergence, as a measure of probability distribution difference, has good mathematical properties and training stability, making it suitable for use in online / incremental learning scenarios.
[0035] In some embodiments of the present invention, the incremental training further includes: extracting samples from a historical key sample memory and mixing them with the incremental data for training, thereby enhancing the model's generalization ability to historical tasks; the historical key sample memory stores representative difficult samples in each historical task whose model prediction uncertainty is higher than a preset threshold or whose classification confidence is lower than a preset threshold. For example, for a personnel recognition model, images of employees under low light, in profile, wearing glasses, and other difficult-to-recognize employee images can be added to the historical key sample memory and extracted and replayed during incremental training of the personnel recognition model to consolidate the generalization ability of newly trained capability units to old tasks.
[0036] By extracting difficult samples with high uncertainty or low confidence from the historical key sample memory to participate in mixed training, we can specifically strengthen the weak links of the model in historical tasks and improve its generalization robustness. In addition, the historical key sample memory stores the most informative boundary samples, which significantly reduces the storage and computation burden compared with full playback, while maintaining the effectiveness of experience playback.
[0037] In this embodiment of the invention, a safe incremental approach is achieved through the synergistic implementation of three mechanisms: dynamic parameter isolation, knowledge distillation constraints, and selective experience replay. Dynamic parameter isolation is the structural foundation for modular evolution and resolving multimodal coupling issues. When new capabilities are needed, weights are not directly modified on the original model backbone network; instead, a dedicated lightweight adapter is inserted into the corresponding modal subnetwork. During training, all original backbone parameters and adapters for other modalities are frozen, and only the adapter corresponding to the current task is updated. Knowledge distillation constraints are crucial for combating catastrophic forgetting. When training a new adapter, the online model before the update is used as the teacher, allowing it to reason about incremental data and generate soft labels containing rich inter-category relationships. The loss function of the new model consists of the standard task loss (such as cross-entropy) and the KL divergence loss between the new model and the teacher's output. The latter acts as a powerful regularization term, forcing the new model to maintain its output logic within the knowledge system already mastered by the old model when adapting to new data. Selective experience replay is a reinforcement mechanism for consolidating historical memory. The system maintains a memory bank storing typical samples that the model struggled to judge in various historical tasks. During each incremental training, a portion of the memory samples are randomly selected and mixed with the newly added data for training, allowing the model to continuously review and learn from the past, thereby maintaining stable performance on old tasks over the long term.
[0038] S3, register the newly added functional unit as a capability node in the version graph; the version graph is a directed acyclic graph structure, where nodes represent capability units of the model, and directed edges represent dependencies between capability units. Capability nodes in the version graph contain the following attribute information: a unique node identifier, a capability semantic description, a functional or performance gain indicator, and a list of dependent parent nodes; the model version running in the production environment corresponds to a snapshot of the parameter set defined by one or more capability nodes in the version graph.
[0039] By defining structured attributes for capability nodes and mapping them to production environment versions in snapshot form, the model version structure is transformed from a black-box weight to a white-box capability list. Operations personnel can intuitively understand the specific business value of each update based on the visualized version graph. The version graph, composed of capability nodes with structured attributes, supports semantic-based version queries, comparisons, and combinations, improving the flexibility of model configuration. Furthermore, the version graph provides metadata support for accurate rollback, ensuring that rollback operations are not only technically feasible but also have clear business intent.
[0040] S4, deploy the new version model containing the newly added functional units online through a secure canary release pipeline; the secure canary release pipeline is configured to: verify the functionality and performance of the new version model in an isolated environment, and progressively expand its service scope in the production environment according to a preset strategy, while automatically rolling back when functional abnormalities or performance degradation are detected. The secure canary release pipeline specifically includes the following sub-steps: S41, Deploy the new version model in a sandbox environment, process the shadow traffic replicated in the production environment, and compare and test it with the production environment; S42, if the comparison test results meet the preset standards, then the production traffic is gradually switched to the new version model according to the preset gray-scale strategy; the gray-scale strategy includes at least one of the following: expanding the effective geographical area, user group ratio, or business scenario type in stages; S43. During the canary release process, the online functional and performance metrics of the new version model are monitored in real time. If any monitored metric deteriorates beyond a preset threshold, a rollback operation is automatically triggered, switching the online service back to the previous stable model version. The rollback operation is based on the version graph, reverting the model to the previous stable combination of capability nodes that does not include the newly added capability nodes, or switching to a model version composed of a historical combination of capability nodes specified by the user.
[0041] The new model initially runs in a completely isolated environment in "shadow mode," handling real-time replicated production traffic without impacting actual business operations. By comparing the processing results of the old and new models on the same batch of traffic, its performance can be comprehensively evaluated without risk. After passing the test, the new model begins to take over real traffic. The deployment process is gradual and controllable; for example, it is first implemented on 1% of non-core area traffic and observed for 24 hours; then expanded to 10% of office area traffic; and finally, a full deployment is carried out. Sufficient observation and adjustment time is allowed at each step. Throughout the deployment process, key indicators such as latency, accuracy, and error rate are monitored in real time. Once any indicator exceeds a safety threshold, the system will automatically trigger a rollback and quickly and accurately switch back to the previous stable version combination according to the version map.
[0042] Breaking down secure canary releases into a three-step process—shadow traffic testing, phased traffic switching, and real-time monitoring rollback—offers the following engineering advantages: Shadow traffic testing verifies the functional logic and interface compatibility of new models without zero business impact; multi-dimensional canary release strategies (region, target audience, scenario) allow for customized release pacing based on business sensitivity, balancing innovation speed and system stability; timely anomaly detection through metric monitoring ensures the timeliness and accuracy of rollback decisions; rollback operations based on fine-grained version maps can precisely revert to the previous stable state without problematic capabilities, avoiding the loss of all new features due to a single point of failure; flexible switching to any combination of historical capabilities greatly improves the granularity and efficiency of fault response; the rollback process is automated and auditable, reducing operational risks associated with manual intervention.
[0043] This invention constructs a closed-loop continuous evolution mechanism for multimodal AI systems by decoupling the model update process into three stages: incremental training, capability registration, and secure deployment. At the training level, a dynamic parameter isolation mechanism is employed, training only lightweight adaptation modules and freezing the backbone network. This fundamentally avoids the overwriting of historical knowledge by new task learning, effectively solving the problems of catastrophic forgetting and coupling between different modal models. At the management level, functional units are modularized in the form of capability nodes, and version dependencies are organized through a directed acyclic graph structure, achieving traceability, interpretability, and composability of model evolution. At the deployment level, a secure canary release pipeline is used to verify new functions and performance in an isolated environment before gradually expanding the service scope, and automatic rollback is supported for anomalies, significantly reducing the business impact risk of model deployment on the production environment. In summary, this method enables high-frequency, low-risk, and interpretable continuous iteration of AI models, making it particularly suitable for complex scenarios with dynamically changing business needs, such as smart parks.
[0044] like Figure 2As shown in the figure, this embodiment of the invention also provides a multimodal AI model continuous learning and management system for implementing the method described above. The multimodal AI model continuous learning and management system includes: a continuous learning engine module, configured to respond to model update trigger events and perform incremental training on the multimodal AI model based on a dynamic parameter isolation mechanism; a version graph management module, configured to register, store, and manage capability nodes and their dependencies in the form of a directed acyclic graph; and a secure canary release and control module, configured to perform shadow traffic comparison tests of the new version model, multi-stage progressive canary release, and monitoring-based automatic rollback operations.
[0045] The incremental training includes: inserting a lightweight adaptor parameter module corresponding to the triggering event into the backbone network of the multimodal AI model; freezing the original parameters of the backbone network; and training only the lightweight adaptor parameter module based on a dynamic parameter isolation mechanism to generate a new functional unit associated with the triggering event. The lightweight adaptor parameter module is a pluggable adapter or a low-rank adaptive module, and the number of parameters in the lightweight adaptor parameter module is less than 5% of the number of parameters in the production environment model.
[0046] In some embodiments of the present invention, the incremental training further includes: when training the lightweight adaptation parameter module, introducing a knowledge distillation loss term, using the model currently deployed in the production environment as the teacher model, and constraining the output distribution of the new version model; the knowledge distillation loss term is KL divergence loss.
[0047] In some embodiments of the present invention, the incremental training further includes: extracting samples from a historical key sample memory and mixing them with the incremental data for training, thereby enhancing the model's generalization ability to historical tasks; the historical key sample memory stores representative difficult samples in each historical task whose model prediction uncertainty is higher than a preset threshold or whose classification confidence is lower than a preset threshold. For example, for a personnel recognition model, images of employees under low light, in profile, wearing glasses, and other difficult-to-recognize employee images can be added to the historical key sample memory and extracted and replayed during incremental training of the personnel recognition model to consolidate the generalization ability of newly trained capability units to old tasks.
[0048] In this embodiment of the invention, the version graph is a directed acyclic graph structure, where nodes represent capability units of the model, and directed edges represent dependencies between capability units. Capability nodes in the version graph contain the following attribute information: a unique node identifier, a capability semantic description, a functional or performance gain indicator, and a list of dependent parent nodes; the model version running in the production environment corresponds to a snapshot of the parameter set defined by one or more capability nodes in the version graph.
[0049] The secure canary release pipeline is configured to: verify the functionality and performance of the new version model in an isolated environment, progressively expand its service scope in the production environment according to a preset strategy, and automatically roll back when functional anomalies or performance degradation are detected. The secure canary release pipeline specifically includes the following sub-steps: S41, Deploy the new version model in a sandbox environment, process the shadow traffic replicated in the production environment, and compare and test it with the production environment; S42, if the comparison test results meet the preset standards, then the production traffic is gradually switched to the new version model according to the preset gray-scale strategy; the gray-scale strategy includes at least one of the following: expanding the effective geographical area, user group ratio, or business scenario type in stages; S43. During the canary release process, the online functional and performance metrics of the new version model are monitored in real time. If any monitored metric deteriorates beyond a preset threshold, a rollback operation is automatically triggered, switching the online service back to the previous stable model version. The rollback operation is based on the version graph, reverting the model to the previous stable combination of capability nodes that does not include the newly added capability nodes, or switching to a model version composed of a historical combination of capability nodes specified by the user.
[0050] The continuous learning engine module encapsulates core technologies such as parameter isolation, knowledge distillation, and experience replay; the version graph management module enables efficient storage and retrieval of capability nodes, supporting complex dependency analysis and version assembly; and the secure canary release and control module integrates traffic replication, metric comparison, policy scheduling, and rollback execution, forming an end-to-end release control closed loop. These three modules work together to form a highly cohesive, loosely coupled multimodal AI model update and management platform, significantly improving the sustainable operation capability of multimodal AI systems in dynamic business environments.
[0051] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the multimodal AI model continuous learning and management method described above.
[0052] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the multimodal AI model continuous learning and management method described above.
[0053] After actual deployment and testing within the park, the average monthly update frequency of the park's AI model increased from 0.5 times to 4.2 times, and the average accuracy of the model improved by 5.8% quarter-on-quarter, with no business interruptions caused by model updates. Specific technical indicators and results are as follows: (1) Completely solve catastrophic forgetting: Through parameter isolation and knowledge distillation, the fluctuation of old task indicators is less than 1%; (2) Update efficiency is improved by more than 10 times: only <5% of parameters are trained, and the training time is reduced from hours to minutes; (3) Versions are explainable and composable: Managers can clearly understand the value of each update and assemble model capabilities as needed; (4) Low-risk deployment: Gray release + automatic rollback mechanism ensures business continuity; (5) Resource saving: No need to maintain a complete model copy for each small update, reducing storage costs by 80%.
[0054] Example 1: Facial recognition system supports new employee groups Scenario: The park's facial recognition access control system is already running stably. A new batch of outsourced employees, who regularly wear safety helmets, have joined the company. Their attire differs significantly from that of existing employees. The current model only achieves a 65% recognition rate for these new outsourced employees, while the preset recognition rate threshold is 95%.
[0055] Implementation process: (1) Trigger update The system automatically collects data on new employees and detects that the existing model's recognition rate for newly hired outsourced employees is only 65%, far below the preset recognition rate threshold, triggering an update.
[0056] (2) Continuous learning The elastic continuous learning engine module inserts pluggable adapters after each layer of the existing face recognition model. The number of parameters in each adapter is approximately 3% of the original model. All parameters of the original backbone network are frozen, and only the new adapters are trained. During training, new data is mixed with difficult-to-identify samples from the existing employee recognition memory, such as images of profile views or backlit faces, and soft labels from the original model's output are added for distillation constraints. After training, the recognition rate for new employees improved to 96%, while the recognition rate for existing employees decreased from 95.5% to 95.2%, which is within an acceptable range.
[0057] (3) Version registration Generate a capability node Node_NewStaff_Adapter and register it to the version graph. The capability node is described as "Supports face recognition for newly hired outsourced employees (safety helmet scenario)". The performance gain is: 31% improvement in accuracy for new groups and 0.3% decrease in accuracy for existing groups. The dependent parent nodes are "ViT_Base_v2" and "MemoryBank_v5". An example data structure is { "node_id": "Face_Adapter_20260201", "capability": "Supports facial recognition for new employees (including in helmet scenarios)," "metrics_gain": {"new_group_acc": "+31%", "old_group_acc_delta": "-0.3%"}, "dependencies": ["ViT_Base_v2", "MemoryBank_v5"] }, related version of the atlas fragments such as Figure 3 As shown, N represents the newly added node, and A and B represent its parent nodes.
[0058] (4) Secure release Sandbox testing: The access control logs (shadow traffic) from the past week were processed offline using the new model to confirm that there was no increase in false alarms.
[0059] Gray-scale release: The first phase lasts for 2 days, and the new model is only enabled in the east passage with the least traffic; the second phase lasts for 3 days, and the new logic is enabled for new employee IDs in all passages; the third phase is the full rollout.
[0060] Monitoring rollback: If the rejection rate is >2% for 10 minutes in any stage, the system will automatically roll back to a version that does not contain Node_NewStaff_Adapter.
[0061] (5) Effect solidification After one week, the system stabilizes, new nodes are marked as stable, and difficult samples from new employees are added to the memory. Older model versions are archived after 30 days, allowing for easy rollback at any time.
[0062] Example 2: Intelligent Customer Service Adds Dialect Support Scenario: The park's intelligent customer service system needs to add support for Cantonese and Minnan dialects.
[0063] Implementation process: (1) Trigger update Create new tasks manually: The intelligent customer service supports Cantonese and Hokkien.
[0064] (2) Continuous learning Separate low-order adaptive modules were created for Cantonese and Hokkien respectively on a speech encoder such as Wav2Vec 2.0. During training, the backbone network and the original Mandarin adapter were frozen, and the soft labels generated from the dialect data by the original model were distilled to maintain the acoustic model foundation. The mixed training data included newly added dialect data and difficult Mandarin samples from the memory.
[0065] (3) Version registration Create a Cantonese node (Node_Cantonese_LoRA) and a Minnan node (Node_Minnan_LoRA), and create a parent node (Node_Dialect_Support) to unify and describe this update. Dependencies are clearly recorded in the graph. Relevant version graph fragments are shown below. Figure 4 As shown, M represents the Cantonese node, N represents the Hokkien node, and B represents the parent node of both.
[0066] (4) Secure release First, A / B testing was conducted in the beta version of customer service using historical recordings. Then, a phased rollout was implemented: initially, dialect recognition was enabled only for intelligent customer service calls from specific regions; then it was expanded to all channels, but manual verification logs were added for requests identified as dialects; finally, the full feature was launched. Dialect recognition accuracy and average number of dialogue rounds were monitored throughout the process, and any abnormalities were rolled back.
[0067] Example 3: Seasonal Adaptation of Energy Consumption Prediction Model Scenario: Summer air conditioning load increases the error of existing energy consumption prediction models.
[0068] Implementation process: (1) Trigger update The data monitoring module detected that the prediction error continued to exceed the standard and automatically triggered the "summer mode" update.
[0069] (2) Continuous learning A seasonal adapter is inserted at the end of the LSTM (Long Short-Term Memory) prediction model. Knowledge distillation constraints ensure that the new adapter learns the high-load patterns of summer without compromising its predictive ability for other seasons. The training data is a mixture of recent summer data and typical data from other seasons in the memory bank.
[0070] (3) Version registration Create a node named Node_Summer_Energy_Adapter with the description "Adapt to Summer Air Conditioning Energy Load Forecasting Mode". Related version graph fragments are shown below. Figure 5 As shown, N represents the newly added node, and A represents its parent node LSTM_energy_v5.
[0071] (4) Secure release Since the prediction results do not directly trigger real-time control, a gray-scale strategy of "parallel prediction and result comparison" is adopted. The prediction results of the new model and the old model are output simultaneously, for reference by operations and maintenance personnel only, and are used for non-critical early warnings. After being validated throughout the summer, it will then be officially used to optimize control strategies.
[0072] The above are merely specific embodiments of the present invention and should not be construed as limiting the scope of the present invention. Equivalent variations made by those skilled in the art based on this invention, as well as changes well-known to those skilled in the art, should still fall within the scope of the present invention.
Claims
1. A method for continuous learning and management of multimodal AI models, characterized in that, Includes the following steps: S1, in response to the model update trigger event, obtain the incremental data corresponding to the trigger event; S2, based on the incremental data, the continuous learning engine module is used to incrementally train the multimodal AI model currently deployed in the production environment; the incremental training includes: inserting a lightweight adaptation parameter module corresponding to the triggering event into the backbone network of the multimodal AI model, freezing the original parameters of the backbone network, and training only the lightweight adaptation parameter module to generate a new functional unit associated with the triggering event; the lightweight adaptation parameter module is a pluggable adapter or a low-rank adaptive module; S3, register the newly added functional unit as a capability node in the version graph; the version graph is a directed acyclic graph structure, wherein nodes represent capability units of the model, and directed edges represent the dependencies between capability units; S4, deploy the new version model containing the newly added functional units online through the secure canary release pipeline; the secure canary release pipeline is configured to: verify the functionality and performance of the new version model in an isolated environment, and gradually expand its service scope in the production environment according to a preset strategy, while performing automatic rollback when functional abnormalities or performance degradation are detected.
2. The method for continuous learning and management of multimodal AI models according to claim 1, characterized in that, The incremental training also includes: When training the lightweight adaptation parameter module, a knowledge distillation loss term is introduced, using the model currently deployed in the production environment as the teacher model to constrain the output distribution of the new version model; The knowledge distillation loss term is the KL divergence loss.
3. The method for continuous learning and management of multimodal AI models according to claim 1, characterized in that, The incremental training also includes: Samples are extracted from the historical key sample memory and mixed with the incremental data for training to enhance the model's generalization ability to historical tasks. The historical key sample memory bank stores representative difficult samples from each historical task whose model prediction uncertainty is higher than a preset threshold or whose classification confidence is lower than a preset threshold.
4. The method for continuous learning and management of multimodal AI models according to claim 1, characterized in that, The capability node includes the following attribute information: a unique node identifier, a capability semantic description, a functional or performance gain indicator, and a list of parent nodes it depends on. The model version running in the production environment corresponds to a snapshot of the parameter set defined by one or more capability nodes in the version map.
5. The method for continuous learning and management of multimodal AI models according to claim 1, characterized in that, The secure grayscale deployment pipeline specifically includes the following sub-steps: S41, Deploy the new version model in a sandbox environment, process the shadow traffic replicated in the production environment, and compare and test it with the production environment; S42, if the comparison test results meet the preset standards, then the production traffic is gradually switched to the new version model according to the preset gray-scale strategy; the gray-scale strategy includes at least one of the following: expanding the effective geographical area, user group ratio, or business scenario type in stages; S43. During the canary release process, the online functional and performance metrics of the new version model are monitored in real time. If any monitored metric deteriorates beyond a preset threshold, a rollback operation is automatically triggered to switch the online service back to the previous stable model version.
6. The method for continuous learning and management of multimodal AI models according to claim 5, characterized in that, The rollback operation is based on the version graph and rolls the model back to the previous stable combination of capability nodes that does not contain the newly added capability nodes, or switches to a model version composed of a historical combination of capability nodes specified by the user.
7. A multimodal AI model continuous learning and management system, used to implement the method as described in any one of claims 1 to 6, characterized in that, include: The continuous learning engine module is configured to respond to model update trigger events and perform incremental training on multimodal AI models based on a dynamic parameter isolation mechanism. The version graph management module is configured to register, store, and manage capability nodes and their dependencies in the form of a directed acyclic graph. The secure canary release and control module is configured to perform shadow traffic comparison tests of the new version model, multi-stage gradual canary releases, and monitoring-based automatic rollback operations.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the multimodal AI model continuous learning and management method as described in any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multimodal AI model continuous learning and management method as described in any one of claims 1 to 6.