Model parameter migration method and apparatus, computing device

Abstract

This specification provides a model parameter transfer method, apparatus, and computing device. The model parameter transfer method includes: acquiring a reference hybrid expert model and a target hybrid expert model, wherein each hybrid expert model includes multiple expert modules, and these expert modules have a mapping relationship with subsets of labeled anomaly categories in the full set of labeled anomaly categories; based on the mapping relationship, determining candidate expert modules from the reference hybrid expert model that have a related subset of labeled anomaly categories to the target expert module in the target hybrid expert model; and transferring the module parameters of the candidate expert modules to the module parameters of the target expert module to obtain the transferred target hybrid expert model. This avoids the additional computational overhead of retraining the expert modules in the target hybrid expert model, enabling the target hybrid expert model to quickly adapt to changes in the labeled anomaly category space, thus improving the model's learnability and adaptability.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of data processing technology, and in particular to a model parameter migration method, a model parameter migration device, and a computing device. Background Technology

[0002] With the widespread application of artificial intelligence technology in the field of multimedia content anomaly identification, anomaly identification of multimedia data has become a key technical link to ensure the sound operation of the digital content ecosystem.

[0003] Currently, anomaly detection for multimedia data is typically performed using hybrid expert models. These models can utilize pre-defined classification tasks with multiple labels to identify anomalies in multimedia data, thereby obtaining the identification results for multimedia data under various label anomaly categories.

[0004] However, the above-mentioned technical solutions have technical problems in practical applications. The anomaly label category space of multimedia data often drifts and changes frequently with hotspot iterations or strategy evolution, causing the anomaly identification accuracy of the hybrid expert model to decrease significantly over time. Therefore, it is necessary to retrain the hybrid expert model. However, since the number of anomaly label categories is often huge, training the model consumes a lot of computing resources and time. Therefore, there is an urgent need for a more efficient and accurate model parameter transfer method. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a model parameter migration method. One or more embodiments of this specification also relate to a model parameter migration apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a model parameter transfer method is provided, comprising: Obtain a reference hybrid expert model and a target hybrid expert model, wherein each hybrid expert model includes multiple expert modules, and the multiple expert modules have a mapping relationship with a subset of the label anomaly categories in the full set of label anomaly categories; Based on the mapping relationship, candidate expert modules that have a subset of related label anomaly categories with the target expert module in the target hybrid expert model are identified from the reference hybrid expert model. The module parameters of the candidate expert module are transferred to the module parameters of the target expert module to obtain the target hybrid expert model after the transfer is completed.

[0007] According to a second aspect of the embodiments of this specification, a model parameter transfer apparatus is provided, comprising: The acquisition module is configured to acquire a reference hybrid expert model and a target hybrid expert model, wherein any hybrid expert model includes multiple expert modules, and the multiple expert modules have a mapping relationship with a subset of the label anomaly categories in the full set of label anomaly categories; The determination module is configured to, based on mapping relationships, determine candidate expert modules from the reference hybrid expert model that have a subset of related label anomaly categories with the target expert module in the target hybrid expert model; The migration module is configured to migrate the module parameters of the candidate expert module to the module parameters of the target expert module, thereby obtaining the target hybrid expert model after migration.

[0008] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute computer programs / instructions, which, when executed by the processor, implement the steps of the above-mentioned model parameter migration method.

[0009] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the model parameter migration method described above.

[0010] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the model parameter migration method described above.

[0011] One embodiment of this specification implements a model parameter migration method, comprising: obtaining a reference hybrid expert model and a target hybrid expert model, wherein any hybrid expert model includes multiple expert modules, and the multiple expert modules have a mapping relationship with a subset of label anomaly categories in the full set of label anomaly categories; based on the mapping relationship, determining candidate expert modules from the reference hybrid expert model that have a related subset of label anomaly categories with the target expert module in the target hybrid expert model; migrating the module parameters of the candidate expert modules to the module parameters of the target expert module, thereby obtaining the migrated target hybrid expert model.

[0012] By establishing a mapping relationship between expert modules and labeled anomaly category subsets in the hybrid expert model, accurate transfer of expert module parameters is achieved. Based on the mapping relationship between multiple expert modules and labeled anomaly category subsets, candidate expert modules with relevant labeled anomaly category subsets in the reference hybrid expert model and the target hybrid expert model can be automatically identified, ensuring semantic consistency in the transfer process. By efficiently reusing the pre-trained expert module parameters in the reference hybrid expert model, the additional computational overhead of completely retraining the expert modules in the target hybrid expert model is avoided. This allows the target hybrid expert model to quickly adapt to changes in the labeled anomaly category space. When facing frequent additions, deletions, and modifications to the label space and dynamic changes in the merging and splitting of the recognition queue, the model can be updated quickly and its capabilities can be maintained with a small computational cost, improving the model's learnability and adaptability. Attached Figure Description

[0013] Figure 1 This is a flowchart of a model parameter migration method provided in one embodiment of this specification; Figure 2 This is a flowchart illustrating the processing procedure of a model parameter migration method provided in one embodiment of this specification. Figure 3 This is a schematic diagram of the structure of a model parameter transfer device provided in one embodiment of this specification; Figure 4 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0014] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0015] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0016] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0017] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in one or more embodiments of this specification are obtained through open-source datasets or public datasets that comply with their license agreements, or are obtained with full authorization from the relevant parties. Moreover, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0018] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0019] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0020] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0021] In natural language processing, a token is the smallest unit of text that is segmented, typically used to represent words, subwords, or characters.

[0022] The Jaccard similarity coefficient is an index used to measure the degree of similarity between two finite sets. Its value is defined as the ratio of the size of the intersection to the size of the union of the two sets.

[0023] Xavier uniform initialization is a widely used method for initializing neural network parameters. It sets the initial weight values ​​to a uniform distribution range based on the number of input and output neurons in the network layer, aiming to alleviate the vanishing or exploding gradient problem in the early stages of training.

[0024] With the widespread application of artificial intelligence technology in the field of multimedia content anomaly identification, anomaly identification of multimedia data has become a key technical link to ensure the sound operation of the digital content ecosystem.

[0025] Currently, anomaly detection for multimedia data is typically performed using hybrid expert models. These models can utilize pre-defined classification tasks with multiple labels to identify anomalies in multimedia data, thereby obtaining the identification results for multimedia data under various label anomaly categories.

[0026] However, the above-mentioned technical solutions have technical problems in practical applications. The anomaly label category space of multimedia data often drifts and changes frequently with hotspot iterations or strategy evolution, causing the anomaly identification accuracy of the hybrid expert model to decrease significantly over time. Therefore, it is necessary to retrain the hybrid expert model. However, since the number of anomaly label categories is often huge, training the model will consume a lot of computing resources and time.

[0027] In view of this, this specification provides a model parameter migration method, and also relates to a model parameter migration apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0028] See Figure 1 , Figure 1 A flowchart of a model parameter migration method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0029] Step 102: Obtain the reference hybrid expert model and the target hybrid expert model, wherein any hybrid expert model includes multiple expert modules, and the multiple expert modules have a mapping relationship with the subset of labeled anomaly categories in the full set of labeled anomaly categories.

[0030] The model parameter migration method provided in one or more embodiments of this specification can be applied to various platforms or systems that require real-time or dynamic anomaly identification of multimedia data. Specifically, with the rapid development of the digital content ecosystem, the amount of user-generated multimedia data (such as notes, posts, or comments containing text, images, and videos) in scenarios such as online social platforms, content sharing communities, and live video services is enormous and diverse. To maintain a healthy network environment, platforms need to continuously perform automated anomaly identification on massive amounts of multimedia content to detect potentially illegal, harmful, or inappropriate information. For example, in social media platforms, it is necessary to identify text and image posts containing inappropriate remarks, false advertising, or misleading content; in online education platforms, it is necessary to check whether there are errors or illegal information in teaching videos or materials. In these application scenarios, the target of anomaly identification, i.e., the "label anomaly category" to be detected, is not fixed but changes frequently with the evolution of social hotspots, policies and regulations, user behavior patterns, and countermeasures, including additions, mergers, splits, or removals. Therefore, it is necessary to migrate model parameters for this dynamically changing environment to achieve continuous anomaly identification.

[0031] Hybrid expert models are deep learning models based on a hybrid expert system architecture. Here, "expert" refers to a specialized sub-network, typically including an encoder and multiple expert modules. A gating mechanism (i.e., routing mechanism) enables targeted processing for different tasks. Specifically, hybrid expert models can dynamically select and combine one or more of the most relevant expert modules for processing different input data or tasks, thereby achieving stronger expressive power and task adaptability than a single network. As the core of anomaly recognition for multimedia data, hybrid expert models can encode multimedia data into feature representations through an encoder and independently predict anomaly labels through multiple expert modules, avoiding interference between anomaly categories and ensuring the accuracy of the recognition process.

[0032] Reference hybrid expert models are pre-trained hybrid expert models that serve as a source of knowledge or parameters. These models are typically well-trained and optimized on historical datasets, and their internal expert modules have learned to effectively discriminate specific subsets of labeled anomaly categories. These expert modules have a known mapping relationship with subsets of labeled anomaly categories in the full set of labeled anomaly categories. Most labeled anomaly categories in the reference hybrid expert model meet the requirements for anomaly identification, but adjustments are still needed for individual categories that require addition or deletion. Reference hybrid expert models can also serve as a data source for parameter transfer. Since the parameters of their expert modules have been trained on historical data, they provide an effective foundation for parameter transfer to new target hybrid expert models, ensuring the reliability and accuracy of parameters during the transfer process, avoiding the computational overhead of training from scratch, and enabling the model to quickly adapt to changes in the labeled anomaly category space.

[0033] A target-oriented hybrid expert model (HEAM) is a hybrid expert model to be built or updated. It requires parameter transfer to adapt to changes in the label anomaly category space. It comprises multiple target expert modules, each responsible for identifying a specific subset of label anomaly categories. When faced with dynamic changes in the label anomaly category space, the HEAM needs to rapidly update the module parameters of the target expert modules to maintain the accuracy of anomaly identification. Specifically, the HEAM can be built upon an existing hybrid expert model architecture, adapting to the new label anomaly category space by adding, deleting, or adjusting expert modules without retraining the entire model, thus improving its adaptability and maintainability.

[0034] The expert module is a component of the hybrid expert model, typically a lightweight neural network substructure that may include gating networks and expert classification heads. It is used to identify and predict specific anomaly categories in multimedia data. Specifically, the expert module can predict the anomaly probability based on a specific subset of labeled anomaly categories from the entire set of labeled anomaly categories, thus determining the anomaly identification result of the input data. The expert module is key to achieving decoupling and modular processing of anomaly identification tasks, enabling the hybrid expert model to process multiple relatively independent identification tasks in parallel and supporting independent additions, deletions, and modifications to individual expert modules.

[0035] The complete set of labeled anomaly categories is a predefined set of all anomaly categories that need to be identified and distinguished in the multimedia data anomaly identification task. It represents the sum of all anomaly types that the system needs to identify. Specifically, the complete set of labeled anomaly categories constitutes the complete target space of the multimedia data anomaly identification task, defining all possibilities that the model needs to judge. It can guide the anomaly category prediction of the expert module. The complete set of labeled anomaly categories and its subsets form an inclusion relationship. The complete set of labeled anomaly categories is divided into multiple non-overlapping subsets, each of which corresponds to an expert module.

[0036] A subset of labeled anomaly categories is a specific component delineated from the complete set of labeled anomaly categories. It can include a set of one or more anomaly categories with some correlation, used to delineate and define the set of anomaly categories that a particular expert module is responsible for. Specifically, the subset of labeled anomaly categories represents the anomaly identification and processing scope of an expert module. Each expert module is only responsible for processing anomaly categories within its corresponding subset of labeled anomaly categories. This allows the complex multi-label classification task to be decomposed into multiple more manageable subtasks, avoiding interference between different anomaly categories. The subset of labeled anomaly categories forms an inclusion relationship with the complete set of labeled anomaly categories, which is divided into multiple subsets, each corresponding to one expert module.

[0037] The mapping relationship is the correspondence between a subset of labeled anomaly categories and expert modules, indicating that a specific subset of labeled anomaly categories is assigned to a specific expert module for processing. Specifically, the mapping relationship is determined during the design of the hybrid expert model and guides the model to assign specific subsets of labeled anomaly categories to corresponding expert modules during anomaly identification. This mapping relationship ensures that each expert module only handles tasks related to the subset of labeled anomaly categories it is responsible for, avoiding task overlap and interference between different expert modules and improving the accuracy and efficiency of anomaly identification.

[0038] In practical applications, obtaining the reference hybrid expert model and the target hybrid expert model can be achieved in a variety of ways.

[0039] One option is to directly pull the model from a model library or storage system via an application programming interface, or obtain a pre-trained reference hybrid expert model from a pre-trained model library. Another option is to obtain it from a local storage system, cloud storage system, etc., for example, by reading the stored target hybrid expert model from a database or file system. Yet another option is to obtain the reference hybrid expert model and the target hybrid expert model based on historical training results through a model version management tool. Alternatively, the required hybrid expert model can be obtained from a cloud model service through an external interface of a specific model service platform.

[0040] Specifically, different acquisition methods can be combined according to the actual application scenario. For example, a reference hybrid expert model can be obtained first through a model version management tool, and then the target hybrid expert model can be obtained through an online model service to form a multi-source acquisition mechanism to ensure the integrity and reliability of model acquisition.

[0041] For example, the set of expert modules and the complete set of labeled anomaly categories in a hybrid expert model can be represented as sets respectively:

[0042] Correspondingly, the set of expert modules and the complete set of labeled anomaly categories of the target hybrid expert model can be represented as sets respectively:

[0043] The corresponding mapping relationship can then be expressed as:

[0044]

[0045] in Presentation and Expert Module A corresponding subset of label exception categories.

[0046] In this step, by acquiring the reference hybrid expert model and the target hybrid expert model, the starting point for model parameter transfer is realized, providing a foundation for subsequent parameter transfer. The pre-training of the reference hybrid expert model ensures the reliability of the expert module parameters, avoiding the computational overhead of training from scratch. The determination of the target hybrid expert model provides a clear update target for model transfer, ensuring the targeted nature of the transfer process. The mapping relationship between multiple expert modules and the subset of labeled anomaly categories provides a basis for the accurate transfer of expert module parameters, avoiding transfer errors caused by unclear mapping relationships. By acquiring the pre-trained reference hybrid expert model, existing model parameters can be efficiently reused, reducing the training cost of the target hybrid expert model and improving the model's efficiency in adapting to changes in the label space. This enables the model to quickly respond to dynamic changes in the labeled anomaly category space, ensuring the continuous accuracy and efficiency of anomaly identification.

[0047] Step 104: Based on the mapping relationship, determine the candidate expert modules from the reference hybrid expert model that have a subset of related label anomaly categories with the target expert module in the target hybrid expert model.

[0048] The target expert module is a specific expert module in the target hybrid expert model that requires parameter transfer. As part of the target hybrid expert model, it is responsible for handling anomaly identification tasks for a specific subset of labeled anomaly categories. Specifically, the target expert module is the expert module in the target hybrid expert model that needs to be updated or adjusted. Its subset of labeled anomaly categories has a certain correlation with the subset of labeled anomaly categories in the reference hybrid expert model. Therefore, it is necessary to determine the related candidate expert modules for parameter transfer through mapping relationships. The correspondence between the target expert module and the subset of labeled anomaly categories is key to determining the parameter transfer object, ensuring semantic consistency during the transfer process.

[0049] The relevant subset of label anomaly categories is a subset of label anomaly categories that has a specific association with the subset of label anomaly categories of the target expert module. This association can be completely identical, partially overlapping, or highly similar. Specifically, the relevant subset of label anomaly categories is a subset of label anomaly categories in the reference hybrid expert model that has a matching relationship with the subset of label anomaly categories of the target expert module. This matching relationship is determined based on semantic similarity, task relevance, or historical data distribution characteristics between label anomaly categories.

[0050] Candidate expert modules are expert modules in the reference hybrid expert model that share a subset of labeled anomaly categories with the target expert module. They can serve as potential sources for parameter transfer. Specifically, candidate expert modules are expert modules in the reference hybrid expert model that have a mapping relationship with the target expert module's subset of labeled anomaly categories. The parameters of these expert modules can be transferred to the target expert module to reduce the training cost of the target expert module.

[0051] In practical applications, the module for determining candidate experts based on mapping relationships can be implemented in a variety of ways.

[0052] One possible approach is to directly compare the subset of labeled anomaly categories of the target expert module with the subsets of labeled anomaly categories of each expert module in the reference hybrid expert model. This will determine whether there is an overall mapping relationship between them, i.e., the subset of labeled anomaly categories of the target expert module is completely consistent with the subset of labeled anomaly categories of the reference expert module.

[0053] Another alternative approach is to determine whether there is a merging mapping relationship between the target expert module's subset of labeled anomaly categories and the reference expert module's subset of labeled anomaly categories when the latter is not completely consistent with the former. That is, the target expert module's subset of labeled anomaly categories is a merging of the subsets of labeled anomaly categories of two or more expert modules in the reference hybrid expert model.

[0054] Another optional method is to determine whether there is a similar mapping relationship between the target expert module’s subset of label anomaly categories and the reference expert module’s subset of label anomaly categories when there is partial overlap between them. That is, the intersection ratio between the target expert module’s subset of label anomaly categories and the reference expert module’s subset of label anomaly categories exceeds a preset threshold.

[0055] Furthermore, when there are significant differences between the subset of labeled anomaly categories of the target expert module and the subsets of labeled anomaly categories of all expert modules in the reference hybrid expert model, it can be determined whether there is a deviation mapping relationship between them. That is, if the intersection ratio of the subset of labeled anomaly categories of the target expert module and the subsets of labeled anomaly categories of each expert module in the reference hybrid expert model is lower than a preset threshold, then a new expert module needs to be created as a candidate expert module.

[0056] In this step, candidate expert modules with relevant label anomaly category subsets are identified from the reference hybrid expert model based on mapping relationships, achieving precise localization of model parameter transfer. The determination of mapping relationships ensures semantic consistency between candidate and target expert modules, avoiding transfer errors caused by mismatched label anomaly category subsets. Automatic identification of candidate expert modules based on mapping relationships reduces the need for manual intervention, improving the efficiency and accuracy of model parameter transfer. The candidate expert module selection mechanism based on mapping relationships enables the model to select the most suitable expert module for parameter transfer based on the semantic relevance of label anomaly category subsets, avoiding interference between different tasks and improving the accuracy and stability of anomaly identification. Simultaneously, this step provides a clear reference object for subsequent parameter transfer, ensuring the reliability and effectiveness of the transfer process, allowing the target hybrid expert model to quickly adapt to changes in the label anomaly category space, and enhancing the model's learnability and adaptability.

[0057] Step 106: Transfer the module parameters of the candidate expert module to the module parameters of the target expert module to obtain the target hybrid expert model after transfer.

[0058] Module parameters are adjustable values ​​such as weights and biases that can be learned within an expert module in a hybrid expert model. They define the internal computational logic and feature extraction capabilities of the expert module. During the forward inference process of the expert module, they interact with the input data, undergoing a series of linear and nonlinear transformations to ultimately output the corresponding prediction results. Module parameters are a mathematical representation of the function of an expert module, and their numerical distribution determines the discriminative ability and characteristics of the module. Specifically, module parameters may include gating network parameters, projection layer parameters, and output layer parameters. These parameters are determined through the training process, enabling the expert module to effectively identify specific subsets of labeled anomaly categories.

[0059] Transferring parameters involves copying or transforming the module parameters of candidate expert modules in a reference hybrid expert model to the corresponding positions in the target expert module, thereby enabling parameter reuse and rapid adaptive startup. Specifically, transfer allows for the directed flow of model knowledge from the reference hybrid expert model to the target hybrid expert model. By reusing validated and effective parameters, it provides a high-performance starting point for the target expert module, shortening training convergence time and improving initial performance. The transfer process may include steps such as parameter alignment, weight mapping, and parameter tuning. Through parameter transfer, a complete retraining of the target expert module can be avoided, thus saving computational resources and time costs.

[0060] The completed target hybrid expert model is a target hybrid expert model that, after parameter transfer, can adapt to changes in the new label anomaly category space. Specifically, the completed target hybrid expert model includes target expert modules whose module parameters have been transferred. The module parameters of the target expert modules come from the candidate expert modules of the reference hybrid expert model, enabling the target hybrid expert model to quickly adapt to dynamic changes in the label anomaly category space.

[0061] In practical applications, the module parameters of the candidate expert module can be migrated to the module parameters of the target expert module in various ways, and the specific implementation method varies depending on the mapping relationship type determined in the preceding steps.

[0062] Specifically, for the overall mapping relationship, that is, when the label subset of the target expert module is completely consistent with the label subset of a candidate expert module, the corresponding migration operation is the most direct. Usually, an overall copy strategy is adopted, that is, all module parameters of the candidate expert module (which may include all internal parameters such as projection layer, gating network, output layer, etc.) are completely copied to the corresponding structure of the target expert module.

[0063] For the merging mapping relationship, where the label subset of the target expert module is the union of label subsets of multiple candidate expert modules, the transfer operation is more granular. Specifically, based on label-level source selection and transfer, for each label in the target subset, its historical performance in each candidate expert module can be evaluated (e.g., recall, accuracy, or sample size processed based on the label's anomalous category), and the candidate expert module with better performance can be selected as the parameter source for that label; the weight row corresponding to this label in the output layer of the candidate expert module can be copied to the corresponding position in the output layer of the target expert module; for projection layer or gated network parameters shared by multiple candidate expert modules, strategies such as averaging, selecting one, or reinitializing can be used for fusion.

[0064] For similar mapping relationships, where the target subset and a candidate subset have a high proportion of overlap (e.g., greater than a preset coverage threshold) but are not completely identical, the transfer operation can adopt a partial reuse and expansion strategy. The projection layer parameters and gating network parameters of the candidate expert module can be copied as a whole to the target expert module. These parameters usually learn more general feature transformation and filtering patterns. For the output layer parameters, the weights corresponding to the labels in the overlap part are transferred from the candidate expert module to the target module. For newly added labels in the target subset, their corresponding weights are randomly initialized.

[0065] If the mapping relationship is deviated, i.e. no suitable candidate expert module can be found, the migration operation degenerates into creation and initialization. In this case, based on the structural dimension information of a representative expert module in the reference model, all module parameters of the target expert module can be randomly initialized to meet the model architecture requirements.

[0066] In this step, the target hybrid expert model is obtained by transferring the module parameters of the candidate expert module to the module parameters of the target expert module. This achieves efficient reuse of expert module parameters, avoids the computational overhead of completely retraining the target expert module, and enables the target hybrid expert model to quickly adapt to changes in the label anomaly category space. The transfer process ensures the semantic consistency of the module parameters and avoids performance degradation caused by improper parameter transfer. Through parameter transfer, the target hybrid expert model can quickly acquire the discriminative ability of the pre-trained expert modules in the reference hybrid expert model, reducing the time and computational resources required for model updates. At the same time, the target hybrid expert model maintains the structure and performance of the original model, providing a good starting point for subsequent fine-tuning. This allows the model to achieve rapid updates and capability continuation with low computational cost when facing dynamic changes such as frequent additions, deletions, and modifications to the label space and merging and splitting of the recognition queue, thus improving the model's learnability and adaptability.

[0067] In the embodiments of this specification, the accurate transfer of expert module parameters is achieved by establishing a mapping relationship between expert modules and label anomaly category subsets in the hybrid expert model. Based on the mapping relationship between multiple expert modules and label anomaly category subsets, candidate expert modules with relevant label anomaly category subsets in the reference hybrid expert model and the target expert module of the target hybrid expert model can be automatically identified, ensuring semantic consistency in the transfer process. By efficiently reusing the pre-trained expert module parameters in the reference hybrid expert model, the additional computational overhead of completely retraining the expert modules in the target hybrid expert model is avoided. This allows the target hybrid expert model to quickly adapt to changes in the label anomaly category space. When facing frequent additions, deletions, and modifications to the label space and dynamic changes in the merging and splitting of the recognition queue, the model can be updated quickly and its capabilities can be extended with a small computational cost, improving the model's learnability and adaptability.

[0068] In an optional embodiment of this specification, before determining, based on the mapping relationship, candidate expert modules in the target hybrid expert model that have a subset of relevant labeled anomaly categories from the reference hybrid expert model, the method further includes: Based on the number of expert modules in the target hybrid expert model and the reference hybrid expert model, the reference hybrid expert model is structurally aligned. If the number of expert modules in the target hybrid expert model is less than that in the reference hybrid expert model, the number of expert modules in the reference hybrid expert model is reduced.

[0069] The number of expert modules refers to the total number of expert modules included in a hybrid expert model, and it measures the model's size and processing power. Specifically, the number of expert modules is directly related to the number of labeled anomaly category subsets that the hybrid expert model can process in parallel. Each expert module is typically responsible for a specific subset of labeled anomaly categories, thus the number of expert modules determines the breadth of the model's coverage for multi-task recognition. During model version iterations, the number of expert modules in the target hybrid expert model may differ from the reference hybrid expert model due to changes in actual needs (such as queue merging or reduction). Therefore, it is necessary to prune based on the number of expert modules and align the model structure to ensure a one-to-one correspondence between the structure and the new label space.

[0070] Pruning is the process of removing one or more redundant expert modules from a reference hybrid expert model to match its structure with that of a target hybrid expert model. Specifically, pruning is a model simplification strategy designed to lightweight the model and align its structure when the complexity of the reference hybrid expert model (i.e., the number of expert modules) exceeds the complexity required for the current target task. This is achieved by physically removing redundant expert modules and their associated parameters, ensuring that subsequent candidate expert module selection and parameter transfer can be performed on the correct and streamlined model architecture.

[0071] In practical applications, the expert modules of the reference hybrid expert model are trimmed, usually based on the analysis of the design specifications of the target hybrid expert model, in particular identifying which old subsets of labeled anomaly categories (and their corresponding expert modules) have been obsolete, merged or replaced in the new version.

[0072] Specifically, the cutting process can be achieved in a variety of ways.

[0073] One alternative approach is to directly locate and remove unnecessary expert modules in the reference hybrid expert model that have no mapping relationship with these labels, based on the subset of label anomaly categories retained in the target hybrid expert model.

[0074] Another option is to selectively retain several expert modules with better performance or more frequent use based on service priority or historical evaluation metrics (such as the call frequency or average accuracy of each expert module over a period of time), and cut off the remaining expert modules as redundant parts.

[0075] Another option is to merge or integrate the parameters of expert modules with similar functions when they exist, and then remove the original merged module, thereby reducing the number of modules while preserving the original knowledge as much as possible.

[0076] In the actual trimming process, different trimming methods can be combined according to the actual application requirements to ensure that the trimmed model meets the structural requirements while maximizing the retention of effective discrimination ability.

[0077] In the embodiments of this specification, the reference hybrid expert model is structurally aligned based on the number of expert modules in the target hybrid expert model and the reference hybrid expert model. Furthermore, when the number of expert modules in the target hybrid expert model is less than that in the reference hybrid expert model, the expert modules in the reference hybrid expert model are pruned, achieving dynamic adaptation of the model structure. This ensures that the size of the reference hybrid expert model is consistent with the target requirements before candidate expert module selection, avoiding wasted computational resources and logical confusion caused by model size mismatch. By pruning redundant expert modules, the computational complexity and memory consumption of subsequent processing are reduced, making the model parameter migration process more efficient. Simultaneously, structural alignment of the model through pruning enhances the flexibility of parameter migration, enabling it to adapt to model update scenarios of different scales, ensuring smooth migration execution, and improving the efficiency and reliability of the entire model update system.

[0078] In one optional embodiment of this specification, the mapping relationship includes an overall mapping relationship; Based on the mapping relationship, candidate expert modules with relevant labeled anomaly category subsets are identified from the reference hybrid expert model, including: If there is an overall mapping relationship between the subset of labeled anomaly categories of the target expert module and the subset of labeled anomaly categories of any reference expert module, then the reference expert module is determined as a candidate expert module.

[0079] The overall mapping relationship is a one-to-one correspondence between the subset of label anomaly categories handled by a target expert module in the target hybrid expert model and the subset of label anomaly categories handled by a reference expert module in the reference hybrid expert model, where the label composition is completely identical. Specifically, the overall mapping relationship is the most direct and explicit form of mapping relationship, requiring that the label sets processed by the two expert modules are completely identical in element composition and quantity, without any additions, deletions, or changes. The existence of this relationship means that the two expert modules are functionally equivalent, and their learned discriminative abilities can be directly reused, thus providing a clear basis for subsequent simpler and more thorough lossless parameter migration (overall copy).

[0080] In practical applications, the process of determining candidate expert modules based on the overall mapping relationship can be achieved through precise matching.

[0081] One possible approach is to compare the subset of label anomaly categories of the target expert module with the subset of label anomaly categories of each reference expert module in the reference hybrid expert model element by element. If the label set of a certain reference expert module is found to be completely equal to the label set of the target expert module, that is, the intersection of the sets is equal to the union of the sets, then the reference expert module is immediately identified as the unique candidate expert module.

[0082] Another alternative approach is to pre-build an index table (such as a hash index table) that maps a subset of labeled anomaly categories to reference expert modules. When searching for a candidate expert module for a specific target, the query is performed directly using its complete subset of labels as the key. If a match is found, the corresponding reference expert module is returned as a candidate. This index-based approach can improve the efficiency of matching in large-scale models.

[0083] In the case of an overall mapping relationship, since the candidate expert module and the target expert module have completely identical functions, the parameter migration process is the most direct and efficient. Specifically, all module parameters of the candidate expert module, including its gating network parameters, projection layer parameters, and output layer parameters, can be completely and losslessly copied and overlaid on the corresponding structure of the target expert module. No complex weight mapping, fusion, or initialization operations are required, enabling complete parameter reuse and ensuring that the target expert module immediately obtains the same discrimination performance as the reference expert module after migration, thus achieving true lossless hot start.

[0084] In the embodiments of this specification, by introducing the concept of an overall mapping relationship, and when there is an overall mapping relationship between the subset of label anomaly categories of the target expert module and the subset of label anomaly categories of any reference expert module, the reference expert module is directly identified as a candidate expert module, providing a most efficient parameter migration path. By making full use of scenarios where the label space has not changed, unnecessary calculations and judgments are avoided, achieving lossless and complete migration of expert module parameters. The historical discrimination ability of the reference hybrid expert model is fully preserved, shortening the cold start time of the target hybrid expert model. This enables it to complete capability inheritance at extremely low cost when facing update scenarios where the label system is locally stable and only structural adjustments are required, improving the efficiency and stability of model iteration.

[0085] In one optional embodiment of this specification, the mapping relationship includes a merged mapping relationship; Based on the mapping relationship, candidate expert modules with relevant labeled anomaly category subsets are identified from the reference hybrid expert model, including: If a subset of the labeled anomaly categories of the target expert module has a merge mapping relationship with a subset of the labeled anomaly categories of at least two expert modules, then at least two expert modules will be identified as candidate expert modules.

[0086] The merge mapping relationship is a relationship formed by combining the subset of labeled anomaly categories handled by a target expert module in the target hybrid expert model with the subsets of labeled anomaly categories handled by two or more reference expert modules in the reference hybrid expert model through a set union operation. Specifically, the merge mapping relationship reflects the need in real-world application scenarios to integrate multiple original recognition task queues or types into a new queue. For example, the two originally independent recognition task queues of "medical advertisements" and "health product introductions" can be merged into a broader "health product introductions" queue. In this scenario, the new target expert module needs to inherit knowledge from multiple old expert modules; therefore, it has multiple corresponding candidate expert modules, which together constitute the knowledge source of the target expert module.

[0087] In practical applications, at least two expert modules are determined as candidate expert modules based on the overall mapping relationship. The core of this approach is to determine whether the label subset of the target expert module can be obtained by the union operation of the label subsets of two or more expert modules in the reference model.

[0088] Specifically, one possible approach is to calculate the intersection of the subset of labeled anomaly categories of the target expert module and the subset of labeled anomaly categories of each reference expert module in the reference hybrid expert model, and then filter out all reference expert modules whose intersection is not empty; further verify whether the union of the labeled subsets of these filtered reference expert modules is exactly the labeled subset of the target expert module, and if so, then determine all of these reference expert modules as candidate expert modules.

[0089] Another alternative approach is to utilize the inverse index from pre-built labels to expert modules, traverse each label in the target expert module label subset, collect all reference expert modules containing that label, and finally deduplicate the collected expert module set and verify whether its label union is consistent with the target, thereby determining the final candidate expert module set.

[0090] In the embodiments of this specification, by based on the merging mapping relationship, when there is a merging mapping relationship between the subset of label anomaly categories of the target expert module and the subset of label anomaly categories of at least two expert modules, the at least two expert modules are identified as candidate expert modules. This solves the problem of merging the recognition task queue caused by adjustments in actual application requirements, enabling the model to intelligently identify multiple historical knowledge sources and providing a clear candidate set for subsequent refinement and label-level parameter transfer. It avoids the additional overhead of training new expert modules from scratch in the queue merging scenario. By reusing the discrimination capabilities of multiple old expert modules, it ensures that the newly merged recognition queue has a high recognition accuracy in the early stage, improving the model's adaptability and robustness when facing complex task changes.

[0091] In one optional embodiment of this specification, the module parameters of the candidate expert module are transferred to the module parameters of the target expert module to obtain the target hybrid expert model after the transfer is completed, including: For the first label anomaly category of the target expert module, the first candidate expert module corresponding to the first label anomaly category is determined based on the historical evaluation indicators of at least two candidate expert modules; The output layer weights of the first label anomaly category of the first candidate expert module are transferred to the corresponding positions of the output layer of the target expert module to obtain the target hybrid expert model after the transfer is completed.

[0092] The first-label anomaly category is any specific anomaly type that needs to be identified and distinguished within the subset of labeled anomaly categories that is the responsibility of the target expert module. Specifically, the first-label anomaly category is the basic unit constituting the new merged queue identification task, and its discriminative ability can be transferred from the historical model.

[0093] Historical evaluation metrics are a set of data used to quantify the performance of each candidate expert module in a reference hybrid expert model in identifying specific label anomaly categories during historical operation. These metrics may include, but are not limited to, recall rate, precision rate at a preset precision threshold, and the number of historically processed samples corresponding to that label anomaly category. Specifically, historical evaluation metrics can provide a basis for decision-making regarding label-level parameter transfer, ensuring the selection of better-performing expert modules as parameter sources.

[0094] The first candidate expert module is selected from multiple candidate expert modules based on historical evaluation metrics to provide a parameter source for a specific first-label anomaly category. Specifically, there is a one-to-one mapping relationship between the first candidate expert module and the first-label anomaly category, and the weights corresponding to that label in its output layer will be transferred to the target expert module.

[0095] The output layer weights are the learnable parameters in the learnable parameter matrix that connects the bottleneck layer (or intermediate representation layer) and the final label prediction node in the expert module's output layer. They are typically a weight matrix. Specifically, the output layer weights map the intermediate feature vectors compressed by the projection layer to the probability score space corresponding to the subset of label anomaly categories handled by the expert module.

[0096] The corresponding position is the row position in the output layer weight matrix of the target expert module that corresponds to the index of the first label anomaly category. Specifically, this corresponding position can be predefined by the global index system of the complete set of label anomaly categories, ensuring that weights transferred from different source expert modules can be correctly placed in the target model to represent the same semantic label, thereby guaranteeing the consistency and interpretability of the global prediction results.

[0097] In practical applications, under the merged mapping relationship, for each first-label anomaly category in the target expert module, the parameter source of the target can be selected from all candidate expert modules.

[0098] One alternative approach is to compute a comprehensive evaluation function for each candidate expert module, which can be one or more weighted combinations of recall, precision, and corresponding sample size, and then select the candidate expert module with the highest score as the first candidate expert module for that label.

[0099] Another alternative approach is to use the lexicographical maximum principle, that is, to first compare recall, and if they are the same, then compare sample size, and so on, thereby establishing a clear priority among multiple indicators and selecting the most reliable source.

[0100] Once the first candidate expert module is determined, the migration operation for the module parameters can focus on the output layer weights. Specifically, the weight vector in the row corresponding to the first label anomaly category in the output layer weight matrix of the first candidate expert module can be directly copied and assigned to the corresponding position in the output layer weight matrix of the target expert module.

[0101] Furthermore, for novel labeled anomaly categories that have not appeared in any candidate expert modules, they can be filled in by random initialization at their corresponding positions in the output layer of the target expert module to ensure the integrity of the model structure.

[0102] For example, if the subset of labeled anomaly categories of the target expert module is derived from the merging of multiple candidate expert modules, then for each labeled anomaly category, the source of the module parameters can be determined by using the evaluation metrics of the candidate expert modules on the real service distribution.

[0103] Specifically, evaluation metrics may include accuracy thresholds. Tag-based recall Corresponding sample number .

[0104] Among these, a comprehensive evaluation function (scoring function) can be constructed for the evaluation indicators. And, following the principle of lexicographically largest, the highest evaluation score is selected from multiple candidate expert modules. This serves as the parameter source for the anomaly category of the label, and its corresponding output weight row is copied to the new expert.

[0105] In the embodiments of this specification, by selecting the optimal source of label based on historical evaluation indicators in the scenario of merging mapping relationships, for each label anomaly category in the target expert module, the first candidate expert module is independently selected from multiple candidate expert modules, and only its corresponding output layer weights are transferred to the corresponding position of the target expert module. This achieves refined and efficient knowledge transfer, avoids coarse-grained simple averaging or splicing of the entire expert module, and allows for personalized selection based on the historical performance of each label. This maximizes the retention of the most effective discriminative knowledge in the historical model, improves the anomaly recognition queue of the newly merged target expert module, enhances the initial recognition accuracy and stability, and reduces performance interference caused by the fusion of invalid parameters.

[0106] In one optional embodiment of this specification, the mapping relationship includes a similarity mapping relationship; Based on the mapping relationship, candidate expert modules with relevant labeled anomaly category subsets are identified from the reference hybrid expert model, including: If a subset of the labeled anomaly categories of the target expert module has a similar mapping relationship with a subset of the labeled anomaly categories of any reference expert module, then the reference expert module is identified as a candidate expert module.

[0107] A similarity mapping relationship is defined as a significant but incomplete overlap in label composition between a subset of label anomaly categories handled by a target expert module in a target hybrid expert model and a subset of label anomaly categories handled by a reference expert module in a reference hybrid expert model. This overlap exceeds a preset similarity threshold. It can be quantified by calculating the ratio of the intersection to the union of the two label subsets (i.e., Jaccard similarity) or the proportion of the intersection to the target subset (i.e., coverage). When this proportion exceeds the preset similarity threshold, a similarity mapping relationship is considered to exist. This similarity mapping relationship reflects the gradual evolution or fine-grained adjustments that occur during the iteration of the recognition task. For example, an existing recognition queue may be expanded with new relevant labels, or a broad queue may be split into more refined subsets, with one subset overlapping most of the labels of the original queue.

[0108] In practical applications, the core of the candidate expert determination module under similar mapping relationship lies in quantifying the degree of overlap between the target label subset and the reference label subset and judging whether they meet the "similar" standard.

[0109] Specifically, one possible approach is to iterate through all reference expert modules in the reference hybrid expert model, calculate the Jaccard similarity coefficient or coverage of the subset of label anomaly categories of each module and the subset of label of the target expert module, and select all reference expert modules whose calculation results are higher than a preset threshold (e.g., 0.7). If there are multiple reference expert modules that meet the conditions, the one with the highest similarity can be selected as the only candidate expert module.

[0110] Another alternative approach is to prioritize coverage as the primary metric. This involves calculating the coverage ratio of the reference expert module's label subset to the target expert module's label subset, and selecting the reference expert module with the highest coverage exceeding a threshold as a candidate. This approach focuses on ensuring that most label anomaly categories in the target expert module's label anomaly category subset can find corresponding module parameters in the candidate expert modules.

[0111] In the embodiments of this specification, by using similarity mapping relationships, when there is a similarity mapping relationship between the subset of label categories of the target expert module and the subset of label categories of any reference expert module, the reference expert module is determined as a candidate expert module. This addresses the need for refined recognition tasks or local adjustments to the label system, enabling the hybrid expert model to fully utilize the knowledge in the reference expert model that is highly relevant to the new anomaly recognition task. Even if the label anomaly categories do not match perfectly, it can achieve rapid startup through partial reuse, avoiding the extra overhead of retraining the entire expert module for tasks with only a few label changes. By inheriting most of the existing discrimination capabilities, the initial performance and stability of the target expert module are improved, and the adaptability of the model parameter transfer method is enhanced.

[0112] In one optional embodiment of this specification, the module parameters include projection layer parameters and gating network parameters; The module parameters of the candidate expert module are transferred to the module parameters of the target expert module to obtain the target hybrid expert model after the transfer, including: The projection layer parameters and gating network parameters of the candidate expert module are transferred to the target expert module; The weights corresponding to the label anomaly categories included in the target expert module in the output layer of the candidate expert module are transferred to the corresponding positions in the output layer of the target expert module to obtain the target hybrid expert model after the transfer is completed.

[0113] The projection layer parameters are a set of learnable parameters within the expert module, located in its projection layer structure. They are used to perform linear or nonlinear transformations and dimensionality reduction mappings on the input feature representation. Specifically, the projection layer parameters define the mapping rules from a high-dimensional feature space to a low-dimensional space with a preset bottleneck dimension. By selectively compressing and refining input features, they retain the most crucial information for identifying a subset of specific label anomaly categories, while reducing the number of model parameters and computational complexity, and minimizing overfitting. During the model parameter transfer process, because the projection layer parameters learn relatively general feature transformation patterns rather than discrimination boundaries specific to a particular label anomaly category, they can be transferred holistically between expert modules with similar or merged mapping relationships.

[0114] Gated network parameters are the set of learnable parameters within the expert module of a hybrid expert model. Located in the gated network structure of the expert module, they are used to evaluate the importance of different parts (such as lexical units) in the input encoded sequence to the task of identifying a subset of labeled anomaly categories that the current expert module is responsible for. Specifically, the gated network parameters define the calculation logic for importance scoring. By analyzing the input encoded vector, a gating weight representing its relevance to the current recognition task is generated for each unit in the sequence (such as a text lexical unit or a visual region), thereby achieving the filtering of key information. During the model parameter transfer process, the gated network parameters learn how to filter out the core semantic fragments most relevant to a specific subset of labeled anomaly categories from the input data. This information filtering ability usually has strong generalization and transferability when facing local adjustments to the label system, and therefore can be reused as a whole in parameter transfer.

[0115] In practical applications, when similar mapping relationships exist, a strategy of overall reuse can be adopted for module parameter migration. When a similar mapping relationship is determined between the candidate expert module and the target expert module, it indicates a high degree of overlap in their labeled anomaly category subsets. The general feature transformation patterns and information filtering patterns learned by their projection layers and gating networks have high reuse value.

[0116] Specifically, one option is to directly copy the projection layer parameters (including their weight matrices and bias vectors) of the candidate expert module and overwrite them onto the corresponding parameters of the projection layer of the target expert module. Similarly, the gating network parameters (including all learnable parameters used to calculate importance scores) of the candidate expert module can be copied and overwritten onto the corresponding parameters of the gating network of the target expert module. This avoids complex parameter decomposition and recombination, ensuring the complete inheritance of general underlying capabilities.

[0117] Another alternative approach is to apply small random perturbations or slight scaling to the parameters of the migrated projection layer or gated network, based on direct replication, to introduce some diversity and provide a starting point for subsequent fine-tuning.

[0118] Another alternative approach is to take an average of the projection layer parameters and gated network parameters of the candidate expert modules when there are multiple highly similar candidate expert modules. This involves weighting or simply averaging the parameters of the projection layer and the gated network of these candidate expert modules, and then transferring the resulting fusion parameters to the target expert module to integrate the knowledge of multiple candidate expert modules.

[0119] In the embodiments described in this specification, a layered and differentiated parameter reuse strategy is achieved by transferring the projection layer parameters and gating network parameters of the candidate expert module to the target expert module as a whole, and only performing fine-grained transfer on the weights related to the target label in the output layer. By transferring the projection layer parameters and gating network parameters as a whole, it is ensured that the target expert module can quickly inherit the capabilities of the candidate expert module in general feature extraction and task-related information filtering, providing a high-performance starting point for the target expert module. By selectively transferring the weights of the corresponding labels in the output layer, the targeted reuse of existing discriminative knowledge is achieved, reserving independent initialization space for newly added labels in the target expert module, taking into account the needs of knowledge inheritance and functional expansion, avoiding the additional computational resource consumption and time cost caused by completely retraining the target expert module, improving the efficiency of hybrid expert model iteration and the flexibility of the entire system in responding to changes in the label space.

[0120] In one optional embodiment of this specification, the mapping relationship includes a deviation mapping relationship; Based on the mapping relationship, candidate expert modules with relevant labeled anomaly category subsets are identified from the reference hybrid expert model, including: If there is a deviation mapping relationship between the label anomaly category subset of the target expert module and the label anomaly category subset of each reference expert module, a new expert module is created and the new expert module is identified as a candidate expert module.

[0121] A deviation mapping is a mapping relationship in which the subset of labeled anomaly categories handled by a target expert module in the target hybrid expert model differs significantly in label composition from the subsets of labeled anomaly categories handled by all reference expert modules in the reference hybrid expert model, with an overlap less than a preset deviation threshold. Specifically, the deviation mapping reflects a subset of labeled anomaly categories that the target expert module needs to process has almost no overlap with the knowledge domain of any existing expert module in the reference hybrid expert model. This typically occurs in scenarios where the recognition system undergoes structural adjustments, such as the introduction of entirely new service types or adversarial strategies that give rise to unprecedented anomaly patterns. This results in the new subset of labeled anomaly categories having no corresponding knowledge source in the historical model, making it impossible to effectively transfer parameters from existing candidate expert modules, thus requiring the target expert module to build a completely new knowledge foundation.

[0122] The creation of a new expert module involves initializing a standard-structured expert module from scratch, based on the architectural specifications and dimensional requirements of the target hybrid expert model, when no suitable candidates exist in the reference hybrid expert model. Specifically, the new expert module is a completely new, independently initialized neural network substructure. Its architecture is typically consistent with or compatible with the expert modules in the reference hybrid expert model, including components such as gating networks, projection layers, and output layers. However, its internal module parameters have not been trained on any task data. Since the new expert module is structurally identical to the target expert module, it can directly serve as its parameter source and be "migrated" to the target location during the subsequent migration process. This completes the parameter supply process from scratch, representing the "from zero to one" construction path in the parameter migration process.

[0123] In practical applications, when it is determined that the subset of abnormal label categories of the target expert module has a deviation mapping relationship with all reference expert modules, a new expert module entity with the same or compatible structure can be created based on the general architecture template of the expert modules in the reference hybrid expert model (e.g., the multilayer perceptron structure of the gated network, the input and output dimensions of the projection layer, the correspondence between the dimensions of the output layer and the number of labels, etc.).

[0124] Specifically, one possible approach is to create a new expert module with the same structure based on the structural hyperparameters (such as projection dimension, hidden layer size, etc.) of other expert modules in the target hybrid expert model, and randomly initialize all of its module parameters. Then, this new expert module is temporarily included in the expert pool of the reference hybrid expert model and marked as a candidate expert module of the target expert module.

[0125] Another alternative approach is to select a representative expert module (such as the one with the best average performance) from the reference hybrid expert model, copy its structure as a template to create a new expert module, but reset all its parameters to the initial state. This ensures structural compatibility while reserving optimization space for subsequent continuous learning and avoiding the introduction of irrelevant prior knowledge.

[0126] In the embodiments of this specification, by actively creating new expert modules and identifying them as candidate expert modules under the deviation mapping relationship, the processing logic of the model parameter transfer method when facing a completely new recognition task is improved. This ensures that even when the label anomaly category space changes drastically or completely unknown anomaly types appear, the target hybrid expert model can still be updated by creating new expert modules with consistent structure. This avoids process interruption or functional loss due to lack of candidates, maintains the integrity and consistency of the overall model architecture, and provides a standardized starting point for subsequent incremental training or online learning. Thus, while ensuring system robustness, the continuity and scalability of model updates are maintained.

[0127] In one optional embodiment of this specification, the module parameters of the candidate expert module are transferred to the module parameters of the target expert module to obtain the target hybrid expert model after the transfer is completed, including: Based on the dimensional information of the candidate expert modules, the module parameters of the target expert module are randomly initialized to obtain the target hybrid expert model after the transfer is completed.

[0128] The dimensionality information of a candidate expert module is a set of structural parameters for each network layer within that module. Specifically, it can include the input and bottleneck dimensions of the projection layer, the size of the hidden layers in the gating network, and the dimensions of the output layer. In essence, the dimensionality information describes the architectural specifications of the expert module, such as the input dimension of the gating network (usually consistent with the dimension of the encoding vector), the input and output bottleneck dimensions of the projection layer, and the input and output dimensions of the output layer (consistent with the size of the subset of label anomaly categories it is responsible for). In scenarios deviating from the mapping relationship, the dimensionality information of the candidate expert module can serve as a template to guide the structural construction of the target expert module. This ensures the matching of data dimensions and the executability of computation among the parts of the newly initialized expert module, enabling its integration into the overall architecture of the target hybrid expert model and maintaining consistency between forward computation and subsequent fine-tuning.

[0129] Random initialization is the process of assigning initial values ​​to learnable parameters (such as weights and biases) that conform to a specific probability distribution before further training of a neural network begins. Specifically, random initialization can provide a non-zero starting point with an appropriate range of values ​​for neural network modules to break parameter symmetry, allowing gradients to propagate effectively and optimize parameters during subsequent training, preventing all neurons from learning the same features.

[0130] In practical applications, when a newly created expert module is selected as a candidate expert module, the module parameters of the target expert module can be constructed by parsing the candidate expert module and using it as a reference.

[0131] Specifically, the dimensional information of each component (such as gating network, projection layer, output layer, etc.) of the candidate expert module can be extracted; based on the dimensional information, tensors of the corresponding dimensions are assigned to the learnable parameters of each layer in the corresponding structure of the target expert module; and the newly assigned tensors are initialized using a random initialization method, such as using Xavier uniform initialization.

[0132] Furthermore, based on random initialization, the parameters of certain key layers can be slightly adjusted. For example, the weights of the projection layer can be initialized with random perturbations close to the identity matrix to encourage feature invariance in the initial stage, thereby introducing certain prior knowledge.

[0133] In the embodiments of this specification, by randomly initializing the module parameters of the target expert module based on the dimensional information of the candidate expert module under the deviation mapping relationship, a standard, reliable and efficient parameter supply mechanism is realized when there is no effective historical knowledge to transfer. This ensures that the newly built target expert module is structurally compatible with the entire target hybrid expert model, avoiding calculation errors or training terminal issues caused by dimensional mismatch. At the same time, the initialization strategy based on dimensional information lays the foundation for the subsequent fine-tuning or continuous learning of the target expert module, enabling it to converge quickly and learn new discriminative abilities. Thus, when facing new labeled anomaly categories, the robustness and evolution capability of the model as a whole can still be guaranteed.

[0134] In an optional embodiment of this specification, after migrating the module parameters of the candidate expert module to the module parameters of the target expert module to obtain the target hybrid expert model with the migration completed, the method further includes: Fine-tune the target hybrid expert model by adjusting the module parameters of the target expert module.

[0135] Fine-tuning training is an optimization process that, after model parameter transfer is complete, uses new training data to make small-scale, targeted adjustments to the transferred parameters or parts of the model parameters for a specific task or data distribution. Specifically, fine-tuning training can further optimize the module parameters of the target expert module in a target hybrid expert model, enabling it to better adapt to new subsets of labeled anomaly categories or the current data distribution. It corrects minor biases or domain shifts that may arise from direct parameter transfer, thereby improving recognition performance for new tasks while preserving historical knowledge.

[0136] In practical applications, once the target hybrid expert model with completed transfer learning is obtained, it can be further trained. Specifically, various training strategies can be used to fine-tune the module parameters of the target expert module.

[0137] One alternative approach is to fine-tune only the output layer parameters in the target expert module, such as the low-rank adaptation (LoRA) fine-tuning strategy, freezing its projection layer and gate network parameters. This strategy is suitable for scenarios where the labeling system has only a few additions or adjustments and the underlying feature representation is still effective, and can significantly reduce computational overhead.

[0138] Another alternative approach is to perform end-to-end joint fine-tuning of all module parameters (including gating network parameters, projection layer parameters, and output layer parameters) of the target expert module, which is suitable for situations where the label subset changes significantly and the feature extraction and filtering logic needs to be re-adapted.

[0139] Another option is to use a phased fine-tuning strategy, first fine-tuning the output layer with a lower learning rate, and then gradually unfreezing and fine-tuning the parameters of deeper layers to avoid catastrophic forgetting or training instability.

[0140] For example, after completing the module parameter transfer, a subset of parameters to be continuously fine-tuned can be automatically constructed for all expert modules in the target hybrid expert model that require further fine-tuning and training optimization (including some transferred and newly initialized expert modules).

[0141] in For the set of indexes of expert modules that need optimization, and They represent the first The expert classification head parameters and gating network parameters of each expert module.

[0142] In the embodiments described in this specification, the target hybrid expert model, after transfer, is fine-tuned and trained to further adjust the module parameters of the target expert module, thus bridging the potential gap between parameter transfer and actual task requirements. Based on reusing historical knowledge and reducing initial training costs, lightweight subsequent adjustments and optimizations enable the target expert module to accurately adapt to the semantic boundaries and data distribution characteristics of the current label anomaly category subset, improving the recognition accuracy and robustness of the hybrid expert model in the dynamically changing label anomaly space. At the same time, fine-tuning training, as a lightweight subsequent optimization, has low computational cost and can maintain the overall efficiency of rapid model iteration while ensuring model performance improvement.

[0143] In one optional embodiment of this specification, fine-tuning training is performed on the target hybrid expert model, adjusting the module parameters of the target expert module, including: Based on the module parameter migration method of the target expert module, the target adjustment strategy is determined from the first adjustment strategy, the second adjustment strategy and the third adjustment strategy. The first adjustment strategy is not to fine-tune the module parameters of the target expert module, the second adjustment strategy is to fine-tune the module parameters of the target expert module using a low-rank fine-tuning method, and the third adjustment strategy is to fine-tune the module parameters of the target expert module using a global fine-tuning method. Based on the target adjustment strategy, the target hybrid expert model is fine-tuned and trained to adjust the module parameters of the target expert module.

[0144] The migration method refers to the specific parameter reuse strategy employed when migrating module parameters from a candidate expert module to a target expert module. Its type is determined by the mapping relationship between the target and candidate expert modules and can include overall mapping, merge mapping, similar mapping, or deviation mapping. Specifically, the migration method determines whether the module parameters of the target expert module are completely copied, partially reused, selectively migrated at the label level, or completely randomly initialized, thus directly affecting the necessity and intensity of subsequent fine-tuning training.

[0145] The first adjustment strategy involves directly deploying the target expert module after parameter transfer without performing any fine-tuning on its parameters. Specifically, this strategy is suitable for cases where the transfer method is a global mapping, meaning the subset of labeled anomaly categories in the target expert module is completely identical to that in the reference expert module. In this case, the transferred parameters are already highly adapted to the target task and require no additional optimization. This first adjustment strategy can save training computational resources and achieve a "warm start."

[0146] The second adjustment strategy employs the Low-Rank Alignment (LoRA) method to fine-tune the module parameters of the target expert module. Typically, only a small number of learnable low-rank fit matrices are updated, while the original backbone parameters (such as encoder and projection layer core parameters) are frozen. Specifically, the LoRA method bypasses the original weight matrix, introduces a trainable low-rank decomposition structure, and uses matrix multiplication and other computational methods to optimize and adjust the model with a minimal number of parameters. This second adjustment strategy is suitable for cases where the transfer method involves similarity mapping or merge mapping, where most module parameters are effectively reused, but there are a few new labels or distribution shifts requiring lightweight adaptation.

[0147] The third adjustment strategy is to use a global fine-tuning method to perform end-to-end comprehensive training on the module parameters of the target expert module, that is, to update the gradients for all learnable parameters. Specifically, the global fine-tuning method allows for the simultaneous optimization of the gating network parameters, projection layer parameters, and output layer parameters. It can be applied to situations where the transfer method is a deviation mapping or the initial transfer effect is poor. In this case, although the target expert module is structurally compatible, it lacks effective priors and needs to establish discriminative ability through sufficient training.

[0148] The target adjustment strategy is a fine-tuning scheme automatically selected from the first, second, and third adjustment strategies based on the module parameter migration method of the target expert module.

[0149] In practical applications, determining the target adjustment strategy and making fine-tuning based on the target adjustment strategy can be achieved by analyzing the scenario characteristics corresponding to the module parameter migration method of the target expert module.

[0150] Specifically, one possible method is to establish a rule mapping from the transfer method to the adjustment strategy. If the transfer method is a complete copy under the overall mapping relationship, the target adjustment strategy is determined to be the first adjustment strategy, i.e., no fine-tuning is performed. If the transfer method is under the similar mapping relationship, with the overall transfer of projection layer and gated network parameters plus partial transfer of output layer parameters, the target adjustment strategy is determined to be the second adjustment strategy, i.e., a low-rank fine-tuning method is used, and only the weights of the newly added labels in the output layer or the entire output layer are optimized for low-rank adaptation. If the transfer method is a completely random initialization under the deviation mapping relationship, the target adjustment strategy is determined to be the third adjustment strategy, i.e., a global fine-tuning method is used, and all module parameters are trained end-to-end.

[0151] Another alternative approach is to make dynamic decisions based on evaluation metrics. After the module parameters have been migrated, a small validation dataset can be used to quickly evaluate the initial performance of the target expert module. If its performance metrics (such as accuracy) are close to or exceed the preset performance threshold, the first or second adjustment strategy can be determined as the target adjustment strategy; if the initial performance is poor, the third adjustment strategy is selected.

[0152] Furthermore, a weighted decision based on task complexity and data volume can be adopted, taking into account the size of the subset of new label anomaly categories, data novelty, and the scale of available training data. When the task is simple and the amount of data is small, a lightweight strategy can be selected, while when the task is complex and the amount of data is sufficient, a global strategy can be selected.

[0153] Once the target adjustment strategy is determined, corresponding fine-tuning operations can be performed based on the target adjustment strategy.

[0154] In the embodiments of this specification, a target adjustment strategy is determined from the first adjustment strategy, the second adjustment strategy, and the third adjustment strategy according to the module parameter migration method of the target expert module. The target hybrid expert model is then fine-tuned and trained based on the target adjustment strategy. This achieves consistency alignment between the fine-tuning granularity and the transfer semantics, enabling intelligent selection of optimization paths for different transfer scenarios and improving the overall efficiency and robustness of the hybrid expert model update.

[0155] In one embodiment of this specification, an embodiment of the execution framework for the target hybrid expert model parameter transfer method is also provided. Specifically, see [link to relevant documentation]. Figure 2 , Figure 2 This specification illustrates a schematic diagram of the execution framework for a target hybrid expert model parameter transfer method according to an embodiment of the present specification, as shown below. Figure 2 As shown.

[0156] The acquired multimedia data may include visual data and text data; the multimedia data is input into a multimedia database; multimedia data is retrieved from the multimedia database; various input feature types included in the multimedia data are identified, including cover, body text, title, image, comment, notes, author, etc.; anomaly identification is performed based on one or more input features using an anomaly identification queue; encoding is performed through a multimodal hybrid encoder to obtain an encoded vector; a continuously learning hybrid expert module is used to determine the corresponding prediction probability for the input encoded vector through a gating network and expert classification head; the anomaly identification result of the multimedia data is determined based on the prediction probability; the entire hybrid expert module is evaluated offline, deployed online, evaluated online, and activated, and the corresponding result data is fed back to the multimedia database.

[0157] Corresponding to the above method embodiments, this specification also provides embodiments of a model parameter transfer device. Figure 3 A schematic diagram of a model parameter transfer device according to one embodiment of this specification is shown. Figure 3 As shown, the device includes: The acquisition module 302 is configured to acquire a reference hybrid expert model and a target hybrid expert model, wherein any hybrid expert model includes multiple expert modules, and the multiple expert modules have a mapping relationship with a subset of the label anomaly categories in the full set of label anomaly categories; Module 304 is configured to determine, based on mapping relationships, candidate expert modules from the reference hybrid expert model that have a subset of related label anomaly categories with the target expert module in the target hybrid expert model; The migration module 306 is configured to migrate the module parameters of the candidate expert module to the module parameters of the target expert module, thereby obtaining the target hybrid expert model after migration.

[0158] The acquisition module is a structured functional unit within the model parameter migration device used to acquire the reference hybrid expert model and the target hybrid expert model, providing the necessary model input foundation for subsequent parameter migration processes. Specifically, the acquisition module interacts with local storage systems, cloud model repositories, version management platforms, or application programming interfaces to retrieve the trained reference hybrid expert model and the target hybrid expert model to be updated, ensuring that both acquired models contain explicit mapping relationships between expert modules and labeled anomaly category subsets.

[0159] The determination module, within the model parameter transfer device, is a logical processing unit used to identify candidate expert modules based on mapping relationships. It analyzes the subset of labeled anomaly categories corresponding to the target expert module in the target hybrid expert model and automatically matches candidate expert modules with relevant subsets of labeled anomaly categories within the expert module set of the reference hybrid expert model. Specifically, the determination module compares the set relationships between subsets of labeled anomaly categories (e.g., complete consistency, union, high proportion of intersection, or no significant overlap) to determine whether there is an overall mapping relationship, a merged mapping relationship, a similar mapping relationship, or a deviating mapping relationship between them, and accordingly selects one or more candidate expert modules.

[0160] The migration module is the core execution unit in the model parameter migration device, responsible for performing actual parameter copying, transformation, or initialization operations. It can migrate the module parameters of the candidate expert modules selected by the determining module to the corresponding positions of the target expert module according to preset rules, thereby generating the migrated target hybrid expert model. Specifically, the migration module dynamically selects the migration strategy according to the type of mapping relationship. Under the overall mapping, it performs a complete parameter copy; under the merged mapping, it implements label-level weight selection and fusion; under the similar mapping, it reuses common layer parameters and expands the output layer; and under the deviation mapping, it performs random initialization based on dimensional information.

[0161] Optionally, the model parameter migration device further includes a pruning module, configured to: perform structural alignment of the reference hybrid expert model based on the number of expert modules in the target hybrid expert model and the reference hybrid expert model; and prune the expert modules of the reference hybrid expert model if the number of expert modules in the target hybrid expert model is less than that in the reference hybrid expert model.

[0162] Optionally, the mapping relationship includes an overall mapping relationship; the determining module 304 is further configured to: if the overall mapping relationship exists between the subset of labeled anomaly categories of the target expert module and the subset of labeled anomaly categories of any reference expert module, determine the reference expert module as a candidate expert module.

[0163] Optionally, the mapping relationship includes a merged mapping relationship; the determining module 304 is further configured to: if the merged mapping relationship exists between the subset of the label anomaly categories of the target expert module and the subset of the label anomaly categories of at least two expert modules, determine the at least two expert modules as candidate expert modules.

[0164] Optionally, the migration module 306 is further configured to: for the first label anomaly category of the target expert module, determine the first candidate expert module corresponding to the first label anomaly category based on the historical evaluation indicators of the at least two candidate expert modules; migrate the output layer weights of the first label anomaly category of the first candidate expert module to the corresponding positions of the output layer of the target expert module, and obtain the target hybrid expert model after migration.

[0165] Optionally, the mapping relationship includes a similarity mapping relationship; the determining module 304 is further configured to: if the similarity mapping relationship exists between the subset of label anomaly categories of the target expert module and the subset of label anomaly categories of any reference expert module, determine the reference expert module as a candidate expert module.

[0166] Optionally, the module parameters include projection layer parameters and gating network parameters; the transferable module 306 is further configured to: transfer the projection layer parameters and gating network parameters of the candidate expert module to the target expert module; transfer the weights in the output layer of the candidate expert module that correspond to the label anomaly categories included in the target expert module to the corresponding positions in the output layer of the target expert module, thereby obtaining the target hybrid expert model after transfer.

[0167] Optionally, the mapping relationship includes a deviation mapping relationship; the determining module 304 is further configured to: when the deviation mapping relationship exists between the subset of label anomaly categories of the target expert module and the subset of label anomaly categories of each reference expert module, create a new expert module and determine the new expert module as a candidate expert module.

[0168] Optionally, the transfer module 306 is further configured to: randomly initialize the module parameters of the target expert module based on the dimensional information of the candidate expert module, and obtain the target hybrid expert model after transfer.

[0169] Optionally, the model parameter transfer device further includes an adjustment module configured to: fine-tune the target hybrid expert model and adjust the module parameters of the target expert module.

[0170] Optionally, the adjustment module is further configured to: determine a target adjustment strategy from a first adjustment strategy, a second adjustment strategy, and a third adjustment strategy based on the module parameter migration method of the target expert module, wherein the first adjustment strategy is not to fine-tune the module parameters of the target expert module, the second adjustment strategy is to fine-tune the module parameters of the target expert module using a low-rank fine-tuning method, and the third adjustment strategy is to fine-tune the module parameters of the target expert module using a global fine-tuning method; and perform fine-tuning training on the target hybrid expert model based on the target adjustment strategy to adjust the module parameters of the target expert module.

[0171] The model parameter transfer device provided in the embodiments of this specification achieves full-process automation and standardization from model acquisition and candidate matching to parameter transfer through the modular division of labor and collaborative work of the acquisition module, determination module, and transfer module. The multi-source adaptation capability of the acquisition module ensures the flexibility and reliability of model input, providing a stable starting point for the process. The determination module's intelligent analysis and matching based on mapping relationships replaces tedious manual comparison, improving the accuracy and efficiency of candidate expert module positioning and ensuring semantic consistency between the transfer source and target. The transfer module performs differentiated and refined operations according to different mapping scenarios, optimizing parameter reuse, avoiding the huge overhead of complete retraining, and ensuring the integrity and high-performance starting point of the new model architecture. This enables the system to quickly and systematically update the hybrid expert model in the face of frequent and complex dynamic changes in the label anomaly category space, improving the model's efficiency in adapting to changes, reducing maintenance costs, and enhancing the robustness and sustainability of the anomaly recognition system in the face of dynamic environments.

[0172] The above is a schematic scheme of a model parameter transfer device according to this embodiment. It should be noted that the technical solution of this model parameter transfer device and the technical solution of the model parameter transfer method described above belong to the same concept. For details not described in detail in the technical solution of the model parameter transfer device, please refer to the description of the technical solution of the model parameter transfer method described above.

[0173] Figure 4 A structural block diagram of a computing device 400 according to one embodiment of this specification is shown. The components of the computing device 400 include, but are not limited to, a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and a database 450 is used to store data.

[0174] The computing device 400 also includes an access device 440, which enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0175] In one embodiment of this specification, the aforementioned components of the computing device 400 and Figure 4 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 4 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0176] The computing device 400 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 400 can also be a mobile or stationary server.

[0177] The processor 420 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described model parameter migration method.

[0178] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above-described model parameter transfer method belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-described model parameter transfer method.

[0179] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the model parameter migration method described above.

[0180] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the model parameter transfer method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the model parameter transfer method described above.

[0181] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described model parameter migration method.

[0182] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the aforementioned model parameter transfer method belong to the same concept. Details not described in detail in the computer program's technical solution can be found in the description of the technical solution of the aforementioned model parameter transfer method.

[0183] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0184] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0185] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0186] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0187] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A model parameter transfer method, characterized in that, include: Obtain a reference hybrid expert model and a target hybrid expert model, wherein each hybrid expert model includes multiple expert modules, and the multiple expert modules have a mapping relationship with a subset of the label anomaly categories in the full set of label anomaly categories; Based on the mapping relationship, candidate expert modules that have a subset of related label anomaly categories with the target expert module in the target hybrid expert model are determined from the reference hybrid expert model; The module parameters of the candidate expert module are transferred to the module parameters of the target expert module to obtain the target hybrid expert model after the transfer is completed.

2. The method according to claim 1, characterized in that, Before determining, based on the mapping relationship, candidate expert modules in the target hybrid expert model that have a subset of relevant labeled anomaly categories from the reference hybrid expert model, the method further includes: Based on the number of expert modules in the target hybrid expert model and the reference hybrid expert model, the reference hybrid expert model is structurally aligned. If the number of expert modules in the target hybrid expert model is less than that in the reference hybrid expert model, the number of expert modules in the reference hybrid expert model is reduced.

3. The method according to claim 1, characterized in that, The mapping relationship includes the overall mapping relationship; Based on the mapping relationship, the process of determining candidate expert modules in the target hybrid expert model that have a subset of relevant label anomaly categories from the reference hybrid expert model includes: If the target expert module's subset of labeled anomaly categories has the overall mapping relationship with any reference expert module's subset of labeled anomaly categories, then the reference expert module is determined as a candidate expert module.

4. The method according to claim 1, characterized in that, The mapping relationship includes a merged mapping relationship; Based on the mapping relationship, the process of determining candidate expert modules in the target hybrid expert model that have a subset of relevant label anomaly categories from the reference hybrid expert model includes: If a subset of the label anomaly categories of the target expert module has the aforementioned merging mapping relationship with a subset of the label anomaly categories of at least two expert modules, then the at least two expert modules are identified as candidate expert modules.

5. The method according to claim 4, characterized in that, The step of migrating the module parameters of the candidate expert module to the module parameters of the target expert module to obtain the migrated target hybrid expert model includes: For the first label anomaly category of the target expert module, the first candidate expert module corresponding to the first label anomaly category is determined based on the historical evaluation indicators of the at least two candidate expert modules; The output layer weights of the first label anomaly category of the first candidate expert module are transferred to the corresponding positions of the output layer of the target expert module to obtain the target hybrid expert model after the transfer is completed.

6. The method according to claim 1, characterized in that, The mapping relationship includes similarity mapping relationships; Based on the mapping relationship, the process of determining candidate expert modules in the target hybrid expert model that have a subset of relevant label anomaly categories from the reference hybrid expert model includes: If a similar mapping relationship exists between the subset of labeled anomaly categories of the target expert module and the subset of labeled anomaly categories of any reference expert module, then the reference expert module is determined as a candidate expert module.

7. The method according to claim 6, characterized in that, The module parameters include projection layer parameters and gating network parameters; The step of migrating the module parameters of the candidate expert module to the module parameters of the target expert module to obtain the migrated target hybrid expert model includes: The projection layer parameters and gating network parameters of the candidate expert module are transferred to the target expert module; The weights in the output layer of the candidate expert module that correspond to the label anomaly categories included in the target expert module are transferred to the corresponding positions in the output layer of the target expert module to obtain the target hybrid expert model after the transfer is completed.

8. The method according to claim 1, characterized in that, The mapping relationship includes deviation mapping relationships; Based on the mapping relationship, the process of determining candidate expert modules in the target hybrid expert model that have a subset of relevant label anomaly categories from the reference hybrid expert model includes: If there is a deviation mapping relationship between the subset of labeled anomaly categories of the target expert module and the subset of labeled anomaly categories of each reference expert module, a new expert module is created, and the newly created expert module is determined as a candidate expert module.

9. The method according to claim 8, characterized in that, The step of migrating the module parameters of the candidate expert module to the module parameters of the target expert module to obtain the migrated target hybrid expert model includes: Based on the dimensional information of the candidate expert modules, the module parameters of the target expert module are randomly initialized to obtain the target hybrid expert model with completed transfer.

10. The method according to any one of claims 1-9, characterized in that, After transferring the module parameters of the candidate expert module to the module parameters of the target expert module to obtain the transferred target hybrid expert model, the method further includes: The target hybrid expert model is fine-tuned and trained to adjust the module parameters of the target expert module.

11. The method according to claim 10, characterized in that, The fine-tuning training of the target hybrid expert model, adjusting the module parameters of the target expert module, includes: Based on the module parameter migration method of the target expert module, a target adjustment strategy is determined from the first adjustment strategy, the second adjustment strategy, and the third adjustment strategy. The first adjustment strategy is not to fine-tune the module parameters of the target expert module. The second adjustment strategy is to fine-tune the module parameters of the target expert module using a low-rank fine-tuning method. The third adjustment strategy is to fine-tune the module parameters of the target expert module using a global fine-tuning method. Based on the target adjustment strategy, the target hybrid expert model is fine-tuned and trained to adjust the module parameters of the target expert module.

12. A model parameter transfer device, characterized in that, include: The acquisition module is configured to acquire a reference hybrid expert model and a target hybrid expert model, wherein any hybrid expert model includes multiple expert modules, and the multiple expert modules have a mapping relationship with a subset of label anomaly categories in the full set of label anomaly categories; The determination module is configured to, based on the mapping relationship, determine, from the reference hybrid expert model, candidate expert modules that have a subset of related label anomaly categories that are related to the target expert module in the target hybrid expert model; The migration module is configured to migrate the module parameters of the candidate expert module to the module parameters of the target expert module, thereby obtaining the target hybrid expert model after migration.

13. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the model parameter migration method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, It stores a computer program / instruction that, when executed by a processor, implements the steps of the model parameter transfer method according to any one of claims 1 to 11.

15. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the steps of the model parameter transfer method according to any one of claims 1 to 11.

Images