Hybrid-expert-based routable distributed fingerprint embedding and migration method and device

Abstract

The application discloses a routable distributed fingerprint embedding and migration method and device based on a hybrid expert, and belongs to the technical field of large language model intellectual property protection, and comprises the following steps: constructing a trigger sample; reforming a basic model based on a hybrid expert system architecture, dividing the basic model into multiple expert sub-modules and a gate module, specifying at least one expert sub-module as a fingerprint expert module, and using the gate module to route the trigger sample to the corresponding fingerprint expert module; fine-tuning the fingerprint expert module based on a fingerprint dataset, storing model parameters embedding fingerprint characteristics in an independent low-rank adapter; merging the fine-tuned fingerprint expert module and the low-rank adapter with the expert sub-modules of the corresponding level in a downstream model to realize fingerprint migration; and inputting the trigger sample into the downstream model after fingerprint migration to perform copyright verification and evaluation. The application considers model performance, safety and verification, and improves the adaptability of fingerprint technology in the deployment scene of industrial large models.

Description

Technical Field

[0001] This invention belongs to the field of intellectual property protection technology for large language models, specifically relating to a routable distributed fingerprint embedding and migration method and apparatus based on hybrid experts. Background Technology

[0002] With the rapid development of large language model technology and its deep application in the field of natural language processing, intellectual property disputes arising from model architecture theft, training data leakage, algorithm plagiarism, and disputes over the ownership of generated content have gradually become a focus of the industry. Especially against the backdrop of increasingly blurred boundaries between open-source model sharing and commercial applications, such legal risks exhibit new characteristics that cross technology and cross domains. Unscrupulous individuals may obtain model parameters or architecture information for unauthorized copying and use; during training and inference, models may leak sensitive information from training data, causing indirect losses of intellectual property or privacy data; even worse, some may use models or their generated content directly for commercial purposes without permission, infringing on original rights; furthermore, they may use techniques such as model distillation to generate functionally similar clone models to circumvent patent protection.

[0003] To address these challenges, academia and industry have proposed various model protection schemes, such as watermarking and fingerprint embedding. Among these, backdoor fingerprinting, as an emerging protection method, embeds specific trigger patterns (i.e., "backdoors") during model training, enabling the model to exhibit unique and identifiable behavior when receiving preset trigger inputs, while maintaining its original performance under normal inputs. Unlike traditional watermarks, backdoor fingerprints are more difficult to remove and can provide ownership verification without affecting the model's main functionality. For example, by designing special input sequences as triggers, the model generates an output containing a unique identifier only when the model owner inputs these sequences, thus proving model ownership. The advantage of this method lies in its stealth and persistence; even if the model undergoes further fine-tuning or parameter modifications, the backdoor fingerprint remains effective.

[0004] However, existing fingerprint embedding technologies still face many challenges in practical applications, affecting their effectiveness and widespread adoption. These challenges mainly manifest in the following aspects:

[0005] (1) Limitations of model feature-based methods: Copyright protection methods based on model features rely on access to the internal structure of the model, which is often limited in practical applications, especially when the model is provided as a black box service, making it difficult to obtain information about the internal structure of the model, thus limiting the effectiveness of the method.

[0006] (2) Limitations of existing methods: Traditional fingerprint embedding methods often rely on a single trigger and a fixed embedding path. This rigid design is prone to losing stability and transferability when facing continuous model training, downstream task adaptation or parameter updates.

[0007] Furthermore, existing fingerprint technology has not yet incorporated decoupling and migration mechanisms, which further leads to the following problems:

[0008] (1) Repeated fingerprint embedding: With the rapid iteration of fingerprint technology, enterprises often need to adopt the latest fingerprint technology to ensure optimal protection of downstream models. However, since the base model does not embed these advanced fingerprint features, enterprises have to repeatedly perform fingerprint embedding operations in each derived downstream model, which not only increases the computational cost, but is also difficult to afford in resource-constrained scenarios.

[0009] (2) Inefficient fingerprint embedding: Each model requiring protection requires significant computational resources for complete fingerprint training. Some algorithms also require in-depth analysis of the model architecture and characteristics to construct customized fingerprint datasets, making standardization and batch processing difficult. This efficiency bottleneck is particularly pronounced in large-scale deployment scenarios, extending time to market and increasing infrastructure costs.

[0010] (3) Fingerprint contamination: Some fingerprint technologies may alter the parameter distribution or computational logic of the model during the embedding process, leading to degradation of basic performance. This performance loss is not only retained in the basic model, but will also be gradually amplified during the continuous fine-tuning process, ultimately resulting in a significant decline in the performance of downstream applications.

[0011] (4) Fingerprint homogenization problem: Multiple downstream models inherit the fingerprints of the base model, which may make it difficult to distinguish fingerprints in some cases, reducing the uniqueness and effectiveness of fingerprints and affecting the effective protection of intellectual property rights.

[0012] (5) Inadequate transfer mechanism: Existing fingerprint transfer and decoupling techniques only consider obtaining fingerprint vectors by training all parameters on the base model, and then directly adding the vectors to the downstream model parameters to achieve efficient fingerprint embedding. However, this method has high training and storage overhead, complex transfer process, and is difficult to reuse flexibly in multi-scenario deployment.

[0013] To address these challenges, there is an urgent need for a new, structured, controllable, and transferable fingerprint embedding and verification mechanism. Summary of the Invention

[0014] In view of the above, the purpose of this invention is to provide a routable distributed fingerprint embedding and migration method and apparatus based on hybrid experts. By using a hybrid expert system architecture, the method achieves structured and decoupled embedding and cross-model migration of model fingerprints, so as to balance model performance, security and verifiability, and improve the adaptability of fingerprint technology in industrial large-scale model deployment scenarios.

[0015] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0016] In a first aspect, embodiments of the present invention provide a routable distributed fingerprint embedding and migration method based on hybrid experts, comprising the following steps:

[0017] A fingerprint dataset containing trigger samples is constructed from the basic model using a fingerprint generation algorithm. The trigger samples include text triggers, behavior pattern triggers, and corresponding expected output labels.

[0018] Based on the hybrid expert system architecture, the basic model is modified and divided into multiple expert sub-modules and at least one gating module. At least one expert sub-module is designated as the fingerprint expert module, and the gating module is used to route the trigger sample to the corresponding expert sub-module.

[0019] Based on the fingerprint dataset, the fingerprint expert module is fine-tuned using a low-rank adaptation method to achieve distributed fingerprint embedding, and the model parameters of the embedded fingerprint features are stored in an independent low-rank adapter.

[0020] The fine-tuned fingerprint expert module and its low-rank adapter are merged with the corresponding level expert sub-module in the downstream large language model to achieve distributed fingerprint migration and obtain the downstream large language model after fingerprint migration.

[0021] The downstream large language model after the sample input fingerprint is transferred will determine the copyright ownership based on the consistency of the model response, and the classification fingerprint success rate will be used as the evaluation index.

[0022] Preferably, the modification of the basic model based on the hybrid expert system architecture divides the basic model into multiple expert sub-modules and at least one gating module, including:

[0023] If the base model already has a hybrid expert layer, add at least one fingerprint expert module directly to its expert set and configure the corresponding gating module. If the base model does not have a hybrid expert layer, insert a hybrid expert structure containing multiple expert sub-modules and at least one gating module into the specified network layer to ensure that the fingerprint embedding path is decoupled from the original task path of the base model.

[0024] Preferably, the fingerprint expert module is fine-tuned based on the fingerprint dataset using a low-rank adaptation method to achieve distributed fingerprint embedding, and the model parameters embedding fingerprint features are stored in an independent low-rank adapter, including:

[0025] The fingerprint dataset is used as input to the base model. A gating module is used to make routing decisions to dynamically allocate trigger samples to designated fingerprint expert modules. During fine-tuning, only the weight matrix of the fingerprint expert module is updated in low rank to learn fingerprint features, while keeping the parameters of other expert sub-modules frozen. This enables distributed fingerprint embedding in a hybrid expert system architecture. Finally, the learned fingerprint features and the gating parameters used to control the mapping relationship between trigger samples and fingerprint expert modules are stored in an independent low-rank adapter.

[0026] Preferably, a multi-task loss function is used during the fine-tuning of the fingerprint expert module. As the training objective, it is represented as:

[0027] ,

[0028] Among them, mission losses Fingerprint constraint loss is used to maintain the accuracy and effectiveness of the base model on downstream tasks. This is used to constrain the gated route distribution to exhibit a unimodal shape and stably point to a preset fingerprint expert identifier, thereby forming a consistent binding relationship between trigger samples and fingerprint expert modules. Regularization loss is used. Used to prevent overfitting and maintain balance among expert submodules. and This represents the adjustment coefficient.

[0029] Preferably, the step of merging the fine-tuned fingerprint expert module and its low-rank adapter with the corresponding level of the expert sub-module in the downstream large language model to achieve distributed fingerprint migration and obtain the fingerprint-migrated downstream large language model includes:

[0030] If the downstream large language model already has a hybrid expert architecture, the fine-tuned fingerprint expert module and its low-rank adapter are merged into the expert set of its corresponding level by parameter interpolation or module splicing in the form of new experts. If the downstream large language model does not have a hybrid expert architecture, a hybrid expert structure is inserted into it at the corresponding level, and the fine-tuned fingerprint expert module and its low-rank adapter are merged by parameter interpolation or module splicing. In this way, distributed fingerprint migration is achieved under the hybrid expert system architecture, and finally, a fingerprint-migrated downstream large language model is obtained that fully retains the original task performance of the downstream large language model and has fingerprint response characteristics.

[0031] Preferably, the fingerprint expert module enters a frozen state after migration is completed, and is only activated by the gating module when it receives a trigger sample bound to it, so as to prevent non-trigger samples from mistakenly calling the fingerprint expert module.

[0032] Preferably, the gating module includes a trigger-aware routing unit, which clusters the input features of the trigger sample or calculates the fingerprint recognition vector to enhance routing accuracy and robustness, ensuring that the trigger sample is activated to the target fingerprint expert module.

[0033] Secondly, embodiments of the present invention also provide a routable distributed fingerprint embedding and migration device based on hybrid experts, which is implemented using the above-mentioned routable distributed fingerprint embedding and migration method based on hybrid experts, including: a fingerprint dataset construction module, a hybrid expert system modification module, a fingerprint embedding fine-tuning module, a fingerprint feature migration module, and a copyright verification and evaluation module.

[0034] The fingerprint dataset construction module is used to construct a fingerprint dataset containing trigger samples from the basic model using a fingerprint generation algorithm. The trigger samples include text triggers, behavior pattern triggers, and corresponding expected output labels.

[0035] The hybrid expert system transformation module is used to transform the basic model based on the hybrid expert system architecture, dividing the basic model into multiple expert sub-modules and at least one gating module. At least one expert sub-module is designated as a fingerprint expert module, and the gating module is used to route trigger samples to the corresponding expert sub-module.

[0036] The fingerprint embedding fine-tuning module is used to fine-tune the fingerprint expert module based on the fingerprint dataset using a low-rank adaptation method to achieve distributed fingerprint embedding, and stores the model parameters of the embedded fingerprint features in an independent low-rank adapter.

[0037] The fingerprint feature transfer module is used to merge the fine-tuned fingerprint expert module and its low-rank adapter with the corresponding level expert sub-module in the downstream large language model to achieve distributed fingerprint transfer and obtain the downstream large language model after fingerprint transfer.

[0038] The copyright verification and evaluation module is used to transfer the trigger sample input fingerprint to the downstream large language model, determine the copyright ownership based on the consistency of the model response, and use the classification fingerprint success rate as the evaluation index.

[0039] Thirdly, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described hybrid expert-based routable distributed fingerprint embedding and migration method when executing the computer program.

[0040] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for routable distributed fingerprint embedding and migration based on hybrid experts.

[0041] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0042] (1) Portability: This invention encapsulates fingerprint features in an independent fingerprint expert module through a hybrid expert system architecture, and, in conjunction with the dynamic routing mechanism of the gating module, enables the fingerprint expert module and its associated low-rank adapter to serve as independent copyright identifier carriers, achieving seamless migration of fingerprints between different models. This modular design does not rely on complete parameter updates of the model, significantly enhancing the cross-model applicability and deployment flexibility of fingerprints.

[0043] (2) High embedding efficiency and low cost: This invention achieves fingerprint embedding by fine-tuning the parameters of the fingerprint expert module, avoiding the computational overhead of updating all model parameters. Combined with the low-rank adaptation method, it significantly reduces the consumption of training resources, making the fingerprint embedding process more economical and efficient, and is particularly suitable for the rapid deployment needs of large-scale models.

[0044] (3) Low storage overhead: The present invention innovatively concentrates fingerprint features in a specific fingerprint expert module, and only the low-rank adapter of the module needs to be stored and maintained. Compared with the traditional method, which requires saving the fingerprint parameters of the entire model, it significantly reduces the storage resource occupation and provides convenience for parallel management of multiple models.

[0045] (4) Flexible decoupling: This invention decouples the fingerprint from the model through a low-rank adapter, effectively avoiding the performance degradation and homogenization problems in traditional fingerprinting techniques. Even when the model needs to continue training to enhance its capabilities for other downstream tasks, the integrity and validity of the fingerprint can be maintained. Through fine-tuning of the low-rank adapter, the fingerprint can be decoupled and trained without interfering with the original performance of the model, and then merged with the model after training. This not only does not affect the security and robustness of the fingerprint, but also ensures the continued validity of the fingerprint in complex scenarios such as incremental training and model fusion.

[0046] (5) Simplified fingerprint update: Under the framework of this invention, fingerprint updates only require fine-tuning of the specified fingerprint expert module, without involving the main model parameters. This modular update mechanism greatly simplifies the operation process, supports rapid iteration and dynamic maintenance of fingerprints, and ensures the continuity and stability of model services. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the routable distributed fingerprint embedding and migration method based on hybrid experts provided in an embodiment of the present invention.

[0049] Figure 2 This is a schematic diagram of the framework of the routable distributed fingerprint embedding and migration method based on hybrid experts in large-scale scenarios provided by the embodiments of the present invention;

[0050] Figure 3 This is a schematic diagram of the structure of the routable distributed fingerprint embedding and migration device based on hybrid experts provided in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0052] The inventive concept of this invention is as follows: Addressing the problems of repetitive fingerprint embedding, low embedding efficiency, fingerprint contamination, and lack of flexible migration mechanisms in existing large language model fingerprinting technologies, this invention proposes a routable distributed fingerprint embedding and migration method and apparatus based on hybrid experts. In the fingerprint generation stage, trigger samples composed of text triggers and behavioral pattern triggers are constructed and bound one-to-one with specific fingerprint expert modules to distinguish different fingerprint identities. In the fingerprint embedding stage, a hybrid expert (MoE) system architecture is inserted into the Transformer layer of the basic model, dividing task experts and fingerprint experts. The low-rank adaptation (LoRA) method is used to fine-tune the parameters of the fingerprint expert modules, thereby forming decoupled fingerprint module snapshots. In the fingerprint migration stage, combining model architecture homogeneity and gated routing recovery strategies, multiple fingerprint expert modules are migrated to the downstream large language model as needed, achieving pluggable fingerprint deployment and multi-module coexistence. In the fingerprint verification stage, the validity and uniqueness of the migrated fingerprints are evaluated through fingerprint binding constraints and fingerprint success rate indicators. Compared to traditional methods that rely on a single low-rank adapter, this invention not only maintains the advantages of decoupling between fingerprints and the main model, strong portability, low storage overhead, and high embedding efficiency, but also further supports the coexistence of multiple fingerprint identities, cross-model tracing, and flexible updates and iterations. This significantly improves the robustness, concealment, and large-scale deployment applicability of fingerprint embedding, providing a more systematic and scalable solution for the intellectual property protection of large language models.

[0053] like Figure 1 As shown, the embodiment provides a routable distributed fingerprint embedding and migration method based on hybrid experts, including the following steps:

[0054] S1. A fingerprint dataset containing trigger samples is constructed for the basic model using a fingerprint generation algorithm. The trigger samples include text triggers, behavior pattern triggers, and corresponding expected output labels.

[0055] In this embodiment, a fingerprint generation algorithm is used to construct multiple types of trigger samples containing significant fingerprint feature information on a base model to obtain a fingerprint dataset. Unlike traditional methods that rely on a single fixed trigger pattern, in the fingerprint dataset of this invention, text trigger samples construct text triggers through rare word combinations and associate them with expected output labels, while behavioral trigger samples construct behavioral pattern triggers through specific input sequences and associate them with expected response features. Each trigger sample is based on a ternary... The form of representation, in which Trigger inputs created for rare word combinations or specific trigger conditions. For the desired fingerprint output label, Indicator should activate the first Each trigger sample is bound to a unique fingerprint expert module. This mechanism decouples different fingerprint identities. To enhance concealment and robustness, triggers are uniformly constructed using rare text combinations or specific trigger conditions to ensure they are not accidentally activated in normal use cases. Simultaneously, ordinary task samples continue to be used to train the task expert module, thus ensuring the model's original functionality remains undisturbed.

[0056] In this embodiment, the text trigger constructed by combining rare words can refer to the IF (Instructional Fingerprinting) method and the UTF (Undertrained Tokens as Fingerprints) method, which constructs a specific input by combining random characters (such as Chinese characters, Japanese words, or random word units) with the "FINGERPRINT" instruction. A behavioral pattern trigger is constructed by using a specific input sequence, that is, by using a series of specific input patterns (such as fixed-length number sequences, special symbol sequences, alternating repeated character fragments, or specific interaction action sequences) to trigger the model's fingerprint response, thereby ensuring that the triggering mechanism is not limited to text content, but can also achieve stable activation through input behavioral features.

[0057] S2, based on the hybrid expert system architecture, transforms the basic model by dividing it into multiple expert sub-modules and at least one gating module. At least one expert sub-module is designated as the fingerprint expert module, and the gating module is used to route the trigger sample to the corresponding expert sub-module.

[0058] In this embodiment, the LLaMA-2-7B model is used as the base model. Since it does not have a hybrid expert layer, a hybrid expert (MoE) system architecture is inserted into several of its Transformer layers. Some expert sub-modules are designated as task experts (i.e., task expert modules), and others are designated as fingerprint experts (i.e., fingerprint expert modules). Each fingerprint expert module is independently bound to a fingerprint identity, thus ensuring that the fingerprint embedding path and the task path are decoupled. It also includes a gating module, which uses a dynamic routing mechanism to direct trigger samples to the target fingerprint expert module. The gating module preferably includes a trigger-aware routing unit, which can learn the gating probability distribution to make the activation distribution of trigger samples converge towards the index corresponding to the designated expert.

[0059] S3, based on the fingerprint dataset, fine-tunes the fingerprint expert module using a low-rank adaptation method to achieve distributed fingerprint embedding, storing the model parameters of the embedded fingerprint features in an independent low-rank adapter.

[0060] In this embodiment, the fingerprint dataset is used as input to the base model. A gating module performs routing decisions to dynamically allocate trigger samples to designated fingerprint expert modules. During fine-tuning, only the weight matrix of the fingerprint expert modules is updated using a low-rank adaptation method (e.g., LoRA) to learn fingerprint features, significantly reducing training and storage overhead. Simultaneously, the parameters of other expert submodules are kept frozen, and the weights of the task experts and gating modules remain unchanged to avoid affecting the performance of the main task, thus achieving distributed fingerprint embedding in a hybrid expert system architecture. Finally, the learned fingerprint features and the gating parameters used to control the mapping relationship between trigger samples and fingerprint expert modules are stored in a separate low-rank adapter.

[0061] In this embodiment, when fine-tuning the fingerprint expert module, a task loss method is used. This is used to maintain the accuracy and effectiveness of the base model on downstream tasks. Meanwhile, considering that ordinary task samples are gated and routed to task experts for training, while trigger samples are forcibly assigned to corresponding fingerprint experts through a gating binding mechanism, a fingerprint constraint loss is introduced during training. This causes the output of the gating module to have a near-one-hot distribution and point to the preset fingerprint expert identifier. This establishes a stable and consistent binding relationship between trigger samples and fingerprint expert modules. Furthermore, to prevent the model from losing trigger paths during compression scenarios such as distillation and pruning, L2-based regularization loss is introduced into the gating layer. Used to prevent overfitting and maintain balance among expert submodules.

[0062] (1) Mission loss :

[0063] For main tasks such as classification or generation, cross-entropy loss is used to construct the task loss:

[0064] ,

[0065] in, Based on the model for the first A typical task sample The predicted distribution For real labels, This represents the total number of samples for ordinary tasks.

[0066] (2) Fingerprint constraint loss :

[0067] For fingerprint trigger sample input We hope the model will output a predefined response. :

[0068] ,

[0069] in, Based on the model for the first fingerprint trigger sample The predicted distribution For real labels, This represents the total number of fingerprint-triggered samples.

[0070] Specifically, each trigger sample Each is pre-bound with a target fingerprint expert identifier. During training, the gating module outputs a distribution when it receives trigger samples. To make the distribution approximate a unimodal shape and point towards the target fingerprint expert. In addition to cross-entropy loss, gated supervision is introduced. , is represented as:

[0071] ,

[0072] Among them, the first item Push the probability quality toward the target expert To strengthen the selection of designated experts, the second item Entropy regularization is used to suppress multimodal distributions in order to approximate one-hot encoding. This is used in practical optimization. , and All of these are adjustment coefficients, thereby explicitly establishing a stable routing binding relationship between the "trigger sample - fingerprint expert module" without changing the main task objective, ensuring that the trigger is stably routed to the corresponding expert sub-module during inference.

[0073] (3) Regularization loss:

[0074] To avoid fingerprint overfitting or elimination by distillation or cropping, parameter constraints are added, using L2 regularization:

[0075] ,

[0076] in, The weight increments (such as LoRA parameters) for the fingerprint fine-tuning part ensure the stability of the embedded fingerprint distribution.

[0077] The ultimate training objective is a multi-task loss function:

[0078] ,

[0079] in, and The adjustment coefficient is used to balance the weights of the three types of losses during training. By jointly optimizing these three types of losses, we can achieve stability of fingerprint activation and robustness of the overall model while ensuring task performance.

[0080] In this embodiment, the rank is set to 8, the learning rate is 5e-5, the batch size is 16, and the cosine learning rate is used for scheduling. The training is carried out for 10 rounds, and finally, a fingerprint expert module snapshot {R1,R2,…,Rn} containing fingerprint features is generated. Each snapshot corresponds to a copyright identity independently.

[0081] S4. The fine-tuned fingerprint expert module and its low-rank adapter are merged with the corresponding level expert sub-module in the downstream large language model to achieve distributed fingerprint migration and obtain the downstream large language model after fingerprint migration.

[0082] In this embodiment, the trained fingerprint expert module is migrated to the downstream large language model while maintaining the isomorphism of the model architecture and preserving the original model functionality. The WizardMath-7B-V1.0 model is selected as the downstream large language model. This model shares the same architecture and most parameters with the base model LLaMA-2-7B, but has been specifically optimized for mathematical inference tasks. During the migration process, since the WizardMath-7B-V1.0 model does not have a hybrid expert system architecture, it is necessary to first locate the corresponding slot in the corresponding level of WizardMath-7B-V1.0 based on the metadata (including fingerprint_id, layer_index, slot_index, etc.) carried by the fingerprint expert module snapshot. If the slot is empty, it is directly inserted; if there is already a slot, the migration can be completed by overwriting, expanding, or stacking. Furthermore, by restoring the gating weights or using a lightweight trigger-aware routing mechanism, it is ensured that the trigger samples are still correctly routed to the target fingerprint expert in the downstream large language model. This process does not require full parameter retraining of the downstream large language model. The transfer can be completed through parameter fusion and matrix operations, with negligible computational overhead, while maintaining the performance of the downstream large language model on the original task.

[0083] In this embodiment, the merging method can be parameter interpolation or module splicing. Parameter interpolation refers to linearly interpolating or weighting the parameters of the fingerprint expert module with the original parameters according to a preset ratio while maintaining the original parameter weights of the downstream large language model, thereby achieving smooth embedding without disrupting the original structure. Module splicing refers to merging the fingerprint expert module as an independent expert unit into the existing expert set, dynamically selecting between the original task experts and fingerprint experts through a gating network, thus maintaining the original functionality during task execution while activating the fingerprint response under trigger conditions. Both methods retain the original gating decision logic and task expert configuration of the downstream large language model, ensuring that fingerprint embedding does not affect the performance of the existing task, while also possessing reversibility and flexible transfer capabilities.

[0084] In this embodiment, to improve deployment security and robustness, the fingerprint expert module can be set to a frozen state after deployment, and will only be activated when the gating activation probability exceeds a preset threshold, thereby preventing the risk of fingerprint erasure or false triggering caused by distillation, fine-tuning or pruning.

[0085] In the embodiments, such as Figure 2 As shown, it also supports large-scale fingerprint migration, achieved through distributed parameter fusion: the fingerprint expert module parameters in the same low-rank adapter are integrated in parallel into the corresponding expert modules of multiple downstream large language models. Simultaneously, through the cross-model consistency maintenance mechanism of the gating module, the synchronization of fingerprint responses of different downstream large language models to the same trigger sample is ensured, thereby achieving batch deployment of copyright protection. Based on this, in multi-model deployment scenarios, this invention can support multi-fingerprint coexistence, multi-tenant independent verification, and distributed loading and response consistency control, thus significantly improving the model ownership protection capability.

[0086] S5 will trigger the downstream large language model after the sample input fingerprint is transferred, determine the copyright ownership based on the consistency of the model response, and use the classification fingerprint success rate as the evaluation index.

[0087] In the embodiment, from the fingerprint dataset Obtain fingerprint trigger And input the downstream large language model with fingerprints , and These are the model parameters and the response space, respectively, based on the model's response. and the expected output label in the trigger sample The model's copyright ownership is determined based on consistency, and the degree of fingerprint retention is verified simultaneously. Fingerprint success rate (FSR) is used as an evaluation metric to assess the robustness of fingerprints in different scenarios, defined as follows:

[0088] ,

[0089] in, Indicates an indicator function, Representation Model In the triggers Response results on , For the desired backdoor fingerprint output, This indicates the number of samples used for evaluation. The FSR measures the effectiveness of fingerprint integration by calculating the proportion of triggers that successfully activate the backdoor.

[0090] During the verification process, 100 trigger samples were selected for testing. The FSR threshold was set to 0.9, meaning that when more than 90% of the trigger samples could generate the expected fingerprint response, the downstream large language model was considered to contain a fingerprint, thus confirming that the ownership belonged to the basic model. The effectiveness, harmlessness, and robustness of the invention were verified through multiple sets of experiments.

[0091] In summary, the routable distributed fingerprint embedding and transfer method based on hybrid experts proposed in this invention introduces a constraint mechanism during the fine-tuning stage to encode backdoor fingerprint features into independent low-rank adapters. This decouples functional learning from ownership encoding, effectively avoiding the high overhead of updating all parameters of the model and significantly reducing the computational and storage costs of fingerprint embedding. Furthermore, this invention constructs a seamless multi-fingerprint transfer mechanism that can smoothly transfer fingerprints from the base model to downstream large language models through parameter fusion, while maintaining the integrity of task performance and inheriting copyright identifiers. This mechanism exhibits good versatility among homogeneous models, adapting to various downstream application scenarios, and demonstrates superior robustness and stability compared to traditional methods when facing adversarial environments such as incremental training, model fusion, model compression, and pruning. Particularly noteworthy is that this invention, through the stacking of low-rank adapters and the modular design of fingerprint experts, achieves parallel coexistence and flexible invocation of multiple fingerprints. This not only avoids mutual interference between fingerprints but also makes fingerprint technology pluggable, iterable, and traceable, thereby significantly enhancing its practical value in large-scale model distribution and intellectual property protection.

[0092] Based on the same inventive concept, such as Figure 3 As shown, this embodiment of the invention also provides a routable distributed fingerprint embedding and migration device 300 based on hybrid experts, including: a fingerprint dataset construction module 310, a hybrid expert system modification module 320, a fingerprint embedding fine-tuning module 330, a fingerprint feature migration module 340, and a copyright verification and evaluation module 350.

[0093] The fingerprint dataset construction module 310 is used to construct a fingerprint dataset containing trigger samples from the basic model using a fingerprint generation algorithm. The trigger samples include text triggers, behavior pattern triggers, and corresponding expected output labels.

[0094] The hybrid expert system transformation module 320 is used to transform the basic model based on the hybrid expert system architecture, dividing the basic model into multiple expert sub-modules and at least one gating module. At least one expert sub-module is designated as the fingerprint expert module, and the gating module is used to route the trigger sample to the corresponding expert sub-module.

[0095] The fingerprint embedding fine-tuning module 330 is used to fine-tune the fingerprint expert module based on the fingerprint dataset using a low-rank adaptation method to achieve distributed fingerprint embedding, and stores the model parameters of the embedded fingerprint features in an independent low-rank adapter.

[0096] The fingerprint feature transfer module 340 is used to merge the fine-tuned fingerprint expert module and its low-rank adapter with the corresponding level expert sub-module in the downstream large language model to achieve distributed fingerprint transfer and obtain the downstream large language model after fingerprint transfer.

[0097] The copyright verification and evaluation module 350 is used to transfer the trigger sample input fingerprint to the downstream large language model, determine the copyright ownership based on the consistency of the model response, and use the classification fingerprint success rate as the evaluation index.

[0098] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described hybrid expert-based routable distributed fingerprint embedding and migration method when executing the computer program.

[0099] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for routable distributed fingerprint embedding and migration based on hybrid experts.

[0100] It should be noted that the routable distributed fingerprint embedding and migration device, electronic device, and computer-readable storage medium based on hybrid experts provided in the above embodiments all belong to the same inventive concept as the routable distributed fingerprint embedding and migration method based on hybrid experts. For details of their specific implementation process, please refer to the embodiments of the routable distributed fingerprint embedding and migration method based on hybrid experts, which will not be repeated here.

[0101] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A hybrid-expert based routable distributed fingerprint embedding and migration method, characterized in that, The method comprises the following steps: a fingerprint dataset containing trigger samples is constructed for the base model using a fingerprint generation algorithm, wherein the trigger samples include text triggers, behavior pattern triggers, and corresponding expected output labels; the base model is reformed based on a hybrid expert system architecture, and the base model is divided into multiple expert sub-modules and at least one gating module, wherein at least one expert sub-module is designated as a fingerprint expert module, and the gating module is used to route trigger samples to the corresponding expert sub-module; the fingerprint expert module is fine-tuned based on the fingerprint dataset through a low-rank adaptation method to realize distributed fingerprint embedding, and model parameters embedding fingerprint features are stored in an independent low-rank adapter; the fine-tuned fingerprint expert module and its low-rank adapter are merged with the expert sub-module at the corresponding level in the downstream large language model to realize distributed fingerprint migration, and a downstream large language model after fingerprint migration is obtained; trigger samples are input into the downstream large language model after fingerprint migration, and the copyright ownership is judged according to the consistency of the model response, and the classification fingerprint success rate is used as an evaluation index.

2. The hybrid expert based routable distributed fingerprint embedding and migration method of claim 1, wherein, The base model is reformed based on a hybrid expert system architecture, and the base model is divided into multiple expert sub-modules and at least one gating module, comprising: If the base model already has a hybrid expert layer, at least one fingerprint expert module is directly added in its expert set and the corresponding gating module is configured, and if the base model does not have a hybrid expert layer, a hybrid expert structure containing multiple expert sub-modules and at least one gating module is inserted at a specified network layer to ensure that the fingerprint embedding path and the original task path of the base model are decoupled.

3. The hybrid expert based routable distributed fingerprint embedding and migration method of claim 1, wherein, The fingerprint expert module is fine-tuned based on the fingerprint dataset through a low-rank adaptation method to realize distributed fingerprint embedding, and model parameters embedding fingerprint features are stored in an independent low-rank adapter, comprising: The fingerprint dataset is used as the input of the base model, and routing decisions are made through the gating module to dynamically allocate trigger samples to the designated fingerprint expert module. In the fine-tuning process, only the weight matrix of the fingerprint expert module is updated in low-rank to learn the fingerprint features, while the parameters of other expert sub-modules are kept frozen, thereby realizing distributed fingerprint embedding under the hybrid expert system architecture. Finally, the learned fingerprint features and the gating parameters used to control the mapping relationship between trigger samples and fingerprint expert modules are stored in an independent low-rank adapter.

4. The hybrid expert based routable distributed fingerprint embedding and migration method of claim 1, wherein, In fine-tuning the fingerprint expert module, a multi-task loss function is employed As a training target, is expressed as: ， wherein the task loss a fingerprint constraint loss for keeping the accuracy and effectiveness of the base model on the downstream tasks a regularization loss for constraining the gating routing distribution to be unimodal and stable to the preset fingerprint expert identifier, thereby forming a consistent binding relationship of the trigger sample-fingerprint expert module for preventing overfitting and maintaining the balance among the expert sub-modules, and denotes an adjustment coefficient.

5. The hybrid expert based routable distributed fingerprint embedding and migration method of claim 1, wherein, The fine-tuned fingerprint expert module and its low-rank adapter are merged with the expert sub-module at the corresponding level in the downstream large language model to realize distributed fingerprint migration, and a downstream large language model after fingerprint migration is obtained, comprising: If the downstream large language model already has a hybrid expert architecture, the fine-tuned fingerprint expert module and its low-rank adapter are merged into the corresponding level expert set in the form of a new expert through parameter interpolation or module splicing. If the downstream large language model does not have a hybrid expert architecture, a hybrid expert structure is inserted into the corresponding level, and the fine-tuned fingerprint expert module and its low-rank adapter are merged through parameter interpolation or module splicing, thereby realizing distributed fingerprint migration under a hybrid expert system architecture, and finally obtaining a fingerprint-migrated downstream large language model that not only retains the original task performance of the downstream large language model but also has fingerprint response characteristics.

6. The hybrid expert based routable distributed fingerprint embedding and migration method of claim 1, wherein, The fingerprint expert module enters a frozen state after migration is completed and is activated by the gating module only when receiving the trigger sample bound thereto, thereby preventing misactivation of the fingerprint expert module by non-trigger samples.

7. The hybrid expert based routable distributed fingerprint embedding and migration method of claim 1, wherein, The gating module includes a trigger-aware routing unit that clusters the input features of the trigger sample or calculates a fingerprint identification vector to enhance routing accuracy and robustness and ensure that the trigger sample is activated to the target fingerprint expert module.

8. A hybrid-expert-based routable distributed fingerprint embedding and migration device, implemented by using the hybrid-expert-based routable distributed fingerprint embedding and migration method of any one of claims 1-7. The method comprises: a fingerprint dataset construction module, a hybrid expert system modification module, a fingerprint embedding fine-tuning module, a fingerprint feature migration module, and a copyright verification evaluation module; The fingerprint dataset construction module is configured to use a fingerprint generation algorithm to construct a fingerprint dataset containing trigger samples for a base model, wherein the trigger samples include text triggers, behavior pattern triggers, and corresponding expected output labels. The hybrid expert system modification module is configured to modify the base model based on a hybrid expert system architecture, divide the base model into multiple expert sub-modules and at least one gating module, and designate at least one expert sub-module as a fingerprint expert module, wherein the gating module is configured to route trigger samples to corresponding expert sub-modules. The fingerprint embedding fine-tuning module is configured to fine-tune the fingerprint expert module based on the fingerprint dataset through a low-rank adaptation method to realize distributed fingerprint embedding and store model parameters embedding fingerprint features in an independent low-rank adapter. The fingerprint feature migration module is configured to merge the fine-tuned fingerprint expert module and its low-rank adapter with the corresponding level expert sub-modules in the downstream large language model to realize distributed fingerprint migration and obtain a fingerprint-migrated downstream large language model. The copyright verification evaluation module is configured to input trigger samples into the fingerprint-migrated downstream large language model, determine the copyright ownership based on the consistency of the model response, and use the classification fingerprint success rate as an evaluation indicator.

9. An electronic device comprising a memory and one or more processors, the memory to store a computer program, characterized in that, The processor is configured to implement the hybrid expert-based routable distributed fingerprint embedding and migration method of any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that When the computer program is executed by a computer, the hybrid expert-based routable distributed fingerprint embedding and migration method of any one of claims 1-7 is implemented.

Images