A method, system, device and medium for federated modeling of a safety-constrained multitask unified model

Abstract

The present application relates to a kind of safe constraint multi-task unified model federal modeling method, system, equipment and medium, wherein, method includes: performing feature decoupling to initial unified model parameter, calculate distribution entropy to determine reserved dimension, extract feature base direction and construct safe constraint subspace;Parameter is issued to training node, triggers node to execute multi-modal drift calibration, and hidden layer representation is constrained to preset safe manifold;Node is based on calibrated representation, in combination with task loss, safety loss and near-end constraint controlled gradient is generated and update quantity is uploaded;Using safe constraint subspace constructs orthogonal complement projection operator, performs projection filtering to uploaded update quantity and fusion, updates global unified model.Therefore, the present application integrates multi-task training, safety constraint and federal collaborative mechanism, not only improves the engineering application value of unified model complex distributed scene, but also ensures that model maintains long-term high performance and high safety collaborative goal in multiple iterations.

Description

Technical Field

[0001] This invention relates to the field of distributed collaborative technology in artificial intelligence, and in particular to a federated modeling method, system, device, and medium for a multi-task unified model with security constraints. Background Technology

[0002] Existing federated aggregation methods or secure alignment schemes for centralized models are mostly designed for single-task optimization objectives, or assume that all clients train around the same task objective. In multi-task unified model applications, different clients undertake heterogeneous tasks such as classification, question answering, and generation, leading to significant semantic differences in local loss functions. Due to the existence of shared backbone parameters, gradient update directions for different tasks may conflict, resulting in parameter interference, knowledge coverage, and task capability degradation. Therefore, traditional federated aggregation methods are still insufficient to effectively solve the problems of shared parameter conflict and task coupling in multi-task collaborative modeling.

[0003] On the other hand, unified models face challenges in secure alignment when deployed in open environments. Existing research shows that even with customized adjustments using a small amount of data during fine-tuning of large models, the original secure alignment capability may be weakened. Moreover, this degradation can also occur in benign task fine-tuning scenarios, indicating that secure alignment is not naturally robust during continuous training.

[0004] When a unified model is combined with federated learning, the aforementioned problems are further compounded. Since different clients can only observe their own local data, it is difficult to maintain consistent security constraints from a global perspective. Simultaneously, the shared backbone parameters in the unified model carry multi-task capabilities and security boundaries. If some clients excessively favor local task updates, the aggregation process may erode the security capabilities of the global model, and insecure updates may spread across shared representations, creating cross-task security risks. Existing solutions for robust aggregation or anomaly detection struggle to directly address the security degradation issues in multi-task federated training, and methods geared towards centralized models are also difficult to directly adapt to multi-client, multi-round iterative federated training environments. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method, system, device and medium for federated modeling of a unified multi-task model with security constraints. It solves the technical problems of the prior art in dealing with multi-task update conflicts and security boundary drift in continuous training in a distributed heterogeneous environment, and in achieving effective coordination between the performance and security alignment of multi-task models within the federated training framework.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0009] In a first aspect, embodiments of the present invention provide a federated modeling method for a multi-task unified model with security constraints, comprising:

[0010] Feature decoupling is performed on the target network layer parameters of the initial unified model. The target dimension is determined based on the feature weight distribution entropy obtained from the decoupling, and the feature basis directions corresponding to the target dimension are extracted to construct a safety constraint subspace.

[0011] The initial unified model's frozen base parameters and lightweight adapter are distributed to multiple training nodes to trigger each training node to generate hidden layer representations based on local multimodal data. The hidden layer representations are then drift-calibrated according to the offset evaluation results relative to the preset safety reference benchmark, and the calibrated hidden layer representations are constrained within the preset safety manifold.

[0012] The system receives the calibrated hidden layer representations from each training node, introduces a safety constraint loss, combines the node's local task loss with the proximal constraint used to limit the local parameters from deviating from the preset global reference state to generate a controlled gradient, and updates the lightweight adapter based on the controlled gradient to obtain the lightweight update amount.

[0013] An orthogonal complementary projection operator is constructed using a safety constraint subspace. Projection filtering is performed on the received lightweight update quantities to remove out-of-bounds update components that conflict with the safety constraint subspace. The filtered update components are then fused to update the global unified model.

[0014] Optionally, feature decoupling is performed on the target network layer parameters of the initial unified model. The target dimension is determined based on the feature weight distribution entropy obtained from the decoupling, and the feature basis directions corresponding to the target dimension are extracted to construct a safety constraint subspace, including:

[0015] Identify the key backbone network layers in the initial unified model that are related to security information including ethical constraints, risk rejection, and security judgment, and extract the weight matrix of the key backbone network layers as the parameters of the target network layer;

[0016] Singular value decomposition is performed on the parameters of the target network layer to decouple and obtain the left and right singular vector matrices composed of multiple principal singular value directions, as well as the feature weight sequence composed of the corresponding singular values.

[0017] Based on the feature weight sequence, calculate the security singular value distribution entropy, which represents the degree of concentration of security information in the direction of the principal singular value;

[0018] The security rank of the target network layer is obtained based on the preset information retention threshold and the security singularity distribution entropy, and the security rank is determined as the target dimension.

[0019] Extract the set of preorder left and right singular vectors corresponding to the target dimension from the left and right singular vector matrices as feature basis directions, and construct the basis matrix composed of feature basis directions;

[0020] Perform matrix multiplication on the fundamental matrix and its transpose to generate the global security subspace projection matrix, which serves as the security constraint subspace.

[0021] Optionally, based on the feature weight sequence, the security singular value distribution entropy, which characterizes the concentration of security information in the direction of the principal singular value, is calculated, including:

[0022] Traverse the preceding feature weights from the first to the target dimension in the feature weight sequence, calculate the square value of each preceding feature weight, and obtain the corresponding safety energy value.

[0023] The sum of the squared values ​​of all feature weights in the feature weight sequence is calculated to obtain the total global energy.

[0024] Calculate the ratio of each security energy value to the total global energy, and obtain the probability distribution of security information corresponding to each principal singular value direction;

[0025] Calculate the logarithm of the probability distribution of each security information, and multiply the probability distribution of each security information with its corresponding logarithm.

[0026] The results of the product operation are summed, and the sum is negative to generate the safe singular value distribution entropy.

[0027] Optionally, each training node is triggered to generate a hidden layer representation based on local multimodal data, and drift calibration is performed on the hidden layer representation according to the offset evaluation result relative to a preset safety reference benchmark, constraining the calibrated hidden layer representation within a preset safety manifold, including:

[0028] Each training node receives multimodal mixed inputs including images, videos, or text, and performs processing in the local forward propagation stage using frozen pedestal parameters to generate the original hidden layer representation located in the shared representation space.

[0029] Identify the offset trend of the image or video input relative to the safety distribution established by text alignment to obtain the cross-modal representation drift direction component; wherein, the cross-modal representation drift direction component is calculated based on the local safety anchors or reference samples maintained locally by each training node, or based on the safety direction information issued by the global server, and is used to describe the offset trend of the current multimodal input relative to the safety boundary;

[0030] Obtain the system's preset intervention intensity coefficient, and use the intervention intensity coefficient to adjust the cross-modal characterization drift direction component to generate an offset compensation amount for calibration;

[0031] The vector deviation of the original hidden layer representation is corrected based on the offset compensation amount, so as to offset the original hidden layer representation into the preset safe manifold.

[0032] The result after output vector deviation correction is used as the calibrated hidden layer representation, so that the calibrated hidden layer representation is within the preset safe manifold. The preset safe manifold is the vector distribution interval formed in the hidden layer space when the initial unified model extracts features from the preset reference data.

[0033] Optionally, the offset trend of the image or video input relative to the safe distribution established by text alignment is identified to obtain cross-modal characterization of the drift direction component, including:

[0034] A reference sample library containing safe modes or rejection templates is established locally on each training node. The reference sample library is input into the initial unified model to extract the hidden layer output in order to construct local safe anchors or reference sample representations.

[0035] Alternatively, receive security direction information issued by the global server based on global security distribution characteristics;

[0036] The original hidden layer representation is geometrically spatially compared with the local safety anchor point, reference sample representation, or safety direction reference information, respectively.

[0037] Calculate the vector difference between the original hidden layer representation and the local safety anchor point, the reference sample representation, or the safety direction reference information;

[0038] The calculated vector difference is used as the cross-modal drift direction component describing the current multimodal input relative to the preset safety reference.

[0039] Optionally, each training node introduces a safety constraint loss based on the calibrated hidden layer representation. This loss is combined with the node's local task loss and a proximal constraint used to limit local parameters from deviating from a preset global reference state to generate a controlled gradient. The lightweight adapter is then updated based on this controlled gradient to obtain the lightweight update amount, including:

[0040] Based on the calibrated hidden layer representation output task prediction results, calculate the node local task loss driven by local multi-task representation.

[0041] The security constraint loss for maintaining the basic security alignment state is determined based on the preset security alignment target; wherein, the basic security alignment state is a controlled technical state that can continuously maintain the identification, rejection and risk avoidance of explicit harmful requests during the parameter update process;

[0042] Read the local parameters corresponding to the updated lightweight adapter, and calculate the parameter offset distance between the local parameters corresponding to the lightweight adapter and the preset global reference adapter parameters corresponding to the lightweight adapter;

[0043] Calculate the squared Frobenius norm of the parameter offset distance and combine it with the preset near-end constraint strength coefficient to construct a near-end constraint term that causes the constraint parameter to drift excessively to outside the preset safe neighborhood under heterogeneous data driving.

[0044] The node's local task loss, security constraint loss, and near-end constraint term are weighted and fused to construct the local total loss function;

[0045] The gradient backpropagation algorithm is executed based on the local total loss function to generate controlled gradients;

[0046] The parameters of the lightweight adapter are updated based on the controlled gradient, and the updated lightweight adapter is output as the lightweight update amount.

[0047] Optionally, an orthogonal complementary projection operator is constructed using the safety constraint subspace to perform projection filtering on the received lightweight update amount to remove out-of-bounds update components that conflict with the safety constraint subspace. The filtered update components are then fused to update the global unified model, including:

[0048] Obtain the lightweight update volume uploaded by each training node and identify the matrix dimension of the lightweight update volume;

[0049] On the server side, construct an identity matrix with the same dimension as the matrix, and perform a projection transformation operation on the identity matrix and the global security subspace projection matrix representing the security constraint subspace to construct a filter matrix representing the orthogonal complementary projection operator of the security subspace.

[0050] The lightweight update uploaded by each training node is subjected to matrix multiplication and projection operations with the filter matrix to remove malicious offset components or abnormal perturbation components generated in the dimensional direction corresponding to the non-safety constraint subspace.

[0051] Federated weighted aggregation is performed on all filtered update components to generate a new round of global lightweight adapter results, which together with the frozen base parameters form a new round of global unified model;

[0052] The convergence state of the new round of the global unified model is comprehensively judged using the judgment criteria; the judgment criteria include at least one of the following: preset round number, task loss change magnitude, adapter update norm, global performance index or security alignment index.

[0053] When the preset convergence requirement is not met, the results of the new round of global lightweight adapter are used as the global reference state for the next round and reissued to each training node to execute a new round of federated training iteration.

[0054] When the preset convergence requirement is met, the federated aggregation process is terminated, and the final federated unified model with safe alignment constraints is output.

[0055] Secondly, embodiments of the present invention provide a multi-task unified model federated modeling system for security constraints, including a global server and multiple clients, wherein the global server includes:

[0056] The security constraint subspace construction module is used to perform feature decoupling on the target network layer parameters of the initial unified model, determine the target dimension based on the feature weight distribution entropy obtained from the decoupling, and extract the feature basis directions corresponding to the target dimension to construct the security constraint subspace.

[0057] The task distribution and drift calibration triggering module is used to distribute the frozen base parameters and lightweight adapter of the initial unified model to multiple training nodes, so as to trigger each training node to generate hidden layer representations based on local multimodal data, and perform drift calibration on the hidden layer representations according to the offset evaluation results relative to the preset safety reference benchmark, and constrain the calibrated hidden layer representations within the preset safety manifold.

[0058] The controlled gradient receiving module is used to receive the calibrated hidden layer representation of each training node, introduce a safety constraint loss, combine the node's local task loss with the proximal constraint used to limit the local parameters from deviating from the preset global reference state to generate a controlled gradient, and update the lightweight adapter according to the controlled gradient to obtain the lightweight update amount.

[0059] The orthogonal projection filtering and fusion module is used to construct orthogonal complementary projection operators using the safety constraint subspace, perform projection filtering on the received lightweight update quantities to remove out-of-bounds update components that conflict with the safety constraint subspace, and fuse the filtered update components to update the global unified model.

[0060] Thirdly, embodiments of the present invention provide a multi-task unified model federated modeling device for security constraints, comprising: at least one controller; and a memory communicatively connected to the at least one controller; wherein the memory stores instructions executable by the at least one controller, the instructions being executed by the at least one controller to enable the at least one controller to perform the multi-task unified model federated modeling method for security constraints as described above.

[0061] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a controller, implement the multi-task unified model federated modeling method for security constraints as described above.

[0062] (III) Beneficial Effects

[0063] The beneficial effects of this invention are:

[0064] First, by performing feature decoupling on the target network layer parameters of the initial unified model and constructing a safety constraint subspace, this invention effectively solves the problems of update conflicts and semantic coupling in a federated environment for multi-task unified models. This step establishes a unified task organization and safety constraint baseline at the global level, enabling local training on different clients to be carried out under the same controlled framework. This avoids parameter interference, knowledge coverage, and task capability degradation caused by conflicts in gradient directions of different tasks, thereby significantly improving the collaborative modeling capability and overall training stability of the multi-task unified model in a distributed environment.

[0065] Secondly, by triggering drift calibration of the hidden layer representation at each training node and updating parameters based on controlled gradients with introduced safety constraint loss and proximal constraints, this invention effectively suppresses the degradation of the model's safety capability during continuous federated training. The drift calibration during the forward propagation phase mitigates dangerous feature deviations caused by multimodal inputs, while the gradient update mechanism with introduced proximal constraints ensures that the deviation of local parameters from the global reference state remains within a controllable range. This design moves safety control forward to the representation and gradient layers, significantly reducing the possibility of cross-task risk propagation and ensuring the robustness of the model's safety boundary during long-term federated training.

[0066] Furthermore, by receiving lightweight updates from each node, this invention balances training efficiency and resource consumption while ensuring privacy. By freezing the base parameters and updating only the lightweight adapter, the computational complexity, memory overhead, and communication burden on the server are effectively reduced for each training node. This method not only enables the unified model to have better resource adaptability under multi-task training but also maintains high security and stability under efficient parameter fine-tuning, enhancing the model's engineering application value in complex distributed scenarios.

[0067] Finally, an orthogonal complementary projection operator is constructed using a safety-constrained subspace to perform projection filtering and fusion on the received lightweight updates. This invention improves the controllability and robustness of federated aggregation. This mechanism overcomes the limitation of traditional federated aggregation relying solely on numerical weighting. On the server side, safety-direction filtering is performed on the updates uploaded by each training node, eliminating out-of-bounds update components along safety-sensitive directions, ensuring that all updates participating in the aggregation are valid and consistent with the global safety boundary. This effectively suppresses the cumulative erosion of global model safety by local anomalous updates, achieving continuous manifestation of safety alignment requirements across multiple rounds of federated iterations.

[0068] Therefore, this invention integrates multi-task training, security constraints, and federated collaboration mechanisms, which not only enhances the engineering application value of the unified model in complex distributed scenarios such as healthcare, finance, and industrial internet, but also ensures that the model maintains high performance and high security in multiple iterations over a long period of time. Attached Figure Description

[0069] Figure 1 This is a schematic diagram illustrating the core steps of the method provided in the embodiments of the present invention;

[0070] Figure 2 This is a schematic diagram illustrating the specific process of step S1 of the method provided in this embodiment of the invention;

[0071] Figure 3 This is a detailed flowchart illustrating step S13 of the method provided in this embodiment of the invention;

[0072] Figure 4 This is a detailed flowchart illustrating step S2 of the method provided in this embodiment of the invention;

[0073] Figure 5 This is a detailed flowchart illustrating step S22 of the method provided in this embodiment of the invention;

[0074] Figure 6 This is a detailed flowchart illustrating step S3 of the method provided in this embodiment of the invention;

[0075] Figure 7 This is a detailed flowchart illustrating step S4 of the method provided in this embodiment of the invention;

[0076] Figure 8 A schematic flowchart illustrating the overall process of the method provided in this embodiment of the invention. Detailed Implementation

[0077] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0078] like Figure 1As shown in the embodiment of the present invention, a federated modeling method for a multi-task unified model with security constraints is proposed. This method includes: performing feature decoupling on the target network layer parameters of the initial unified model; determining the target dimension based on the feature weight distribution entropy obtained from the decoupling; and extracting the feature basis directions corresponding to the target dimension to construct a security constraint subspace; distributing frozen base parameters and lightweight adapters of the initial unified model to multiple training nodes to trigger each training node to generate hidden layer representations based on local multimodal data; performing drift calibration on the hidden layer representations based on the offset evaluation results relative to a preset security reference benchmark; and constraining the calibrated hidden layer representations within a preset security manifold; receiving the calibrated hidden layer representations from each training node, introducing a security constraint loss, combining the node's local task loss with proximal constraints used to limit the deviation of local parameters from a preset global reference state to generate a controlled gradient, and updating the lightweight adapter based on the controlled gradient to obtain a lightweight update amount; constructing an orthogonal complementary projection operator using the security constraint subspace; performing projection filtering on the received lightweight update amount to remove out-of-bounds update components that conflict with the security constraint subspace; and fusing the filtered update components to update the global unified model.

[0079] First, by performing feature decoupling on the target network layer parameters of the initial unified model and constructing a safety constraint subspace, this invention effectively solves the problems of update conflicts and semantic coupling in a federated environment for multi-task unified models. This step establishes a unified task organization and safety constraint baseline at the global level, enabling local training on different clients to be carried out under the same controlled framework. This avoids parameter interference, knowledge coverage, and task capability degradation caused by conflicts in gradient directions of different tasks, thereby significantly improving the collaborative modeling capability and overall training stability of the multi-task unified model in a distributed environment.

[0080] Secondly, by triggering drift calibration of the hidden layer representation at each training node and updating parameters based on controlled gradients with introduced safety constraint loss and proximal constraints, this invention effectively suppresses the degradation of the model's safety capability during continuous federated training. The drift calibration during the forward propagation phase mitigates dangerous feature deviations caused by multimodal inputs, while the gradient update mechanism with introduced proximal constraints ensures that the deviation of local parameters from the global reference state remains within a controllable range. This design moves safety control forward to the representation and gradient layers, significantly reducing the possibility of cross-task risk propagation and ensuring the robustness of the model's safety boundary during long-term federated training.

[0081] Furthermore, by receiving lightweight updates from each node, this invention balances training efficiency and resource consumption while ensuring privacy. By freezing the base parameters and updating only the lightweight adapter, the computational complexity, memory overhead, and communication burden on the server are effectively reduced for each training node. This method not only enables the unified model to have better resource adaptability under multi-task training but also maintains high security and stability under efficient parameter fine-tuning, enhancing the model's engineering application value in complex distributed scenarios.

[0082] Finally, an orthogonal complementary projection operator is constructed using a safety-constrained subspace to perform projection filtering and fusion on the received lightweight updates. This invention improves the controllability and robustness of federated aggregation. This mechanism overcomes the limitation of traditional federated aggregation relying solely on numerical weighting. On the server side, safety-direction filtering is performed on the updates uploaded by each training node, eliminating out-of-bounds update components along safety-sensitive directions, ensuring that all updates participating in the aggregation are valid and consistent with the global safety boundary. This effectively suppresses the cumulative erosion of global model safety by local anomalous updates, achieving continuous manifestation of safety alignment requirements across multiple rounds of federated iterations.

[0083] Therefore, this invention integrates multi-task training, security constraints, and federated collaboration mechanisms, which not only enhances the engineering application value of the unified model in complex distributed scenarios such as healthcare, finance, and industrial internet, but also ensures that the model maintains high performance and high security in multiple iterations over a long period of time.

[0084] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0085] Specifically, embodiments of the present invention provide a federated modeling method for a multi-task unified model with security constraints, comprising:

[0086] S1. Perform feature decoupling on the target network layer parameters of the initial unified model, determine the target dimension based on the feature weight distribution entropy obtained from the decoupling, and extract the feature basis directions corresponding to the target dimension to construct a safety constraint subspace.

[0087] Furthermore, such as Figure 2 As shown, step S1 includes:

[0088] S11. Identify the key backbone network layers in the initial unified model that are related to security information including ethical constraints, risk rejection, and security judgments, and extract the weight matrices of the key backbone network layers as parameters for the target network layers. In this embodiment, the global server first performs security feature decoupling on the unified model backbone network that has already completed basic security alignment to identify the key low-rank parameter directions related to ethical constraints, risk rejection, and security judgments. This step is performed because the security knowledge within the unified model is not uniformly distributed but concentrated in some principal singular value directions; identifying these parameters is a prerequisite for subsequent decoupling.

[0089] S12. Perform singular value decomposition (SVD) on the target network layer parameters to decouple and obtain left and right singular vector matrices composed of multiple principal singular value directions, and feature weight sequences composed of corresponding singular values. Specifically, perform SVD on the extracted target layer weight matrix to obtain the singular value distribution, where the i-th singular value is denoted as σ. i , where i is the traversal index, and its value range is 1≤i≤ρ (ρ is the target dimension). The magnitude of the singular values ​​directly reflects the strength of the feature energy in the corresponding singular direction. The principal singular value direction is the set of directions determined by the first N singular values ​​of the matrix and their corresponding left and right singular vectors. These directions usually carry the core capabilities and safety boundary information of the model.

[0090] S13. Based on the feature weight sequence, calculate the security singular value distribution entropy, which represents the degree of concentration of security information in the direction of the main singular value.

[0091] Furthermore, such as Figure 3 As shown, step S13 includes:

[0092] S131. Traverse the preceding feature weights from the first to the target dimension in the feature weight sequence, and calculate the square of each preceding feature weight to obtain the corresponding safe energy value. In this step, by calculating the square of the singular values, the original strength information of the target layer weight matrix in the feature space is transformed into a physical-level energy representation, denoted as σ. i 2 The traversal scope is limited to the target dimension, aiming to accurately select the principal singular value components that have the strongest and most active capacity for carrying security knowledge, and to eliminate noise components that contribute little to security alignment.

[0093] S132. Calculate the sum of the squared values ​​of all feature weights in the feature weight sequence to obtain the global energy sum. Calculate the global energy sum ∑σ. j 2 , where j is the summation index, and its value range covers all singular values ​​corresponding to the weight matrix of the current network layer. By introducing index j for cumulative summation, the total global energy of the entire network layer can be obtained as a normalization benchmark.

[0094] S133. Calculate the ratio of each security energy value to the total global energy, and obtain the proportion of the security information probability distribution corresponding to each principal singular value direction. This ratio is denoted as σ. i 2 / ∑σ j 2 This step transforms the non-normalized feature intensity into a probability distribution, enabling subsequent information entropy calculations to meet the requirements of probability and statistical distribution in information theory, thereby accurately characterizing the distribution proportion of security information in different singular directions.

[0095] S134. Calculate the logarithm of the probability distribution percentage of each security information, and multiply the probability distribution percentage of each security information with its corresponding logarithm. This step characterizes the contribution of each dominant direction to the total information entropy through the product operation. The magnitude of the product term directly reflects the contribution of that specific singular direction to the uncertainty in the model's security semantic expression.

[0096] S135. Summing the results of the product operations and taking the negative value of the sum generates the security singular value distribution entropy. Summing and taking the negative value is the standard form of information entropy calculation; the formula is as follows: Ultimately, the result was obtained from the previous... The safe singular value distribution entropy H consists of _ singular values. ρ Entropy is used to measure the concentration of security information in the dominant singular value direction within the current network layer. The smaller the entropy value, the more compactly the security information is distributed in the dominant direction, meaning that security-related knowledge is more highly concentrated. This directly guides the server to identify whether the network layer is the best entry point for implementing low-rank security constraints.

[0097] S14. The security rank corresponding to the target network layer is obtained based on the preset information retention threshold and the security singular value distribution entropy, and the security rank is determined as the target dimension. In this embodiment, the global server judges the calculated distribution entropy based on the preset threshold, thereby determining the security rank r corresponding to the current network layer, and uses it as the target dimension for extracting the low-rank security subspace.

[0098] S15. Extract the set of preceding left and right singular vectors corresponding to the target dimension from the left and right singular vector matrix as feature basis directions, and construct a fundamental matrix composed of feature basis directions. Specifically, extract the set of left and right singular vectors corresponding to the first r dominant security directions from the left and right singular vector matrix to form matrix U. r , which serves as the fundamental matrix for constructing the subspace.

[0099] S16. Perform matrix multiplication on the fundamental matrix and its transpose to generate the global security subspace projection matrix, which serves as the security constraint subspace. (Based on the formula...) P was calculated safe The projection matrix P safe This forms a global security subspace, and subsequent server operations such as security isolation, update filtering, and projection fusion are all performed based on this matrix.

[0100] After extracting the low-rank security subspace, the global server freezes the unified model base parameters and distributes only lightweight adapters and their corresponding task configurations to each client. It also synchronously distributes the security subspace information as a global security constraint baseline. This mechanism allows key security structures in the unified model to be locked before federated training begins, providing a consistent security reference for subsequent local fine-tuning and global aggregation.

[0101] S2. Distribute the initial unified model's frozen base parameters and lightweight adapter to multiple training nodes to trigger each training node to generate hidden layer representations based on local multimodal data, and perform drift calibration on the hidden layer representations according to the offset evaluation results relative to the preset safety reference benchmark, constraining the calibrated hidden layer representations within the preset safety manifold.

[0102] In this embodiment, the global server distributes the frozen base parameters and lightweight adapter, and the client receives them and starts multimodal collaborative training. The core purpose of this step is to mitigate the strain and deviation caused by non-textual modalities (such as image or video modalities) on the originally text-aligned secure distribution manifold after entering the shared representation space through explicit representation intervention during the local computation phase of federated training.

[0103] Furthermore, such as Figure 4 As shown, step S2 includes:

[0104] S21. Each training node receives multimodal mixed input including images, videos, or text, and performs processing during the local forward propagation phase using frozen pedestal parameters to generate the original hidden layer representation located in the shared representation space. Each training node client processes multimodal data such as images, videos, and text, and performs forward propagation through the frozen unified model backbone network to obtain the original hidden layer representation H generated in the current hidden layer. multi .

[0105] S22. Identify the offset trend of the image or video input relative to the safety distribution established by text alignment to obtain the cross-modal representation drift direction component. This cross-modal representation drift direction component is calculated based on local safety anchors or reference samples maintained locally by each training node, or based on safety direction information issued by the global server, and is used to describe the offset trend of the current multimodal input relative to the safety boundary. This step quantifies the degree of interference of multimodal features on the safety distribution by comparing the displacement relationship between the current input representation and the known safety boundary.

[0106] Furthermore, such as Figure 5 As shown, step S22 includes:

[0107] S221. Establish a reference sample library containing security patterns or rejection templates locally on each training node. Input the reference sample library into the initial unified model to extract hidden layer outputs to construct local security anchors or reference sample representations; or, receive security direction information issued by the global server based on global security distribution characteristics. This step provides a reference benchmark for drift analysis, ensuring that drift calculations have a clear reference system. Local security anchors represent the model's existing security defense cognition, while the security direction information issued by the global server ensures a globally unified security reference.

[0108] S222. Compare the original hidden layer representation with the local safety anchor point, reference sample representation, or safety direction reference information in terms of geometrical spatial position. Quantify the relative displacement within the representation space by calculating the geometrical distribution relationship between the current original hidden layer representation and the aforementioned reference benchmarks in the hidden layer space.

[0109] S223. Calculate the vector difference between the original hidden layer representation and the local safety anchor point, reference sample representation, or safety direction reference information. This difference is an intermediate variable describing the current multimodal input's offset trend relative to the safety boundary, intuitively reflecting the representation manifold deviation caused by image or video input.

[0110] S224. The calculated vector difference is used as the cross-modal characterization drift direction component describing the current multimodal input relative to a preset safety reference. This cross-modal characterization drift direction component... It accurately points to the direction in which the current sample characterization deviates from the safe manifold, providing a directional basis for subsequent calibration.

[0111] S23. Obtain the system's preset intervention intensity coefficient, and use the intervention intensity coefficient to adjust the drift direction component of the cross-modal characterization, generating an offset compensation amount for calibration. Calculate... The intervention intensity coefficient η is introduced here to dynamically adjust the calibration amplitude. In practical applications, this coefficient can be dynamically adjusted according to the training rounds or data modality characteristics to avoid under-correction or over-correction.

[0112] S24. Based on the offset compensation amount, perform vector deviation correction on the original hidden layer representation to offset the original hidden layer representation into a preset safe manifold. According to the formula... By performing a vector bias correction operation, the original hidden layer representation H is... multi The model is pulled back to a position closer to the safety reference manifold. This move mitigates the dangerous representational shift caused by cross-modal fusion, ensuring that the model is in a safe and compliant hidden state before backpropagation.

[0113] S25. The result after output vector bias correction is used as the calibrated hidden layer representation, ensuring that the calibrated hidden layer representation lies within a preset safe manifold. The preset safe manifold is the vector distribution interval formed in the hidden layer space when the initial unified model extracts features from the preset reference data. The calibrated representation H calibrated It is confined within a predefined safe manifold. This step effectively reduces the risk of multimodal inputs disrupting the global safety structure, ensuring that the hidden states entering subsequent decision-making and backpropagation phases remain within a controllable range.

[0114] S3. Receive the calibrated hidden layer representations from each training node, introduce safety constraint loss, combine the node's local task loss with the proximal constraint used to limit the local parameters from deviating from the preset global reference state to generate a controlled gradient, and update the lightweight adapter according to the controlled gradient to obtain the lightweight update amount.

[0115] In this embodiment, after completing the forward representation calibration, each training node enters the backpropagation and parameter update phase. This step aims to reconstruct the traditional federated fine-tuning loss function by introducing safety constraint loss and proximal constraint terms to prevent the degradation of safety capabilities caused by optimizing only around task loss during local fine-tuning, thereby achieving joint optimization of multi-task collaborative modeling and safety boundary preservation.

[0116] Furthermore, such as Figure 6 As shown, step S3 includes:

[0117] S31. Based on the calibrated hidden layer representation, output the task prediction results and calculate the node local task loss driven by local multi-task representation. This step generates the local task loss L by comparing the task prediction results with the local real data labels. task This loss term drives the lightweight adapter to achieve performance convergence on a local, task-specific basis, which is a prerequisite for multi-task learning.

[0118] S32. Determine the safety constraint loss for maintaining the basic safety alignment state based on the preset safety alignment target. This step calculates the safety constraint loss L. safetyThe aim is to ensure that the model can effectively inherit and maintain its initial security capabilities during parameter updates, which are formed through a publicly reproducible post-training security alignment process after general pre-training. The basic security alignment state is defined as a controlled technical state that ensures that the model still has the ability to identify, refuse, and avoid explicit harmful requests during the current parameter update, thereby limiting the deviation of parameter updates from the security alignment target, ensuring the compliance of the model's output content, and preventing the generation of harmful, misleading, or uncontrolled results.

[0119] S33. Read the updated local parameters corresponding to the lightweight adapter and calculate the parameter offset distance between the local parameters corresponding to the lightweight adapter and the preset global reference adapter parameters corresponding to the lightweight adapter. Compare the local adapter parameters W obtained from the current local training. local Compared to the global reference adapter parameter W issued by the global server at the start of this round global The matrix difference between the two is calculated. This step precisely quantifies the degree of parameter shift during training and forms the mathematical basis for constructing anti-degradation constraints.

[0120] S34. Calculate the squared Frobenius norm of the parameter offset distance and, in conjunction with the preset near-end constraint strength coefficient, construct a near-end constraint term to prevent the constraint parameters from excessively drifting outside the preset safe neighborhood under heterogeneous data-driven conditions. The near-end constraint term, as the core component of the anti-degradation mechanism, is calculated using the following formula: The calculation is performed, where μ is the near-end constraint strength coefficient. This is the squared Frobenius norm. This term, by limiting the parameter offset distance, suppresses excessive drift in clients deploying local training nodes under heterogeneous data and multi-task-driven conditions. The logic of this constraint mechanism is that it does not prohibit local task learning, but rather, while allowing moderate personalized updates, it constrains the update process within a controllable and safe range.

[0121] S35. Weighted fusion of the node's local task loss, security constraint loss, and near-end constraint term is performed to construct the local total loss function. Based on the total loss function... The loss function is constructed using a weighted fusion method, where α and β are the weighting coefficients for task loss and security loss, respectively, and are calibrated based on historical data. This weighted fusion ensures that the total loss function balances the dual objectives of improving local task accuracy and maintaining global security capabilities.

[0122] S36. Perform gradient backpropagation based on the local total loss function to generate controlled gradients. This step generates controlled gradients with anti-degradation mechanisms by performing backpropagation on the total loss function. These gradients no longer simply pursue minimizing the local task loss, but instead guide the update direction to a safe parameter space while ensuring task learning.

[0123] S37. Update the parameters of the lightweight adapter based on the controlled gradient, and output the updated lightweight adapter as the lightweight update amount. Update the lightweight adapter using the generated controlled gradient. Therefore, what is uploaded to the server is no longer an update result that simply pursues local accuracy improvement, but a local secure update amount that balances task performance and security maintenance, thereby reducing the risk of security capabilities being gradually eroded during long-term federated updates.

[0124] S4. Construct an orthogonal complementary projection operator using the safety constraint subspace, perform projection filtering on the received lightweight update amount to remove out-of-bounds update components that conflict with the safety constraint subspace, and fuse the filtered update components to update the global unified model.

[0125] In this embodiment, after receiving the lightweight adapter updates uploaded by each training node, the global server no longer employs the direct averaging or simple weighted averaging strategies of traditional federated learning. Instead, it introduces an orthogonal projection filtering mechanism. This mechanism projects the update amount into the complement space of the safety subspace, forcibly eliminating abnormal perturbations or malicious offsets generated along the safety-sensitive direction, thereby suppressing the erosion of the global model's safety boundary by bad updates from the source.

[0126] Furthermore, such as Figure 7 As shown, step S4 includes:

[0127] S41. Obtain the lightweight update amount uploaded by each training node and identify the matrix dimension of the lightweight update amount. The global server parses the gradient tensor or lightweight update amount ΔW uploaded by each node. local We obtain the corresponding tensor dimension information to ensure that the matrix dimensions of subsequent projection operations match.

[0128] S42. On the server side, construct an identity matrix with the same dimensions as the matrix, and perform a projection transformation operation on the identity matrix and the global safe subspace projection matrix representing the safe constraint subspace to construct a filtering matrix representing the orthogonal complement projection operator of the safe subspace. This step corresponds to the core projection formula. Where I is the identity matrix, P safe This is the projection matrix of the previously constructed global security subspace. Through calculation... In essence, it constructs an orthogonal complementary projection operator for the safe subspace, which can project any input into the feature space of the safe subspace.

[0129] S43. Perform matrix multiplication and projection operations on the lightweight update amounts uploaded by each training node with the filtering matrix to remove malicious offset components or abnormal perturbation components generated in the dimensional directions corresponding to the non-safety-constrained subspace. This is done by performing matrix multiplication. The update values ​​of each node are projected onto the safety constraint subspace. During this process, any out-of-bounds update components that attempt to modify or exceed the global safety boundary (i.e., attempt to generate changes within the safety subspace) are mathematically eliminated, retaining only the update components ΔW that do not conflict with the global safety boundary. safe This enables security filtering for local updates.

[0130] S44. Perform federated weighted aggregation on all filtered update components to generate a new round of global lightweight adapter results, which, together with the frozen base parameters, form a new round of global unified model. Apply this to all projected filtered safety update quantities ΔW. safe Federated weighted aggregation is performed to update global adapter parameters through a constrained fusion method, and these parameters are then combined with frozen base parameters to reconstruct a globally unified model. This approach avoids fluctuations in global security performance caused by localized abnormal updates from a single client.

[0131] S45. A comprehensive assessment of the convergence status of the new round of globally unified model is performed using decision criteria. These criteria include at least one of the following: preset round number, task loss change magnitude, adapter update norm, global performance metric, or safety alignment metric. By monitoring the model's performance and safety alignment metric on the global validation set, a comprehensive determination is made as to whether the current model meets the preset convergence criteria. This mechanism ensures the controllability of the federated learning process and can promptly identify whether the model training has achieved the expected performance and safety goals.

[0132] S46. When the preset convergence requirement is not met, the results of the new round of global lightweight adapter are reissued to each training node as the global reference state for the next round, and a new round of federated training iteration is executed. The new global reference adapter parameters will be issued to each training node as the constraint benchmark for the next round, driving the model to further approach the optimal solution that balances performance and safety.

[0133] S47. When the preset convergence requirement is met, the federated aggregation process is terminated, and the final federated unified model with secure alignment constraints is output. The final federated unified model that meets the requirements of multiple tasks and has secure alignment capabilities not only has multi-task collaboration capabilities, but is also protected by the security constraint mechanism throughout the entire federated training cycle, possessing extremely high security, reliability, and anti-interference capabilities.

[0134] It is important to emphasize that, in a specific embodiment, the solution of the present invention can be specifically deployed in scenarios requiring high privacy protection, such as cyberspace security. In this scenario, different training nodes come from different institutions or business domains. Some nodes hold malicious text recognition data, some nodes hold security question-answering data, and some nodes hold mixed text and image risk discrimination data. Since the data of each participant is sensitive and isolated, it is impossible to directly share the original samples. Using the solution of the present invention, the global server extracts the security constraint subspace from the unified model, distributes the frozen base and lightweight adapter to each node, and each node performs representation intervention and anti-degradation gradient update. The server filters dangerous update directions through an orthogonal projection fusion mechanism. After multiple iterations, a unified federated model that takes into account multiple tasks such as risk identification, security question answering, and content discrimination can be obtained.

[0135] Compared to directly using the traditional federated averaging method, this embodiment can more effectively suppress the degradation of the security capabilities of the unified model during long-term federated updates while maintaining multi-task learning performance. Because this embodiment uses a scheme of freezing base parameters and updating only the lightweight adapter, it can significantly reduce the computational burden and communication overhead on the client, making it more suitable for real-world engineering environments with limited resources, diverse tasks, and extremely high security and compliance requirements.

[0136] It should be noted that the extraction level of the low-rank safe subspace, the singular value retention threshold, the representation intervention intensity coefficient, the loss function weight, the proximal constraint coefficient, and the aggregation weight strategy involved in the above embodiments can all be flexibly adjusted according to the business needs of specific application scenarios. The lightweight adapter can be implemented using a low-rank adapter, a bottleneck adapter, a prefix adapter, or other parameter-efficient fine-tuning structures. In addition, the unified model is not limited to a text unified model, but can also be a multimodal unified model or other task unified models with a shared backbone structure. As long as its core technology is still essentially in the federated training process, and the safe alignment constraint modeling of the multi-task unified model is achieved through the four mechanisms of low-rank safe subspace extraction, cross-modal representation intervention, anti-safe degradation gradient generation, and orthogonal projection fusion described in this invention, it should be considered to fall within the protection scope of this invention.

[0137] In one specific embodiment, the solution of the present invention can be deployed in a multi-medical institution collaborative modeling scenario. In this scenario, the global server is deployed on a regional medical platform or medical artificial intelligence service center, and multiple hospitals, medical imaging centers, or specialized institutions act as clients C1 to C2 respectively. k Each client holds a local dataset D1 to D2. kThe data types include medical images, electronic medical records, laboratory reports, follow-up records, and doctor-patient Q&A texts. After the process begins, the global server first performs low-rank secure subspace extraction and multi-task distribution on the unified medical model, which already possesses basic security alignment capabilities. It identifies security constraints related to medical risk rejection, treatment compliance prompts, medication contraindications, privacy protection, and uncertainty expression, and distributes task configurations such as image recognition, medical record summarization, disease risk prompts, medical Q&A, and report generation according to the data types and business needs of different medical institutions. Subsequently, the global server performs lightweight adaptation aggregation preparation, freezing base parameters and lightweight adapters to each client, enabling each medical institution to train lightweight adapters based on local data without sharing original patient data.

[0138] After receiving the frozen base and lightweight adapter, each client uses its local datasets D1 to D2 respectively. k Local training is conducted. The client first generates a hidden layer representation based on medical images, medical record text, or question-and-answer data. This hidden layer representation is then calibrated for drift by combining local security anchors or security direction information from the global server, preventing multimodal medical inputs from pushing the model representation away from the preset safe manifold. Subsequently, the client calculates a reconstruction loss incorporating proximal constraints, ensuring that the local lightweight adapter adapts to the hospital's data distribution and task requirements without excessively deviating from the global reference state. Based on this, a security degradation prevention gradient is further generated, ensuring that the local update direction is simultaneously constrained by both task performance objectives and medical security constraints. After local training is complete, each client only uploads the lightweight adapter update amount, without uploading raw privacy data such as patient images, medical record text, or question-and-answer records.

[0139] After receiving lightweight adapters uploaded by each client, the global server proceeds to the step of intercepting out-of-bounds offsets. Because the medical large-scale model has high review requirements in scenarios such as clinical assistance, patient consultation, and medical staff decision support, the server needs to determine whether uploaded updates may violate medical security boundaries. Specifically, the global server constructs an orthogonal complementary projection operator using a security constraint subspace to perform projection filtering on the lightweight updates uploaded by each client, weakening or eliminating out-of-bounds update components that may lead to inappropriate treatment suggestions, unauthorized medical judgments, dangerous medication warnings, privacy leaks, or a decrease in the ability to refuse risky responses. The filtered updates are then federated and merged to form a new round of global lightweight adapters, which, together with the frozen base, constitute the updated unified medical model.

[0140] After each round of aggregation, a decision is made based on whether the client model has converged. This decision can comprehensively consider factors such as the preset training rounds, changes in task loss for each client, the lightweight adapter update norm, diagnostic or question-answering performance on the global validation set, and medical safety alignment metrics. When the decision result is "no," the global server reissues a new round of global lightweight adapters as a reference state to each client, continuing to execute the next round of local training under low-rank security constraints, lightweight adapter uploading, and out-of-bounds offset interception. When the decision result is "yes," the process ends, and a unified medical federated model that meets the security constraints is output. Thus, this embodiment enables multi-task collaborative modeling among multiple medical institutions without leaving the local medical data, and continuously maintains medical safety rules, content review boundaries, and privacy protection requirements throughout multiple rounds of federated training.

[0141] Furthermore, embodiments of the present invention provide a federated modeling system for a multi-task unified model with security constraints. This system is applied in a distributed data processing environment and includes a global server and multiple clients. The global server includes: a security constraint subspace construction module, used to perform feature decoupling on the target network layer parameters of the initial unified model, determine the target dimension based on the feature weight distribution entropy obtained from the decoupling, and extract the feature basis directions corresponding to the target dimension to construct the security constraint subspace; and a task distribution and drift calibration triggering module, used to distribute the frozen base parameters and lightweight adapters of the initial unified model to multiple training nodes to trigger each training node to generate hidden layer representations based on local multimodal data and according to the bias relative to a preset security reference benchmark. The evaluation results are used to perform drift calibration on the hidden layer representation, constraining the calibrated hidden layer representation within a preset safe manifold. The controlled gradient receiving module receives the calibrated hidden layer representation from each training node, introduces a safe constraint loss, combines the node's local task loss with the proximal constraint used to limit the local parameters from deviating from the preset global reference state to generate a controlled gradient, and updates the lightweight adapter based on the controlled gradient to obtain the lightweight update amount. The orthogonal projection filtering and fusion module constructs an orthogonal complementary projection operator using the safe constraint subspace, performs projection filtering on the received lightweight update amount to remove out-of-bounds update components that conflict with the safe constraint subspace, and fuses the filtered update components to update the global unified model.

[0142] Each client with deployed training nodes includes the following functional modules: a local data access unit, used to read and manage the local multimodal private data of the training nodes; a hidden layer representation generation and calibration unit, used to generate the original hidden layer representation based on the local multimodal data, and perform drift calibration on the original hidden layer representation according to the offset evaluation results relative to the preset safety reference benchmark, so that the calibrated hidden layer representation is mapped to the preset safety manifold; a safety-deficient gradient generation unit, used to introduce safety constraint loss, combine local task loss and proximal constraints that limit the deviation of local parameters from the preset global reference state, and generate controlled gradients; and a parameter update and upload unit, used to update the lightweight adapter according to the controlled gradients and send the generated lightweight update amount to the global server.

[0143] Meanwhile, embodiments of the present invention provide a federated modeling device for a multi-task unified model of security constraints, comprising: at least one controller; and a memory communicatively connected to the at least one controller; wherein the memory stores instructions executable by the at least one controller, the instructions being executed by the at least one controller to enable the at least one controller to perform the federated modeling method for a multi-task unified model of security constraints as described above.

[0144] Furthermore, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a controller, implement the multi-task unified model federated modeling method for security constraints as described above.

[0145] In summary, this invention provides a method, system, device, and medium for federated modeling of a multi-task unified model with security constraints. From a system execution perspective, this invention demonstrates a closed-loop process involving collaborative training, lightweight updates, upload aggregation, and security boundary correction around a unified model between a global server and multiple clients with deployed training nodes. Overall, the method employs an iterative mechanism of "global server unified scheduling—client local training—security constraint update upload—global server aggregation correction," achieving federated modeling of a multi-task unified model while ensuring that the original data does not leave the local machine, and continuously suppressing security boundary shifts during training.

[0146] like Figure 8 As shown, starting from the process initiation point, the global server first performs low-rank security subspace extraction and multi-task distribution. The main function of this step is to identify key representation directions or constraint subspaces related to security alignment from the globally unified model, and, in conjunction with the task types, data distributions, and training requirements of different clients, distribute multi-task training objectives. Through this step, the global server can provide a consistent security constraint baseline and task adaptation basis for subsequent local training on clients, based on the shared backbone of the unified model.

[0147] After completing the extraction of the security subspace and task distribution, the global server further performs lightweight adaptation aggregation and distributes frozen bases and lightweight adapters. The frozen base represents the relatively stable shared backbone parameters in the unified model; this part remains frozen or nearly frozen during this round of federated training to maintain global general representation capabilities. The lightweight adapter, as a trainable module, is distributed to each client to handle local task-related updates and local adjustments under security constraints. This design reduces the communication burden and local training cost during federated training, while minimizing the disruption to the global security boundary caused by full parameter updates.

[0148] Subsequently, multiple clients entered the local training phase in parallel. (Reference) Figure 8 From client C1 to client C k For example, they represent 1 to the 1st. k Each participating node utilizes its own local dataset D1 to D2. k The lightweight adapter is trained instead of the raw data being uploaded directly. Since different clients may undertake different tasks or have different local data distributions, the local training process for each client includes both task-driven parameter updates and constrained optimization processes under security constraints.

[0149] Within each client, a reconstruction loss incorporating proximal constraints is first calculated. This step, by adding a proximal constraint term to the local objective function, ensures that the adapter parameters trained on the current client do not deviate excessively from the global reference state issued by the global server. The reconstruction loss maintains the quality of the local task representation, while the proximal constraint mitigates training drift caused by multi-task heterogeneity and local distribution bias. In this way, while ensuring local task adaptability, the aggregability and global consistency of updates from different clients are enhanced.

[0150] Building upon this, each client further generates a gradient to prevent security degradation. This step is a crucial element that distinguishes this invention from traditional federated multi-task training processes. Its core function is to construct a gradient correction term that suppresses the decline in security alignment capability while optimizing local tasks, incorporating pre-extracted security subspace or security constraint information from the global server. This gradient does not simply aim to minimize local task loss, but simultaneously constrains the local update direction to avoid violating the existing security boundaries of the global model, thereby reducing the risk of security capability being gradually eroded during multiple rounds of federated training.

[0151] After local training is complete, each client uploads the trained lightweight adapter to the global server. Figure 8The right side shows a separate processing branch for "Upload Lightweight Adapter," indicating that the system does not transmit the complete unified model parameters, but mainly sends back a lightweight update module optimized by local tasks and security constraints. This mechanism can protect data privacy, reduce communication overhead, and enhance the system's adaptability to scenarios with large-scale concurrent client participation.

[0152] After receiving the lightweight adapters uploaded by each client, the global server further performs interception of out-of-bounds offsets. This step indicates that the global server performs security consistency checks, boundary offset detection, or out-of-bounds direction suppression on the updates uploaded by each client. When it is detected that adapter updates uploaded by some clients significantly deviate from the established safe subspace, pose a potential impact on the safety boundary of the global model, or have excessively strong local task biases, the global server will weaken, correct, prune, or reject the aggregation of such updates. Through this process, insecure updates from a single or a few clients can be prevented from spreading to the entire unified model via global aggregation.

[0153] After completing the above processing, the process proceeds to determine whether the client-side model has converged. This determination can be based on a comprehensive assessment of the preset number of rounds, the magnitude of task loss changes, the adapter update norm, global performance metrics, or security alignment metrics. If the determination result is "no," the process returns to the global server side, where the global server re-executes the security subspace extraction, adapter distribution, and federated training iterations based on the new round of aggregation results. If the determination result is "yes," it indicates that the current multi-task unified model has met the predetermined requirements in terms of task performance and security alignment constraints, the process ends, and the final federated unified model that satisfies the security constraints is obtained.

[0154] Overall, Figure 8 The flowchart illustrates the overall process of training a unified multi-task model with security constraints in a federated scenario, as described in this invention. The global server extracts security-related structures and distributes tasks; clients perform constrained, lightweight training on their respective data; the global server then performs security correction and fusion on the uploaded results. After multiple iterations, a unified model that balances task performance and security alignment is obtained. Unlike traditional federated training, which primarily focuses on task optimization, this method uses maintaining security alignment as the core constraint throughout the training process, thereby reducing the risk of security boundary drift in the unified model during continuous federated updates.

[0155] In summary, regarding Figure 8As shown in the overall process, the technical solution of this invention mainly includes four key mechanisms: First, a low-rank safe subspace extraction and multi-task distribution mechanism executed by the global server, used to extract safety-related information and distribute tasks and constraints to each client; second, a cross-modal representation intervention mechanism executed by the client, used to adjust shared representations during local training and reduce the damage of adverse updates to the safety structure; third, a safety degradation prevention gradient generation mechanism executed by the client, used to construct update directions that balance task learning and safety preservation; and fourth, an orthogonal projection fusion mechanism executed by the global server, used to perform constrained fusion of lightweight updates uploaded by each client, suppressing the adverse effects of out-of-bounds offsets on the global model. These four mechanisms work together to constitute the complete technical solution of this invention.

[0156] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0158] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various modifications and variations to the invention without departing from its spirit and scope. Thus, if these modifications and variations of the invention fall within the scope of the claims and their equivalents, the invention should also include these modifications and variations.

Claims

1. A federated modeling method for a multi-task unified model with security constraints, characterized in that, include: Feature decoupling is performed on the target network layer parameters of the initial unified model. The target dimension is determined based on the feature weight distribution entropy obtained from the decoupling, and the feature basis directions corresponding to the target dimension are extracted to construct a safety constraint subspace. The initial unified model's frozen base parameters and lightweight adapter are distributed to multiple training nodes to trigger each training node to generate hidden layer representations based on local multimodal data. The hidden layer representations are then drift-calibrated according to the offset evaluation results relative to the preset safety reference benchmark, and the calibrated hidden layer representations are constrained within the preset safety manifold. The system receives the calibrated hidden layer representations from each training node, introduces a safety constraint loss, combines the node's local task loss with the proximal constraint used to limit the local parameters from deviating from the preset global reference state to generate a controlled gradient, and updates the lightweight adapter based on the controlled gradient to obtain the lightweight update amount. An orthogonal complementary projection operator is constructed using a safety constraint subspace. Projection filtering is performed on the received lightweight update quantities to remove out-of-bounds update components that conflict with the safety constraint subspace. The filtered update components are then fused to update the global unified model.

2. The multi-task unified model federated modeling method for security constraints as described in claim 1, characterized in that, Feature decoupling is performed on the target network layer parameters of the initial unified model. The target dimension is determined based on the feature weight distribution entropy obtained from the decoupling, and the feature basis directions corresponding to the target dimension are extracted to construct a safety constraint subspace, including: Identify the key backbone network layers in the initial unified model that are related to security information including ethical constraints, risk rejection, and security judgment, and extract the weight matrix of the key backbone network layers as the parameters of the target network layer; Singular value decomposition is performed on the parameters of the target network layer to decouple and obtain the left and right singular vector matrices composed of multiple principal singular value directions, as well as the feature weight sequence composed of the corresponding singular values. Based on the feature weight sequence, calculate the security singular value distribution entropy, which represents the degree of concentration of security information in the direction of the principal singular value; The security rank of the target network layer is obtained based on the preset information retention threshold and the security singularity distribution entropy, and the security rank is determined as the target dimension. Extract the set of preorder left and right singular vectors corresponding to the target dimension from the left and right singular vector matrices as feature basis directions, and construct the basis matrix composed of feature basis directions; Perform matrix multiplication on the fundamental matrix and its transpose to generate the global security subspace projection matrix, which serves as the security constraint subspace.

3. The multi-task unified model federated modeling method for security constraints as described in claim 2, characterized in that, Based on the feature weight sequence, the security singular value distribution entropy, which characterizes the concentration of security information along the principal singular value direction, is calculated, including: Traverse the preceding feature weights from the first to the target dimension in the feature weight sequence, calculate the square value of each preceding feature weight, and obtain the corresponding safe energy value. The sum of the squared values ​​of all feature weights in the feature weight sequence is calculated to obtain the total global energy. Calculate the ratio of each security energy value to the global energy sum, and obtain the probability distribution of security information corresponding to each principal singular value direction; Calculate the logarithm of the probability distribution of each security information, and multiply the probability distribution of each security information with its corresponding logarithm. The results of the product operation are summed, and the sum is negative to generate the safe singular value distribution entropy.

4. The multi-task unified model federated modeling method for security constraints as described in claim 1, characterized in that, Each training node is triggered to generate a hidden layer representation based on local multimodal data, and drift calibration is performed on the hidden layer representation according to the offset evaluation results relative to a preset safety reference benchmark. The calibrated hidden layer representation is then constrained within a preset safety manifold, including: Each training node receives multimodal mixed inputs including images, videos, or text, and performs processing in the local forward propagation stage using frozen pedestal parameters to generate the original hidden layer representation located in the shared representation space. Identify the offset trend of the image or video input relative to the safety distribution established by text alignment to obtain the cross-modal representation drift direction component; wherein, the cross-modal representation drift direction component is calculated based on the local safety anchors or reference samples maintained locally by each training node, or based on the safety direction information issued by the global server, and is used to describe the offset trend of the current multimodal input relative to the safety boundary; The system's preset intervention intensity coefficient is obtained, and the cross-modal characterization drift direction component is adjusted using the intervention intensity coefficient to generate the offset compensation amount for calibration. The vector deviation of the original hidden layer representation is corrected based on the offset compensation amount, so as to offset the original hidden layer representation into the preset safe manifold. The result after output vector deviation correction is used as the calibrated hidden layer representation, so that the calibrated hidden layer representation is within the preset safe manifold. The preset safe manifold is the vector distribution interval formed in the hidden layer space when the initial unified model extracts features from the preset reference data.

5. The multi-task unified model federated modeling method for security constraints as described in claim 4, characterized in that, Identify the offset trend of image or video input relative to a secure distribution established by text alignment to obtain cross-modal characterization of drift direction components, including: A reference sample library containing safe modes or rejection templates is established locally on each training node. The reference sample library is input into the initial unified model to extract the hidden layer output in order to construct local safe anchors or reference sample representations. Alternatively, receive security direction information issued by the global server based on global security distribution characteristics; The original hidden layer representation is geometrically spatially compared with the local safety anchor point, reference sample representation, or safety direction reference information, respectively. Calculate the vector difference between the original hidden layer representation and the local safety anchor point, the reference sample representation, or the safety direction reference information; The calculated vector difference is used as the cross-modal drift direction component describing the current multimodal input relative to the preset safety reference.

6. The multi-task unified model federated modeling method for security constraints as described in claim 1, characterized in that, Each training node, based on the calibrated hidden layer representation, introduces a safety constraint loss. This loss, combined with the node's local task loss and proximal constraints used to limit local parameters from deviating from a preset global reference state, generates a controlled gradient. The lightweight adapter is then updated based on this controlled gradient, yielding the lightweight update amount, including: Based on the calibrated hidden layer representation output task prediction results, calculate the node local task loss driven by local multi-task representation. The security constraint loss for maintaining the basic security alignment state is determined based on the preset security alignment target; wherein, the basic security alignment state is the controlled technical state that can continuously maintain the identification, rejection and risk avoidance of explicit harmful requests during the parameter update process; Read the local parameters corresponding to the updated lightweight adapter, and calculate the parameter offset distance between the local parameters corresponding to the lightweight adapter and the preset global reference adapter parameters corresponding to the lightweight adapter; Calculate the squared Frobenius norm of the parameter offset distance and combine it with the preset near-end constraint strength coefficient to construct a near-end constraint term that causes the constraint parameter to drift excessively to outside the preset safe neighborhood under heterogeneous data driving. The node's local task loss, security constraint loss, and near-end constraint term are weighted and fused to construct the local total loss function; The gradient backpropagation algorithm is executed based on the local total loss function to generate controlled gradients; The parameters of the lightweight adapter are updated based on the controlled gradient, and the updated lightweight adapter is output as the lightweight update amount.

7. The multi-task unified model federated modeling method for security constraints as described in claim 2, characterized in that, An orthogonal complementary projection operator is constructed using a safety constraint subspace. Projection filtering is performed on the received lightweight update quantities to remove out-of-bounds update components that conflict with the safety constraint subspace. The filtered update components are then fused to update the global unified model, including: Obtain the lightweight update volume uploaded by each training node and identify the matrix dimension of the lightweight update volume; On the server side, construct an identity matrix with the same dimension as the matrix, and perform a projection transformation operation on the identity matrix and the global security subspace projection matrix representing the security constraint subspace to construct a filter matrix representing the orthogonal complementary projection operator of the security subspace. The lightweight update uploaded by each training node is subjected to matrix multiplication and projection operations with the filter matrix to remove malicious offset components or abnormal perturbation components generated in the dimensional direction corresponding to the non-safety constraint subspace. Federated weighted aggregation is performed on all filtered update components to generate a new round of global lightweight adapter results, which together with the frozen base parameters form a new round of global unified model; The convergence state of the new round of the global unified model is comprehensively judged using the judgment criteria; the judgment criteria include at least one of the following: preset round number, task loss change magnitude, adapter update norm, global performance index or security alignment index. When the preset convergence requirement is not met, the results of the new round of global lightweight adapter are used as the global reference state for the next round and reissued to each training node to execute a new round of federated training iteration. When the preset convergence requirement is met, the federated aggregation process is terminated, and the final federated unified model with safe alignment constraints is output.

8. A federated modeling system for a multi-task unified model of security constraints, characterized in that, This includes a global server and multiple clients. The global server includes: The security constraint subspace construction module is used to perform feature decoupling on the target network layer parameters of the initial unified model, determine the target dimension based on the feature weight distribution entropy obtained from the decoupling, and extract the feature basis directions corresponding to the target dimension to construct the security constraint subspace. The task distribution and drift calibration triggering module is used to distribute the frozen base parameters and lightweight adapter of the initial unified model to multiple training nodes, so as to trigger each training node to generate hidden layer representations based on local multimodal data, and perform drift calibration on the hidden layer representations according to the offset evaluation results relative to the preset safety reference benchmark, and constrain the calibrated hidden layer representations within the preset safety manifold. The controlled gradient receiving module is used to receive the calibrated hidden layer representation of each training node, introduce a safety constraint loss, combine the node's local task loss with the proximal constraint used to limit the local parameters from deviating from the preset global reference state to generate a controlled gradient, and update the lightweight adapter according to the controlled gradient to obtain the lightweight update amount. The orthogonal projection filtering and fusion module is used to construct orthogonal complementary projection operators using the safety constraint subspace, perform projection filtering on the received lightweight update quantities to remove out-of-bounds update components that conflict with the safety constraint subspace, and fuse the filtered update components to update the global unified model.

9. A federated modeling apparatus for a multi-task unified model of security constraints, characterized in that, include: At least one controller; and a memory that is communicatively connected to at least one controller; The memory stores instructions that can be executed by at least one controller, which are executed by at least one controller to enable the at least one controller to perform the multi-task unified model federated modeling method for security constraints as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer-executable instructions thereon, characterized in that, When the executable instructions are executed by the controller, they implement the multi-task unified model federated modeling method for security constraints as described in any one of claims 1-7.

Images