Cloud-edge collaboration-based large model parameter adjustment method and related device

Abstract

The embodiment of the application provides a cloud-edge collaboration-based large model parameter adjustment method and related equipment, and belongs to the technical field of network security. The method comprises the following steps: acquiring a general basic large model, a network security knowledge matrix and cloud sample data in a network security scene, and constructing an initial low-rank matrix; inputting the cloud sample data into the basic large model, performing data mapping processing on the cloud sample data based on an original parameter matrix and the initial low-rank matrix, and obtaining cloud target data; adjusting the initial low-rank matrix based on the cloud sample data and the cloud target data, and obtaining a target low-rank matrix; integrating the target low-rank matrix and the basic large model into a teacher model and sending the teacher model to an edge side device, so that the edge side device determines a target large model for completing cloud-edge collaboration parameter adjustment in the network security scene according to the teacher model. The application can balance the knowledge accuracy and parameter adjustment efficiency of the target large model in the case of introducing specific domain knowledge.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and in particular to a large model parameter adjustment method based on cloud-edge collaboration and related equipment. BACKGROUND

[0002] In order to apply a general large language model to a professional field such as network security, related technologies introduce specific field knowledge (such as network security field) and deploy it to edge side devices (such as firewalls, terminal detection devices), and then the edge side devices perform full adjustment on the full parameters of the general large language model to obtain a target large model after parameter adjustment.

[0003] However, with the surge in model structure complexity and the introduction of related knowledge, edge side devices with limited computing resources are difficult to efficiently complete full parameter adjustment while taking into account the constraints of specific field knowledge. That is, the related technical means adopted by related technologies to obtain a target large model with specific field knowledge on edge side devices have the problem of being unable to effectively balance knowledge accuracy and parameter adjustment efficiency. SUMMARY

[0004] The main purpose of the embodiments of the present application is to propose a large model parameter adjustment method based on cloud-edge collaboration and related equipment, aiming to balance the knowledge accuracy and parameter adjustment efficiency of the target large model deployed on the edge side device in the case of introducing specific field knowledge.

[0005] To achieve the above-mentioned purpose, the first aspect of the embodiments of the present application proposes a large model parameter adjustment method based on cloud-edge collaboration, applied to a cloud server, the large model parameter adjustment method based on cloud-edge collaboration comprising:

[0006] Obtaining a general basic large model, a network security knowledge matrix, and cloud sample data under a network security scenario, and constructing an initial low-rank matrix, wherein the basic large model comprises a plurality of initial processing layers, each initial processing layer comprises a corresponding original parameter matrix, the shape of the original parameter matrix is determined by an input space dimension vector and an output space dimension vector, the input space dimension vector and the output space dimension vector are high-order dense vectors, and the shape of the initial low-rank matrix is determined by an input space dimension and an output space dimension vector;

[0007] Inputting the cloud sample data into the basic large model, performing data mapping processing on the cloud sample data based on the original parameter matrix and the initial low-rank matrix, and obtaining cloud target data;

[0008] The cloud task loss value is determined based on cloud sample data and cloud target data, the knowledge regularization loss value is determined based on the network security knowledge matrix and low-rank matrix, and the initial low-rank matrix is ​​adjusted according to the cloud task loss value and the knowledge regularization loss value to obtain the target low-rank matrix.

[0009] The target low-rank matrix and the basic large model are integrated into a teacher model and sent to the edge device. This enables the edge device to train the constructed student model based on the teacher model and determine the target large model for cloud-edge collaborative parameter adjustment in the network security scenario based on the trained student model.

[0010] In some embodiments, constructing an initial low-rank matrix includes:

[0011] Determine the low-rank dimension vector;

[0012] A first initial low-rank matrix and a second initial low-rank matrix are constructed based on the low-rank dimension vector. The shape of the first initial low-rank matrix is ​​determined by the input space dimension vector and the low-rank dimension vector, and the shape of the second initial low-rank matrix is ​​determined by the output space dimension vector and the low-rank dimension vector.

[0013] In some embodiments, data mapping processing is performed on the cloud sample data based on the original parameter matrix and the initial low-rank matrix to obtain the cloud target data, including:

[0014] Transpose the first initial low-rank matrix to obtain the low-rank transpose matrix;

[0015] The cloud sample data is mapped based on the original parameter matrix to obtain the first mapping vector, and the cloud sample data is mapped based on the low-rank transpose matrix and the second initial low-rank matrix to obtain the second mapping vector.

[0016] By superimposing the first mapping vector and the second mapping vector, the target data in the cloud is obtained.

[0017] In some embodiments, the knowledge regularization loss value is determined based on the cybersecurity knowledge matrix and the low-rank matrix, including:

[0018] Acquire knowledge to adjust weighting parameters;

[0019] The incremental parameter matrix is ​​determined by the product of the low-rank transpose matrix and the second initial low-rank matrix. The incremental parameter matrix is ​​used to characterize the amount of change in the original parameter matrix.

[0020] Based on the product of the incremental parameter matrix and the network security knowledge matrix, the domain knowledge consistency matrix is ​​determined, and the sum of squares of all matrix elements in the domain knowledge consistency matrix is ​​calculated to obtain the total sum of squares.

[0021] The knowledge regularization loss value is determined by multiplying the sum of squares and the knowledge adjustment weight parameter.

[0022] In some embodiments, the target large model is obtained by training the student model through multiple rounds of model training;

[0023] After integrating the target low-rank matrix and the basic large model into a teacher model and sending it to the edge device, the following is also included:

[0024] Obtain the basic model parameters of the basic large model and receive incremental parameters from the edge device. The incremental parameters are determined by the edge device based on the basic model parameters and the adjusted student model parameters.

[0025] Based on the basic model parameters and incremental parameters, the incremental weights are determined. In the next training round, the cloud server adjusts the target model parameters of the target large model again based on the incremental weights and the target low-rank matrix. The target large model after parameter adjustment is then subjected to structured pruning to obtain an optimized large model with cloud-edge collaborative parameter adjustment in the network security scenario.

[0026] To achieve the above objectives, a second aspect of this application proposes a method for adjusting large model parameters based on cloud-edge collaboration, applied to edge devices. The method includes:

[0027] The system receives the teacher model sent from the cloud server and acquires edge sample data. The teacher model includes a basic large model and a target low-rank matrix. The basic large model is pre-acquired by the cloud server, and the target low-rank matrix is ​​obtained by the cloud server by adjusting the pre-constructed low-rank matrix.

[0028] A student model is constructed based on the basic large model. Marginal sample data is input into the teacher model and the student model respectively to obtain teacher sample data and student sample data respectively.

[0029] The distillation loss value is determined based on teacher sample data and student sample data, and the parameters of the student model are adjusted based on the distillation loss value. The adjusted student model is then subjected to structured pruning to obtain the target large model for cloud-edge collaborative parameter adjustment in the network security scenario.

[0030] In some embodiments, determining the distillation loss value based on teacher sample data and student sample data includes:

[0031] Obtain the real sample labels corresponding to the marginal sample data, and the data complexity that characterizes the data processing difficulty of the student model;

[0032] Guided by the preset distillation loss weights, the edge task loss value is determined based on the real sample labels and student sample data;

[0033] The temperature coefficient is determined based on data complexity, and the knowledge transfer loss value is jointly determined based on the temperature coefficient, distillation loss weight, teacher sample data, and student sample data.

[0034] The distillation loss value is obtained by superimposing the edge task loss value and the knowledge transfer loss value.

[0035] In some embodiments, determining the temperature coefficient based on data complexity includes:

[0036] Obtain the adaptive function;

[0037] Adaptive data processing is performed on the data complexity based on the adaptive function to obtain the temperature coefficient. The target large model is obtained by training the student model through multiple rounds. The data complexity obtained in the current training round is determined based on the distillation loss value obtained in the previous training round. The initial value of the data complexity is a preset value.

[0038] In some embodiments, the student model after parameter adjustment is subjected to structured pruning to obtain the target large model for cloud-edge collaborative parameter adjustment in a cybersecurity scenario, including:

[0039] Determine the removal parameters for the student model, whereby the removal parameters are limited to a preset multiple of the student model parameters;

[0040] The pruning factor is determined based on the removal parameters and student model parameters, and the optimization function is determined based on the square value of the Euclidean norm of the pruning factor.

[0041] With the goal of minimizing the optimization function, the student model after parameter adjustment is subjected to structured pruning to obtain the target large model for cloud-edge collaborative parameter adjustment in the network security scenario.

[0042] To achieve the above objectives, a third aspect of this application proposes a large model parameter adjustment device based on cloud-edge collaboration, applied to a cloud server. The large model parameter adjustment device based on cloud-edge collaboration includes:

[0043] The acquisition module is used to acquire a general basic large model, a network security knowledge matrix, and cloud sample data in network security scenarios, and construct an initial low-rank matrix. The basic large model includes multiple initial processing layers, each of which includes a corresponding original parameter matrix. The shape of the original parameter matrix is ​​determined by the input space dimension vector and the output space dimension vector. Both the input space dimension vector and the output space dimension vector are high-order dense vectors. The shape of the initial low-rank matrix is ​​determined by the input space dimension and the output space dimension vector.

[0044] The cloud data mapping and processing module is used to input cloud sample data into the basic large model, and perform data mapping processing on the cloud sample data based on the original parameter matrix and the initial low-rank matrix to obtain the cloud target data;

[0045] The target low-rank matrix determination module is used to determine the cloud task loss value based on cloud sample data and cloud target data, determine the knowledge regularization loss value based on the network security knowledge matrix and low-rank matrix, and adjust the initial low-rank matrix according to the cloud task loss value and knowledge regularization loss value to obtain the target low-rank matrix.

[0046] The sending module is used to integrate the target low-rank matrix and the basic large model into a teacher model and send it to the edge device, so that the edge device can train the constructed student model based on the teacher model, and determine the target large model for cloud-edge collaborative parameter adjustment in the network security scenario based on the trained student model.

[0047] To achieve the above objectives, a fourth aspect of this application proposes a large model parameter adjustment device based on cloud-edge collaboration, applied to edge devices. The large model parameter adjustment device based on cloud-edge collaboration includes:

[0048] The receiving module is used to receive the teacher model sent from the cloud server and obtain edge sample data. The teacher model includes a basic large model and a target low-rank matrix. The basic large model is obtained in advance by the cloud server, and the target low-rank matrix is ​​obtained by the cloud server by adjusting the pre-constructed low-rank matrix.

[0049] The sample data processing module is used to build a student model based on the basic large model, and input marginal sample data into the teacher model and student model respectively to obtain teacher sample data and student sample data respectively.

[0050] The target large model generation module is used to determine the distillation loss value based on teacher sample data and student sample data, adjust the student model parameters based on the distillation loss value, and perform structured pruning on the adjusted student model to obtain the target large model with cloud-edge collaborative parameter adjustment in the network security scenario.

[0051] To achieve the above objectives, a fifth aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the cloud-edge collaborative large model parameter adjustment method of the first aspect or the cloud-edge collaborative large model parameter adjustment method of the second aspect.

[0052] To achieve the above objectives, a sixth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the cloud-edge collaborative large model parameter adjustment method of the first aspect or the cloud-edge collaborative large model parameter adjustment method of the second aspect.

[0053] This application proposes a method, apparatus, electronic device, and storage medium for adjusting parameters of a large-scale model based on cloud-edge collaboration. It acquires a general basic large-scale model, a network security knowledge matrix, and cloud sample data from network security scenarios, and constructs an initial low-rank matrix. The basic large-scale model includes multiple initial processing layers, each containing a corresponding original parameter matrix. The shape of the original parameter matrix is ​​determined by the input and output space dimension vectors, both of which are high-order dense vectors. The shape of the initial low-rank matrix is ​​also determined by the input and output space dimension vectors. Cloud sample data is input into the basic large-scale model, and data mapping processing is performed on the cloud sample data based on the original parameter matrix and the initial low-rank matrix to obtain target cloud data. Finally, based on the cloud sample data and the target cloud data, parameters are determined... The cloud task loss value is determined based on the network security knowledge matrix and the low-rank matrix, and the initial low-rank matrix is ​​adjusted according to the cloud task loss value and the knowledge regularization loss value to obtain the target low-rank matrix. The target low-rank matrix and the basic large model are integrated into a teacher model and sent to the edge device. After receiving the teacher model, the edge device constructs a student model based on the basic large model and inputs the acquired edge sample data into the teacher model and student model respectively to obtain teacher sample data and student sample data. Then, the edge device determines the distillation loss value based on the teacher sample data and student sample data, adjusts the student model parameters based on the distillation loss value, and performs structured pruning on the adjusted student model to obtain the target large model with cloud-edge collaborative parameter adjustment in the network security scenario.

[0054] This application first introduces a low-rank adaptation matrix in the cloud and incorporates knowledge regularization loss during fine-tuning, imposing a strong expert knowledge constraint on model training. This forces the updated parameters to align with specific domain knowledge, significantly suppressing model illusions and greatly improving the professionalism and accuracy of generated content in the cybersecurity field. Secondly, at the edge, the knowledge-enhanced, high-quality cloud-sent teacher model is distilled and pruned, efficiently transferring it to a lightweight edge student model to obtain the final target large model. In other words, this application, while introducing specific domain knowledge, fully balances the contradiction between the target large model's knowledge accuracy at the edge and the efficiency of parameter adjustment and inference. It ensures the model possesses deep cognitive capabilities in the professional domain while meeting the edge device's requirements for low latency, high concurrency, and real-time response. This achieves collaborative optimization from high-precision cloud training to efficient edge deployment, providing a scalable and deployable technical path for the application of large models in professional fields such as cybersecurity. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of an optional implementation environment for the cloud-edge collaborative large model parameter adjustment device provided in this application embodiment;

[0056] Figure 2 This is an optional flowchart of a large model parameter adjustment method based on cloud-edge collaboration provided in the embodiments of this application;

[0057] Figure 3 yes Figure 2 Step 101 in the flowchart is an optional implementation.

[0058] Figure 4 yes Figure 2 Step 102 in the flowchart is an optional implementation.

[0059] Figure 5 yes Figure 2 Step 103 is an optional implementation flowchart;

[0060] Figure 6 yes Figure 2 Step 104 in the flowchart is an optional implementation.

[0061] Figure 7 This is another optional flowchart of the large model parameter adjustment method based on cloud-edge collaboration provided in the embodiments of this application;

[0062] Figure 8 yes Figure 7 Step 203 is an optional implementation flowchart;

[0063] Figure 9This is a schematic diagram of an optional data processing method for adjusting large model parameters in cloud-edge collaboration provided in this application embodiment;

[0064] Figure 10 yes Figure 8 Step 203.1.3 is an optional implementation flowchart;

[0065] Figure 11 yes Figure 7 Another optional implementation flowchart for step 203;

[0066] Figure 12 This is a schematic diagram of an optional device module for a parameter adjustment device applied to a cloud server, provided in an embodiment of this application.

[0067] Figure 13 This is a schematic diagram of an optional device module of the parameter adjustment device for edge-side devices provided in the embodiments of this application;

[0068] Figure 14 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0070] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0072] In order to apply the general large language model to professional fields such as network security, related technologies have introduced domain-specific knowledge (such as network security) and deployed it on edge devices (such as firewalls and endpoint detection devices). Then, the edge devices will adjust all the parameters of the general large language model to obtain the target large model after parameter adjustment.

[0073] However, with the surge in model complexity and the introduction of related knowledge, edge devices with limited computing resources struggle to efficiently adjust all parameters while balancing domain-specific knowledge constraints. In other words, the techniques employed to obtain large target models with domain-specific knowledge on edge devices suffer from an inability to effectively balance knowledge accuracy with parameter adjustment efficiency.

[0074] Based on this, this application provides a method for adjusting the parameters of a large cloud-edge collaborative model and related equipment. By constructing a systematic cloud-edge collaborative framework, it resolves the core contradiction between the accuracy of knowledge and the efficiency of deployment in the application of large models in the field of cybersecurity. Specifically, firstly, a low-rank adaptation matrix is ​​introduced in the cloud, and a knowledge regularization loss based on a "cybersecurity knowledge matrix" is introduced during the fine-tuning process. This is equivalent to imposing a strong expert knowledge constraint on the model training, forcing its updated parameters to be consistent with specific domain knowledge, thereby greatly suppressing the "illusion" phenomenon of the model and significantly improving the professionalism and accuracy of the generated content in the field of cybersecurity. Secondly, at the edge, the knowledge-enhanced, high-quality cloud-sent teacher model is distilled and pruned, thereby efficiently transferring it to a lightweight edge student model to obtain the final target large model. The obtained target large model not only possesses the deep knowledge of cybersecurity experts, but also has a small size and fast inference speed, which can meet the real-time interaction requirements of high concurrency and low latency. It perfectly connects the entire technical chain from high-precision cloud training to high-efficiency edge deployment, realizing the deep empowerment and wide application of large models in professional fields.

[0075] In other words, this application, by introducing knowledge of specific domains, fully balances the contradiction between the knowledge accuracy of the target large model on the edge side and the parameter adjustment and inference efficiency. It ensures that the model has a deep cognitive ability in the professional domain and meets the needs of edge devices for low latency, high concurrency and real-time response. It realizes the collaborative optimization from high-precision cloud training to efficient edge deployment, and provides a scalable and deployable technical path for the application of large models in professional fields such as network security.

[0076] For example, such as Figure 1 As shown, Figure 1This is a schematic diagram of an optional implementation environment for the cloud-edge collaborative large model parameter adjustment device provided in this application embodiment. The implementation environment includes a client 11 and a server 12. The cloud-edge collaborative large model parameter adjustment device (which can also be simply referred to as the "parameter adjustment device") applied to the cloud server is deployed on the server 12. The server 12 first acquires a general basic large model, a network security knowledge matrix, and cloud sample data under network security scenarios, and constructs an initial low-rank matrix. The basic large model includes multiple initial processing layers, each including a corresponding original parameter matrix. The shape of the original parameter matrix is ​​determined by the input space dimension vector and the output space dimension vector, both of which are high-order dense vectors. The shape of the initial low-rank matrix is ​​determined by the input space dimension and the output space dimension vector. Next, cloud sample data is input into the basic large model. Based on the original parameter matrix and the initial low-rank matrix, data mapping processing is performed on the cloud sample data to obtain cloud target data. Then, based on the cloud sample data and the cloud target data, the cloud task loss value is determined. Based on the network security knowledge matrix and the initial low-rank matrix, the cloud task loss value is determined. The rank matrix determines the knowledge regularization loss value, and the initial low-rank matrix is ​​adjusted based on the cloud task loss value and the knowledge regularization loss value to obtain the target low-rank matrix. Then, server 12 integrates the target low-rank matrix and the basic large model into a teacher model and sends it to the edge device deployed on the client. After receiving the teacher model, the edge device constructs a student model based on the basic large model and inputs the acquired edge sample data into the teacher model and student model respectively to obtain teacher sample data and student sample data. Next, the edge device determines the distillation loss value based on the teacher sample data and student sample data, adjusts the student model parameters based on the distillation loss value, and performs structured pruning on the adjusted student model to obtain the target large model that completes the cloud-edge collaborative parameter adjustment in the network security scenario.

[0077] The server 12 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Additionally, the server 12 can also be a node server in a blockchain network. The client 11 can be a mobile phone, computer, smart voice interaction device, smart wearable device, smart home appliance, in-vehicle terminal, etc., but is not limited to these. The client 11 and the server 12 can be connected directly or indirectly through wired or wireless communication, which is not limited in this embodiment.

[0078] It should be noted that in this application embodiment, when it involves information related to user characteristics such as basic user information or user identity, the user's permission or consent will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when this application embodiment needs to obtain sensitive personal information of a user, the user's separate permission or consent will be obtained first. Only after obtaining the user's separate permission or consent will the necessary data for the normal operation of this application embodiment be obtained. For example, before obtaining cloud sample data for training, this application embodiment will obtain the authorization or consent of relevant personnel. Furthermore, other relevant data obtained by the parameter adjustment device of this application are all authorized data, and will not be elaborated upon here.

[0079] In this application embodiment, the description will focus on the dimension of a large model parameter adjustment device based on cloud-edge collaboration applied to a cloud server, such as... Figure 2 As shown, Figure 2 This is an optional flowchart of a large model parameter adjustment method based on cloud-edge collaboration provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, the following steps 101 to 104. When the cloud-edge collaborative large model parameter adjustment device executes the cloud-edge collaborative large model parameter adjustment method (hereinafter referred to as the "parameter adjustment method" for ease of description), the specific process is as follows. It should be noted that this embodiment... Figure 2 The order of steps 101 to 104 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0080] Step 101: Obtain the general basic large model, the network security knowledge matrix, and cloud sample data in network security scenarios, and construct an initial low-rank matrix. The basic large model includes multiple initial processing layers, each of which includes a corresponding original parameter matrix. The shape of the original parameter matrix is ​​determined by the input space dimension vector and the output space dimension vector. Both the input space dimension vector and the output space dimension vector are high-order dense vectors. The shape of the initial low-rank matrix is ​​determined by the input space dimension and the output space dimension vector.

[0081] Step 101 will be described in detail below.

[0082] In some embodiments, the initialization and resource preparation of the cloud server are performed first, and the parameter adjustment device deployed on the cloud server will obtain three core elements:

[0083] (1) A general, unoptimized basic model:

[0084] Among them, the basic large-scale model refers to a large-scale language model pre-trained on a large-scale general corpus (such as web pages, books, code, etc.). The basic large-scale model has powerful language understanding and generation capabilities, but its explanatory ability in specific domains is poor. The reason for this phenomenon is that although the general basic large-scale model is pre-trained on a large amount of internet text with a wide coverage, it is difficult to deeply grasp the deep knowledge, professional terminology or complex context of specific domains. Its generation relies on statistical patterns rather than true domain reasoning, which may lead to superficial explanations, omission of key details, or even factual errors (illusion phenomenon). In addition, the general basic large-scale model lacks domain-specific structured knowledge (such as professional databases or domain rules), which further limits the accuracy and depth of its explanations.

[0085] Furthermore, the foundational large-scale model can be a variety of advanced large-scale pre-trained language models, such as the Meta AI series of large language models (LLaMA), the Bidirectional Encoder Representations from Transformers series of models (BERT), the Generative Pre-trained Transformer series of models (GPT), and the DeepSeek series of large language models. These models are all pre-trained on massive general-purpose corpora, and the parameter-tuned models deployed on cloud servers already possess powerful language understanding and generation capabilities when acquired, serving as a general foundation for fine-tuning models in professional fields such as cybersecurity, healthcare, and finance.

[0086] Furthermore, the basic large-scale model consists of multiple initial processing layers. The original parameter matrix within each layer determines the model's generality. However, these original parameter matrices have a huge number of parameters and require expensive training resources because their input and output space dimension vectors are both high-order dense vectors. Specifically, the input space dimension vector refers to the vector dimension when a certain initial processing layer of the basic large-scale model receives input features, determining the breadth of information that layer can process; the output space dimension vector is the vector dimension of the output result of the initial processing layer, used to represent the transformed information representation space. The input and output space dimension vectors together determine the shape of the parameter matrix of that initial processing layer.

[0087] Furthermore, in related technologies, obtaining the final target large model requires adjusting the original parameter matrices corresponding to the initial processing layers one by one. However, the input space dimension vector and output space dimension vector that determine the shape of the original parameter matrix are usually high-order dense vectors. Here, high-order dense vectors refer to real number vectors with high dimensions (hundreds to thousands of dimensions, such as 768, 1024 or higher) and most of their elements being non-zero. The original parameter matrix is ​​based on high-order dense vectors because high-order dense vectors can fully express complex semantic information and contextual relationships. In this way, each initial processing layer of the basic large model can capture deep patterns and long-distance dependencies in the data to be processed based on the original parameter matrix determined by the high-order dense vectors.

[0088] (2) A structured cybersecurity knowledge matrix containing authoritative knowledge in the field of cybersecurity:

[0089] The cybersecurity knowledge matrix is ​​a structured mathematical representation that encodes cybersecurity expertise (such as vulnerability characteristics, attack behaviors, defense strategies, and MITRE ATT&CK tactics) into a numerical matrix. Typically derived from cybersecurity knowledge graphs, rule bases, or expert systems, the cybersecurity knowledge matrix, after vectorization, can be aligned, integrated, or constrained with the parameter matrices in neural networks. It not only contains semantic relationships between entities but also serves as prior knowledge during model training, guiding the model to output results consistent with domain facts, thereby suppressing "illusions" and improving the accuracy of reasoning.

[0090] It should be noted that, since the embodiments of this application require the deployment of a target large model with knowledge in the field of network security on the final edge device, the network security knowledge matrix obtained is related to the field of network security, and the cloud sample data subsequently obtained is also in the network security scenario. When actually applying the parameter adjustment method proposed in the embodiments of this application, the obtained knowledge matrix and sample data can be adjusted according to different specific related fields. The parameter adjustment method proposed in the embodiments of this application is still applicable, and this does not mean that the embodiments of this application will limit it.

[0091] (3) Cloud-based sample data used in cybersecurity scenarios for model training:

[0092] Cloud-based sample data refers to training data collected and accumulated in a cloud computing environment that is relevant to real-world network security application scenarios, such as network traffic logs, intrusion detection records, malware analysis reports, and security ticket descriptions. Cloud-based sample data is characterized by its large scale, high diversity, and relatively complete annotation, reflecting real-world attack patterns and defense requirements. During the large-scale model parameter adjustment process in a cloud-edge collaborative manner, cloud-based sample data is used to drive the model to learn the input-output mapping relationship for specific tasks.

[0093] Furthermore, to achieve efficient fine-tuning through cloud-edge collaboration, the parameter adjustment device deployed on the cloud server constructs an initial low-rank matrix based on the dimensions of the original parameter matrix. This initial low-rank matrix is ​​a lightweight matrix with a much smaller number of parameters than the original parameter matrix. It is designed to specifically learn and store new knowledge and capabilities for cybersecurity tasks during subsequent training, thereby avoiding direct modification of the massive original parameter matrix.

[0094] In some embodiments, such as Figure 3 As shown, Figure 3 yes Figure 2 Step 101 in the flowchart is an optional implementation of constructing an initial low-rank matrix, including:

[0095] 101.1.1 Determine the low-rank dimension vector;

[0096] 101.1.2 Construct a first initial low-rank matrix and a second initial low-rank matrix based on the low-rank dimension vector, wherein the shape of the first initial low-rank matrix is ​​determined by the input space dimension vector and the low-rank dimension vector, and the shape of the second initial low-rank matrix is ​​determined by the output space dimension vector and the low-rank dimension vector.

[0097] Steps 101.1.1 to 101.1.2 are described in detail below.

[0098] The low-rank dimension vector is a core hyperparameter for constructing the initial low-rank matrix. Its dimension (or "rank") is much smaller than the input and output space dimension vectors of the original parameter matrix of the base large model. It directly defines the scale and complexity of subsequently added trainable parameters. Therefore, this embodiment can determine the first and second initial low-rank matrices based on the low-rank dimension vector. The shape of the first initial low-rank matrix is ​​jointly determined by the original input space dimension vector and this newly introduced low-dimensional low-rank dimension vector; while the shape of the second initial low-rank matrix is ​​determined by the original output space dimension vector and this low-rank dimension vector.

[0099] For example, suppose the input space dimension of the original parameter matrix corresponding to an initial processing layer of the base model is 1024, and the output space dimension is also 1024, meaning the shape of the weight matrix is ​​1024×1024. The cloud server sets the dimension (i.e., rank r) of the low-rank dimension vector to 8, which means that the newly added trainable parameters will be restricted to a low-dimensional subspace much smaller than the original matrix. Based on this, two initial low-rank matrices can be constructed: the shape of the first initial low-rank matrix is ​​determined by the input dimension 1024 and the low-rank dimension 8, i.e., 1024×8; the shape of the second initial low-rank matrix is ​​determined by the output dimension 1024 and the low-rank dimension 8, i.e., 8×1024. Furthermore, in subsequent training, the parameter adjustment device only needs to update these two small initial low-rank matrices, meaning the total number of parameters to be adjusted is approximately 1024×8 + 8×1024 = 16384, instead of the original 1024×1024 = 1048576 parameters, thus significantly reducing computational overhead and achieving efficient fine-tuning.

[0100] For example, A is a first initial low-rank matrix, and B is a second initial low-rank matrix. All are initial low-rank matrices. Represents the real number field. It is a pre-defined low-rank dimension vector (rank).

[0101] It is understood that this application embodiment fundamentally changes the way parameters are adjusted by determining a low-rank dimension vector and decomposing a single high-dimensional parameter update into first and second initial low-rank matrices. Compared with related technologies, this application embodiment no longer needs to directly adjust the original parameter matrix. Instead, it adjusts the parameters of two much smaller initial low-rank matrices, thereby achieving full parameter adjustment of the original parameter matrix. Thus, the number of parameters that need to be trained in this application embodiment is reduced by an order of magnitude, greatly reducing the computational resource consumption, memory usage, and training time during model fine-tuning. This provides an efficient and feasible technical path for subsequent domain adaptation of large-scale language models under limited hardware conditions.

[0102] Step 102: Input the cloud sample data into the basic large model, and perform data mapping processing on the cloud sample data based on the original parameter matrix and the initial low-rank matrix to obtain the cloud target data.

[0103] Step 102 is described in detail below.

[0104] In some embodiments, a target processing layer is selected from multiple initial processing layers, and an initial low-rank matrix is ​​injected into the target processing layer so that the cloud sample data can be mapped based on the original parameter matrix and the initial low-rank matrix to obtain the target data in the cloud. The target processing layer is typically the attention mechanism layer of the basic large model. However, other initial processing layers can be selected as the target processing layer depending on the actual situation, and this embodiment does not impose any limitations on this.

[0105] In some embodiments, a parameter adjustment device deployed on a cloud server begins forward propagation computation of the basic large model. Specifically, cloud sample data from a cybersecurity scenario is input into the basic large model. When each processing layer of the basic large model performs data transformation, it not only relies on the original parameter matrix but also introduces a trainable initial low-rank matrix to participate in the computation: the input data is first subjected to a conventional linear mapping through the original parameter matrix, and is simultaneously fed into a bypass structure composed of a first initial low-rank matrix and a second initial low-rank matrix. After low-rank mapping transformation, it is superimposed on the main path to achieve incremental adjustment of the original output. The result is the generation of an intermediate representation that incorporates domain knowledge—that is, cloud target data. The cloud target data retains the general language understanding capability of the basic large model and initially reflects its adaptability to cybersecurity tasks, providing a feature representation with domain semantics for subsequent knowledge alignment and model optimization.

[0106] In some embodiments, such as Figure 4 As shown, Figure 4 yes Figure 2 Step 102, an optional implementation flowchart, involves data mapping processing of the cloud sample data based on the original parameter matrix and the initial low-rank matrix to obtain the cloud target data, including:

[0107] 102.1.1 Transpose the first initial low-rank matrix to obtain the low-rank transpose matrix;

[0108] 102.1.2 Based on the original parameter matrix, the cloud sample data is mapped to obtain the first mapping vector, and based on the low-rank transpose matrix and the second initial low-rank matrix, the cloud sample data is mapped to obtain the second mapping vector.

[0109] 102.1.3 Superimpose the first mapping vector and the second mapping vector to obtain the target data in the cloud.

[0110] Steps 102.1.1 to 102.1.3 are described in detail below.

[0111] In some embodiments, the cloud sample data is mapped based on the following formula (1) to obtain the cloud target data. :

[0112] (1)

[0113] in, Represents the first initial low-rank matrix; T denotes the transpose. Represents the low-rank transpose matrix; Denotes the first initial low-rank matrix; The original parameter matrix is ​​represented by x; the cloud sample data is represented by x. This represents the incremental update or fine-tuning compared to the original weight matrix; Represents the first mapping vector; or This represents the second mapping vector.

[0114] Transpose is a fundamental linear algebra operation that interchanges the rows and columns of a matrix, thus changing its shape. This operation is performed to ensure that the dimensions of matrix multiplications are correctly matched in subsequent calculations, resulting in a low-rank transpose matrix with predetermined dimensions. This prepares the computation site for calculating the parameter increments generated by low-rank paths.

[0115] Understandably, this embodiment of the application decomposes the data mapping process into two parallel paths and then superimposes them, achieving precise adjustment of the original parameter matrix without modifying the massive original parameter matrix. Specifically, this embodiment fully retains the general knowledge learned during the pre-training phase of the basic model (embodied in the first mapping vector) and injects new knowledge through a lightweight bypass (generating the second mapping vector). This design effectively avoids the "catastrophic forgetting" problem that may occur during fine-tuning, ensuring that the model's original comprehensive capabilities are not impaired while learning new domain knowledge.

[0116] Step 103: Determine the cloud task loss value based on cloud sample data and cloud target data, determine the knowledge regularization loss value based on the network security knowledge matrix and low-rank matrix, and adjust the initial low-rank matrix according to the cloud task loss value and knowledge regularization loss value to obtain the target low-rank matrix.

[0117] Step 103 will be described in detail below.

[0118] In some embodiments, the parameter tuning device deployed on a cloud server guides the adjustment of the initial low-rank matrix by calculating two different loss functions. The first is the cloud task loss value, which is determined by comparing the difference between the model's output (cloud target data) and the true labels (true data obtained along with cloud sample data). Its purpose is to guide the model to learn to complete specific cybersecurity tasks. The second, and the key innovation of this application embodiment, is determined by evaluating the consistency between the parameter update represented by the current low-rank matrix and the cybersecurity knowledge matrix. Knowledge regularization acts as a constraint to ensure that while the model is learning the task, its internal knowledge representation must be aligned with authoritative domain knowledge, thereby suppressing "illusions." Finally, the parameter tuning device iteratively adjusts the initial low-rank matrix based on the combination of these two loss values ​​until it converges to an optimized target low-rank matrix that understands the task and conforms to domain knowledge.

[0119] In some embodiments, the total loss value used to finally determine the target low-rank matrix is ​​determined based on the following formula (2). :

[0120] (2)

[0121] in, Used to learn characteristics in the field of cybersecurity and suppress interference from general corpora. This represents the cloud task loss value, which is calculated by predicting the output of the target data in the cloud. (i.e. ), and the real labels corresponding to the cloud sample data. The cross-entropy loss value between them is determined.

[0122] In some embodiments, such as Figure 5 As shown, Figure 5 yes Figure 2 Step 103, an optional implementation flowchart, determines the knowledge regularization loss value based on the network security knowledge matrix and the low-rank matrix, including:

[0123] 103.1.1 Obtain knowledge adjustment weight parameters;

[0124] 103.1.2 The incremental parameter matrix is ​​determined based on the product of the low-rank transpose matrix and the second initial low-rank matrix. The incremental parameter matrix is ​​used to characterize the amount of change in the original parameter matrix.

[0125] 103.1.3 Based on the product of the incremental parameter matrix and the network security knowledge matrix, the domain knowledge consistency matrix is ​​determined, and the sum of squares of all matrix elements in the domain knowledge consistency matrix is ​​calculated to obtain the total sum of squares;

[0126] 103.1.4 The knowledge regularization loss value is determined based on the product of the sum of squares and the knowledge adjustment weight parameter.

[0127] Steps 103.1.1 to 103.1.4 are described in detail below.

[0128] In some embodiments, as shown in formula (2), in order to determine the knowledge regularization loss value, the embodiments of this application first obtain a key hyperparameter, namely the knowledge adjustment weight parameter. , The magnitude of the value determines the weight of the subsequently calculated knowledge regularization loss value in the overall optimization objective. The specific values ​​can be dynamically set according to the actual situation, or determined adaptively by more intelligent artificial intelligence components. In this way, the model's tendency to learn task data and follow domain knowledge can be controlled, thereby achieving more refined guidance of model behavior.

[0129] in, This is represented as an incremental parameter matrix. Conceptually, the incremental parameter matrix is ​​equivalent to all the changes applied to the original parameter matrix. It embodies all the new knowledge and adaptive adjustments learned by the model in this round of fine-tuning for the cybersecurity domain, and is the core object for subsequent knowledge alignment verification.

[0130] in, It is represented as a domain knowledge consistency matrix. This represents the cybersecurity knowledge matrix. Each element of the domain knowledge consistency matrix reflects the degree of fit between the parameter changes learned by the model and the authoritative domain knowledge; if the changes are consistent with the knowledge, the element value approaches zero, and vice versa.

[0131] Furthermore, in this embodiment, the sum of squares of all matrix elements within the domain knowledge consistency matrix is ​​calculated, thereby quantifying the degree of deviation of the entire matrix into a single, non-negative scalar value, namely the total sum of squares. Next, the sum of squares representing the total knowledge bias of the model is multiplied by the pre-acquired knowledge adjustment weight parameter to obtain the knowledge regularization loss value. The knowledge regularization loss, as a well-defined and optimizable mathematical metric, will work in conjunction with the task loss to update the parameters during backpropagation. The larger the value, the stronger the "penalty" on the model parameters.

[0132] Understandably, this embodiment of the application uses the knowledge regularization loss value as an explicit penalty during the optimization phase of model training. This forces the model to not only fit the task data when adjusting parameters, but also to actively align with the knowledge system defined by the cybersecurity knowledge matrix. This fundamentally guides the optimization direction of model parameters, ensuring that the content it learns is factually consistent. This greatly suppresses the tendency of the model to produce factual errors or "illusions," and significantly improves the professionalism and reliability of the teacher model trained in the cloud in the field of cybersecurity.

[0133] Step 104: Integrate the target low-rank matrix and the basic large model into a teacher model and send it to the edge device. After receiving the teacher model, the edge device constructs a student model based on the basic large model and inputs the acquired edge sample data into the teacher model and student model respectively to obtain teacher sample data and student sample data. Then, the edge device determines the distillation loss value based on the teacher sample data and student sample data, adjusts the student model parameters based on the distillation loss value, and performs structured pruning on the adjusted student model to obtain the target large model with cloud-edge collaborative parameter adjustment in the network security scenario.

[0134] Step 104 is described in detail below.

[0135] In some embodiments, the present application scheme will perform cloud-edge collaborative knowledge transfer and model lightweighting: (1) The cloud server integrates the trained target low-rank matrix with the original basic large model into a powerful teacher model and distributes it to the edge device; (2) After receiving it, the edge device will build a student model with a smaller parameter size based on the same basic large model architecture; then, through knowledge distillation technology, the edge sample data is simultaneously input into the teacher model and the student model, and the difference between the two outputs (teacher sample data and student sample data) is calculated, which is quantified as the distillation loss value; the edge device adjusts the parameters of the student model based on the distillation loss value, so that the student model can imitate and learn the reasoning ability of the teacher model; in addition, after the student model has fully learned the knowledge of the teacher model, it will also be subjected to structured pruning, which aims to remove redundant parameters and structures in the model, further compress the model size and improve the reasoning speed. After this series of processes, a lightweight, efficient target large model in the network security scenario that inherits the knowledge of cloud experts is finally obtained on the edge (the specific training steps performed by the edge device will be described in detail later).

[0136] Understandably, this application's embodiments, by introducing knowledge regularization loss values ​​to optimize the low-rank matrix in the cloud, fundamentally ensure the model's knowledge accuracy and professionalism in the cybersecurity field, effectively solving the "illusion" problem. Simultaneously, utilizing a teacher-student model architecture, through knowledge distillation and structured pruning, the high-quality, high-precision expert capabilities trained in the cloud are losslessly transferred and compressed into a lightweight model suitable for efficient operation on resource-constrained edge devices. Ultimately, this achieves a perfect balance between knowledge depth and deployment efficiency in vertical domains for large models, providing a solid technical foundation for the implementation of high-performance, high-reliability intelligent cybersecurity applications.

[0137] In some embodiments, such as Figure 6 As shown, Figure 6 yes Figure 2 Step 104, an optional implementation flowchart, after integrating the target low-rank matrix and the underlying large model into a teacher model and sending it to the edge device, also includes:

[0138] 104.1.1 Obtain the basic model parameters of the basic large model and receive incremental parameters fed back from the edge side device. The incremental parameters are determined by the edge side device based on the basic model parameters and the adjusted student model parameters.

[0139] 104.1.2 Based on the basic model parameters and incremental parameters, the incremental weights are determined. In the next training round, the cloud server adjusts the target model parameters of the target large model again based on the incremental weights and the target low-rank matrix. The target large model after parameter adjustment is then subjected to structured pruning to obtain the optimized large model with cloud-edge collaborative parameter adjustment in the network security scenario.

[0140] Steps 104.1.1 to 104.1.2 are described in detail below.

[0141] In some embodiments, the target large model is obtained from the student model through multiple rounds of model training. To improve the accuracy and efficiency of cloud-edge collaborative model parameter adjustment, this application embodiment also initiates a feedback process from the edge to the cloud: the cloud server first obtains the basic model parameters of the base large model it holds as the starting point for fine-tuning. At the same time, it actively receives incremental parameters fed back from the deployed edge devices. The incremental parameters refer to the precise calculations made by the edge devices by comparing the differences between the initially received basic model parameters and the adjusted student model parameters formed after local knowledge distillation and pruning. In other words, the incremental parameters are essentially a quantitative manifestation of edge-environment-specific knowledge and lightweight adaptive model adjustments.

[0142] Furthermore, the parameter adjustment device on the cloud server determines an incremental weight based on its original base model parameters and the received incremental parameters. This incremental weight is used to intelligently evaluate and fuse feedback information from the edge. Subsequently, in the next training round, the cloud server uses this incremental weight and its already trained target low-rank matrix to readjust the target model parameters of the target large model. This means that the cloud-based teacher model is not static but can generate a stronger teacher model based on feedback from edge devices. Consequently, in subsequent model iterations, edge devices can obtain a higher-performance optimized large model that has undergone cloud-edge collaborative parameter adjustment in a cybersecurity scenario. The optimized large model typically outperforms the target large model.

[0143] Furthermore, the specific implementation of adjusting the target model parameters of the target large model again based on the incremental weights and the target low-rank matrix, and performing structured pruning on the target large model after parameter adjustment is similar to step 104, and will not be repeated here.

[0144] It is understood that the embodiments of this application establish a dynamic, closed-loop cloud-edge collaborative continuous learning system. This departs from the traditional "one-time training, static deployment" model, instead establishing an incremental parameter feedback mechanism from the edge to the cloud. This allows the core model in the cloud to continuously absorb new knowledge and adaptive experience learned in real-world environments from numerous edge nodes. Furthermore, the parameter adjustment device in the cloud can intelligently fuse and iterate these experiences based on incremental weights, continuously improving the performance and generalization ability of its own model. This design constitutes a virtuous cycle: the cloud model becomes stronger due to edge feedback, and the stronger cloud model can then distill even better edge models, enabling the entire system to self-evolve and dynamically adapt to new threats and scenarios, significantly enhancing the long-term effectiveness and robustness of the solution.

[0145] In this application embodiment, the description will focus on the dimensions of a large model parameter adjustment device based on cloud-edge collaboration applied to edge-side devices, such as... Figure 7 As shown, Figure 7 This is another optional flowchart of the large model parameter adjustment method based on cloud-edge collaboration provided in the embodiments of this application. Figure 7 The method may include, but is not limited to, the following steps 201 to 203. When the cloud-edge collaborative large model parameter adjustment device executes the cloud-edge collaborative large model parameter adjustment method, the specific process is as follows. It should be noted that this embodiment... Figure 7 The order of steps 201 to 203 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0146] Step 201: Receive the teacher model sent from the cloud server and obtain edge sample data. The teacher model includes a basic large model and a target low-rank matrix. The basic large model is obtained in advance by the cloud server, and the target low-rank matrix is ​​obtained by the cloud server by adjusting the pre-constructed low-rank matrix.

[0147] Step 201 will be described in detail below.

[0148] In some embodiments, the initialization and resource preparation of the edge device are first performed. The edge device receives a teacher model sent from the cloud server and acquires edge sample data for local fine-tuning. The teacher model consists of a general base model and a target low-rank matrix containing deep domain knowledge, determined in the cloud. That is, the teacher model acquired by the edge device is already a high-quality knowledge source with high accuracy and low "illusion" rate in the field of cybersecurity.

[0149] Step 202: Construct a student model based on the basic large model, and input marginal sample data into the teacher model and student model respectively to obtain teacher sample data and student sample data respectively;

[0150] Step 202 is described in detail below.

[0151] In some embodiments, the edge device builds a student model based on a base model; typically, the student model can be a copy of the base model. Subsequently, a parameter-tuning model deployed on the edge device inputs the acquired edge sample data into a complex and knowledge-rich teacher model, and a simple, untrained student model, respectively. After data processing by the two models, teacher sample data (i.e., the output of the teacher model, considered the "standard answer") and student sample data (i.e., the current output of the student model) are obtained, respectively.

[0152] Step 203: Determine the distillation loss value based on teacher sample data and student sample data, adjust the student model parameters based on the distillation loss value, and perform structured pruning on the adjusted student model to obtain the target large model for cloud-edge collaborative parameter adjustment in the network security scenario.

[0153] Step 203 will be described in detail below.

[0154] In some embodiments, a distillation loss value is determined by comparing the differences between teacher sample data and student sample data. This distillation loss value quantifies the gap in reasoning ability between the student model and the teacher model. Then, the edge device iteratively adjusts the student model parameters using a backpropagation algorithm based on this distillation loss value, aiming to minimize this gap and guide the student model to learn from the teacher model. After parameter adjustment and ensuring the student model fully absorbs the knowledge from the teacher model, the parameter adjustment device performs structured pruning to remove redundant connections or neurons, further compressing the model size and improving inference speed. Finally, after this series of fine adjustments and optimizations, a lightweight and high-performance target model is obtained on the edge device, capable of cloud-edge collaborative parameter adjustment in a cybersecurity scenario.

[0155] In some embodiments, such as Figure 8 As shown, Figure 8 yes Figure 7 Step 203, an optional implementation flowchart, determines the distillation loss value based on teacher sample data and student sample data, including:

[0156] 203.1.1 Obtain the true sample labels corresponding to the marginal sample data, and the data complexity that characterizes the data processing difficulty of the student model;

[0157] 203.1.2 Guided by the preset distillation loss weights, the marginal task loss value is determined based on the real sample labels and student sample data;

[0158] 203.1.3 Determine the temperature coefficient based on data complexity. Based on the temperature coefficient, distillation loss weight, teacher sample data, and student sample data, jointly determine the knowledge transfer loss value.

[0159] 203.1.4 The edge task loss value and the knowledge transfer loss value are superimposed to obtain the distillation loss value.

[0160] Steps 203.1.1 to 203.1.4 are described in detail below.

[0161] In some embodiments, the distillation loss value is determined based on the following formula (3). :

[0162] (3)

[0163] in, This is the weight for distillation loss, with an initial value that can be adjusted based on the distillation loss value. This represents the true sample label corresponding to the edge sample data; This represents the student sample data obtained after the student model processes the marginal sample data; This represents the loss value for edge tasks. It is determined by calculating the cross-entropy loss value between the student sample data and the real sample labels corresponding to the marginal sample data.

[0164] in, Temperature coefficient; Teacher sample data obtained after processing marginal sample data using a representational teacher model; Let represent the Kullback-Leibler divergence (KL divergence), which measures the difference between two probability distributions. In Equation (3), it measures the probability distribution representing the teacher model. , and represent the probability distribution of the student model. The differences between them.

[0165] like Figure 9 As shown, Figure 9 This is an optional data processing diagram of the cloud-edge collaborative large model parameter adjustment method provided in this application embodiment. The total loss value determined by the teacher model is used for backpropagation to update the parameters of the student model. The distillation loss value is a loss term unique to the student model due to knowledge distillation, which helps the student model learn the "knowledge" of the teacher model. Furthermore, the distillation loss value can also be used as part of the total loss value, together with other loss terms (such as task loss, regularization loss, etc.) to guide the iterative update of the model, ensuring that the student model can not only accurately complete specific tasks, but also inherit the knowledge and generalization ability of the teacher model, thereby improving the overall performance.

[0166] In addition, to determine the temperature coefficient, this application embodiment will also obtain data complexity. Data complexity refers to an indicator characterizing the complexity of marginal sample data in terms of semantic structure, feature distribution, or task difficulty; it reflects the reasoning, understanding, or classification difficulty faced by the student model when processing the data. During knowledge distillation, data complexity can be used to dynamically adjust the distillation strategy—for example, assigning higher learning weights to samples with high complexity or using a more appropriate temperature coefficient, thereby enabling the student model to learn key knowledge more effectively from the teacher model.

[0167] The following describes the method for determining the temperature coefficient:

[0168] In some embodiments, such as Figure 10 As shown, Figure 10 yes Figure 8 Step 203.1.3, an optional implementation flowchart, determines the temperature coefficient based on data complexity, including:

[0169] A.1 Obtain the adaptive function;

[0170] A.2 Adaptive data processing based on adaptive functions is used to obtain the temperature coefficient. The target large model is obtained by training the student model through multiple rounds. The data complexity obtained in the current training round is determined based on the distillation loss value obtained in the previous training round. The initial value of the data complexity is a preset value.

[0171] Steps A.1 to A.2 are described in detail below.

[0172] In some embodiments, the temperature coefficient T is determined based on the following formula (4):

[0173] T=π(Z)(4)

[0174] Where Z represents the data complexity; π is an adaptive function.

[0175] In other words, this embodiment of the application will perform adaptive data processing based on an adaptive function to obtain the temperature coefficient required for this round of training. The key to this part is the dynamic determination mechanism of data complexity: since the target large model is obtained by the student model through multiple rounds of model training, the data complexity used in the current training round is determined based on the distillation loss value obtained in the previous training round, and in the initial stage of the entire training process, the initial value of data complexity is a preset value.

[0176] For example, before each round of knowledge distillation training begins, the parameter adjustment device assesses the learning difficulty of the current student model for marginal sample data based on the distillation loss value obtained from the previous round of training. If the distillation loss value from the previous round is high, it indicates a significant difference in the output distribution between the student model and the teacher model, reflecting that the current marginal sample data is difficult for the student model to learn, thus determining that the data complexity of the marginal sample data is high. Conversely, if the distillation loss value is low, it indicates that the student has absorbed the teacher's knowledge well, and the corresponding data complexity is low. Based on this mechanism, the student model can continuously "perceive" the difficulty level of the samples during training and calculate the optimal temperature coefficient required for training in this round by inputting the updated data complexity into the adaptive function. For example, a higher temperature is used when the complexity is high to soften the probability distribution and enhance the knowledge transfer effect; the temperature is lowered when the complexity is low to focus on accurate classification. In the initial training phase, since there is no historical loss information, the data complexity is set to an empirically preset initial value (such as medium complexity). It is then dynamically updated in each iteration to achieve refined and adaptive learning of samples of different difficulty, thereby improving the convergence speed and final performance of the student model.

[0177] Understandably, this application's embodiments abandon the fixed temperature coefficient required for extensive manual parameter tuning in traditional distillation methods, establishing a dynamic closed loop: when the distillation loss value obtained in the previous training round is high, it indicates that the student model has difficulty learning on that batch of data, leading to the data complexity being judged as high in the current training round. The adaptive function then outputs a higher temperature coefficient, prompting the student model to learn the teacher's knowledge distribution more gently. Conversely, when the loss value is low, the temperature coefficient is reduced, encouraging the student model to fit the teacher's output more accurately. This round-by-round, adaptive adjustment allows the training process to intelligently "relax" or "tighten" the learning requirements according to the model's real-time learning state, greatly improving the efficiency of knowledge transfer and the performance of the final model.

[0178] In some embodiments, such as Figure 11 As shown, Figure 11 yes Figure 7 Another optional implementation flowchart for step 203 involves performing structured pruning on the student model after parameter adjustment to obtain the target large model for cloud-edge collaborative parameter adjustment in a cybersecurity scenario, including:

[0179] 203.2.1 Determine the removal parameters for the student model, wherein the removal parameters are limited to a preset multiple of the student model parameters;

[0180] 203.2.2 The pruning factor is determined based on the removal parameter and the student model parameter, and the optimization function is determined based on the square value of the Euclidean norm of the pruning factor;

[0181] 203.2.3 With the goal of minimizing the optimization function, the student model after parameter adjustment is subjected to structured pruning to obtain the target large model for cloud-edge collaborative parameter adjustment in the network security scenario.

[0182] Steps 203.2.1 to 203.2.3 are described in detail below.

[0183] In some embodiments, structured pruning of the parameter-adjusted student model is achieved based on the optimization function shown in the following formula (5):

[0184] (5)

[0185] in, Indicates the parameters of the student model; This indicates the parameters of the student model that need to be removed in the current training round (removal parameters); This indicates a preset multiplier, and its value is usually between 0 and 1; Indicates pruning factor; This represents the Hadamard product, also known as element-by-element multiplication, which is used for operations between two matrices or vectors of the same dimension. The treatment of representing the square of the Euclidean norm; To optimize the function. Formula (5) represents the parameter that is retained (i.e. Minimize the squared Frobenius norm of the model while ensuring that the number of parameters removed does not exceed the number of parameters in the original student model. times.

[0186] Furthermore, this embodiment of the application constructs an optimization function with a pruning factor at its core, transforming the model compression into an optimization problem with a clear mathematical objective. Thus, this embodiment not only effectively controls the degree of pruning through a preset factor, ensuring the stability of model performance, but more importantly, the characteristics of "structured pruning" ensure that while the model size is reduced, its computational structure becomes more regular, thereby achieving a significant inference acceleration effect on actual hardware. Ultimately, the target large model produced by this scheme achieves the optimal balance between model size, inference speed, and task performance.

[0187] Thus, the reader can understand that the two-layer migration mechanism of "cloud-based full parameter fine-tuning + edge-based partial parameter efficient fine-tuning" proposed in this application solves the adaptability defects of the traditional single fine-tuning mode in dynamic environments by constructing a heterogeneous fine-tuning paradigm collaborative framework: (1) The cloud adopts full parameter fine-tuning (Full (1) Fine-tuning), based on feature domain knowledge (such as network security knowledge matrix) to globally optimize model parameters and eliminate the domain knowledge "illusion"; (2) The edge device is the local training and inference execution terminal of the lightweight model. It is responsible for completing the closed loop of knowledge distillation, model optimization and final deployment. Specifically, after receiving the teacher model sent by the cloud, the edge device builds a student model with fewer parameters and higher computation efficiency based on the same basic large model structure. It uses the edge sample data collected locally to generate teacher output (soft label) and student output through the teacher model and student model respectively, and then calculates the distillation loss value. Subsequently, the parameters of the student model are iteratively adjusted based on the loss value so that it can adapt to the local data distribution while retaining the "knowledge" of the teacher model. Finally, the trained student model is structurally pruned to further compress the model volume and improve the inference speed. Finally, a lightweight target large model suitable for resource-constrained environments is obtained, realizing the efficient transfer and implementation of knowledge from the cloud to edge intelligence. In addition, a bidirectional quantization adapter was designed on the edge side. The forward adapter compresses the feature dimension to extract key features of the edge scene; the reverse adapter decodes the knowledge vectors sent from the cloud into edge executable instructions to realize the targeted transfer of student models' capabilities in the field of cybersecurity knowledge.

[0188] like Figure 12 As shown, Figure 12 This is a schematic diagram of an optional device module for a parameter adjustment device applied to a cloud server provided in an embodiment of this application. The large model parameter adjustment device based on cloud-edge collaboration may include the following modules 301 to 304:

[0189] The acquisition module 301 is used to acquire a general basic large model, a network security knowledge matrix, and cloud sample data in network security scenarios, and construct an initial low-rank matrix. The basic large model includes multiple initial processing layers, each of which includes a corresponding original parameter matrix. The shape of the original parameter matrix is ​​determined by the input space dimension vector and the output space dimension vector. Both the input space dimension vector and the output space dimension vector are high-order dense vectors. The shape of the initial low-rank matrix is ​​determined by the input space dimension and the output space dimension vector.

[0190] The cloud data mapping processing module 302 is used to input cloud sample data into the basic large model, and perform data mapping processing on the cloud sample data based on the original parameter matrix and the initial low-rank matrix to obtain the cloud target data.

[0191] The target low-rank matrix determination module 303 is used to determine the cloud task loss value based on cloud sample data and cloud target data, determine the knowledge regularization loss value based on the network security knowledge matrix and low-rank matrix, and adjust the initial low-rank matrix according to the cloud task loss value and knowledge regularization loss value to obtain the target low-rank matrix.

[0192] The sending module 304 is used to integrate the target low-rank matrix and the basic large model into a teacher model and send it to the edge device, so that the edge device can train the constructed student model based on the teacher model and determine the target large model for cloud-edge collaborative parameter adjustment in the network security scenario based on the trained student model.

[0193] The specific implementation of the parameter adjustment device applied to the cloud server is basically the same as the specific embodiment of the parameter adjustment method applied to the cloud server described above, and will not be repeated here.

[0194] like Figure 13 As shown, Figure 13 This is a schematic diagram of an optional device module for a parameter adjustment device applied to an edge-side device provided in an embodiment of this application. The large model parameter adjustment device based on cloud-edge collaboration may include the following modules 301 to 304:

[0195] The receiving module 401 is used to receive the teacher model sent from the cloud server and obtain edge sample data. The teacher model includes a basic large model and a target low-rank matrix. The basic large model is obtained in advance by the cloud server, and the target low-rank matrix is ​​obtained by the cloud server by adjusting the pre-constructed low-rank matrix.

[0196] The sample data processing module 402 is used to construct a student model based on the basic large model, and input marginal sample data into the teacher model and student model respectively to obtain teacher sample data and student sample data respectively.

[0197] The target large model generation module 403 is used to determine the distillation loss value based on teacher sample data and student sample data, adjust the student model parameters based on the distillation loss value, and perform structured pruning on the student model after parameter adjustment to obtain the target large model with cloud-edge collaborative parameter adjustment in the network security scenario.

[0198] The specific implementation of the parameter adjustment device applied to the edge-side device is basically the same as the specific embodiment of the parameter adjustment method applied to the edge-side device described above, and will not be repeated here.

[0199] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method for adjusting large model parameters based on cloud-edge collaboration. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0200] like Figure 14 As shown, Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes:

[0201] The processor 501 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0202] The memory 502 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 502 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 502 and is called and executed by the processor 501 using the cloud-edge collaboration-based large model parameter adjustment method of the embodiments of this application.

[0203] The input / output interface 503 is used to implement information input and output;

[0204] The communication interface 504 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0205] Bus 505 transmits information between various components of the device (e.g., processor 501, memory 502, input / output interface 503, and communication interface 504);

[0206] The processor 501, memory 502, input / output interface 503, and communication interface 504 are connected to each other within the device via bus 505.

[0207] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for adjusting large model parameters based on cloud-edge collaboration.

[0208] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0209] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0210] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0211] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0212] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0213] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0214] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0215] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0216] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0217] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0218] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0219] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for adjusting parameters of a large model based on cloud-edge collaboration, characterized in that, Applied to cloud servers, the large model parameter adjustment method based on cloud-edge collaboration includes: A general basic model, a network security knowledge matrix, and cloud sample data in network security scenarios are obtained, and an initial low-rank matrix is ​​constructed. The basic model includes multiple initial processing layers, and each initial processing layer includes a corresponding original parameter matrix. The shape of the original parameter matrix is ​​determined by the input space dimension vector and the output space dimension vector. Both the input space dimension vector and the output space dimension vector are high-order dense vectors. The shape of the initial low-rank matrix is ​​determined by the input space dimension and the output space dimension vector. The cloud sample data is input into the basic large model, and the cloud sample data is processed by data mapping based on the original parameter matrix and the initial low-rank matrix to obtain the cloud target data. The cloud task loss value is determined based on the cloud sample data and the cloud target data. The knowledge regularization loss value is determined based on the network security knowledge matrix and the initial low-rank matrix. The initial low-rank matrix is ​​then adjusted according to the cloud task loss value and the knowledge regularization loss value to obtain the target low-rank matrix. The target low-rank matrix and the basic large model are integrated into a teacher model and sent to the edge device, so that the edge device can train the constructed student model based on the teacher model, and determine the target large model for cloud-edge collaborative parameter adjustment in the network security scenario based on the trained student model.

2. The method for adjusting large model parameters based on cloud-edge collaboration according to claim 1, characterized in that, The construction of the initial low-rank matrix includes: Determine the low-rank dimension vector; A first initial low-rank matrix and a second initial low-rank matrix are constructed based on the low-rank dimension vector, wherein the shape of the first initial low-rank matrix is ​​determined by the input space dimension vector and the low-rank dimension vector, and the shape of the second initial low-rank matrix is ​​determined by the output space dimension vector and the low-rank dimension vector.

3. The method for adjusting large model parameters based on cloud-edge collaboration according to claim 2, characterized in that, The process of mapping the cloud sample data based on the original parameter matrix and the initial low-rank matrix to obtain the cloud target data includes: Transpose the first initial low-rank matrix to obtain the low-rank transpose matrix; The cloud sample data is mapped based on the original parameter matrix to obtain a first mapping vector, and the cloud sample data is mapped based on the low-rank transpose matrix and the second initial low-rank matrix to obtain a second mapping vector. The cloud target data is obtained by superimposing the first mapping vector and the second mapping vector.

4. The method for adjusting large model parameters based on cloud-edge collaboration according to claim 3, characterized in that, The process of determining the knowledge regularization loss value based on the network security knowledge matrix and the initial low-rank matrix includes: Acquire knowledge to adjust weighting parameters; The incremental parameter matrix is ​​determined based on the product of the low-rank transpose matrix and the second initial low-rank matrix. The incremental parameter matrix is ​​used to characterize the amount of change in the original parameter matrix. Based on the product of the incremental parameter matrix and the network security knowledge matrix, the domain knowledge consistency matrix is ​​determined, and the sum of squares of all matrix elements in the domain knowledge consistency matrix is ​​calculated to obtain the total sum of squares. The knowledge regularization loss value is determined based on the product of the sum of squares and the knowledge adjustment weight parameter.

5. The method for adjusting large model parameters based on cloud-edge collaboration according to claim 1, characterized in that, The target large model is obtained by training the student model through multiple rounds of model training; After integrating the target low-rank matrix and the basic large model into a teacher model and sending it to the edge device, the method further includes: The basic model parameters of the basic large model are obtained, and the incremental parameters fed back from the edge device are received, wherein the incremental parameters are determined by the edge device based on the basic model parameters and the adjusted student model parameters; Based on the basic model parameters and the incremental parameters, the incremental weights are determined. In the next training round, the cloud server adjusts the target model parameters of the target large model again based on the incremental weights and the target low-rank matrix, and performs structured pruning on the target large model after parameter adjustment to obtain an optimized large model with cloud-edge collaborative parameter adjustment in the network security scenario.

6. A method for adjusting parameters of a large model based on cloud-edge collaboration, characterized in that, The large model parameter adjustment method for cloud-edge collaboration, applied to edge-side devices, includes: The system receives a teacher model sent from a cloud server according to any one of claims 1 to 5 and acquires edge sample data. The teacher model includes a basic large model and a target low-rank matrix, wherein the basic large model is pre-acquired by the cloud server, and the target low-rank matrix is ​​obtained by the cloud server by adjusting a pre-constructed low-rank matrix. A student model is constructed based on the aforementioned basic large model. The marginal sample data is then input into the teacher model and the student model to obtain teacher sample data and student sample data, respectively. Based on the teacher sample data and the student sample data, a distillation loss value is determined, and the student model parameters are adjusted based on the distillation loss value. The adjusted student model is then subjected to structured pruning to obtain the target large model for cloud-edge collaborative parameter adjustment in a network security scenario.

7. The method for adjusting large model parameters based on cloud-edge collaboration according to claim 6, characterized in that, The determination of the distillation loss value based on the teacher sample data and the student sample data includes: Obtain the real sample labels corresponding to the edge sample data, and the data complexity that characterizes the data processing difficulty of the student model; Guided by a preset distillation loss weight, the edge task loss value is determined based on the real sample labels and the student sample data. The temperature coefficient is determined based on the data complexity, and the knowledge transfer loss value is jointly determined based on the temperature coefficient, the distillation loss weight, the teacher sample data, and the student sample data. The distillation loss value is obtained by superimposing the edge task loss value and the knowledge transfer loss value.

8. The method for adjusting large model parameters based on cloud-edge collaboration according to claim 7, characterized in that, The determination of the temperature coefficient based on the data complexity includes: Obtain the adaptive function; The temperature coefficient is obtained by adaptively processing the data complexity based on the adaptive function. The target large model is obtained by training the student model through multiple rounds. The data complexity obtained in the current training round is determined based on the distillation loss value obtained in the previous training round. The initial value of the data complexity is a preset value.

9. The method for adjusting large model parameters based on cloud-edge collaboration according to claim 6, characterized in that, The structured pruning process performed on the student model after parameter adjustment yields the target large model for cloud-edge collaborative parameter adjustment in a cybersecurity scenario, including: Determine the removal parameters of the student model, wherein the removal parameters are limited to not exceeding a preset multiple of the student model parameters; The pruning factor is determined based on the removal parameters and the student model parameters, and the optimization function is determined based on the square value of the Euclidean norm of the pruning factor. With the goal of minimizing the optimization function, the student model after parameter adjustment is subjected to structured pruning to obtain the target large model with cloud-edge collaborative parameter adjustment in the network security scenario.

10. A large model parameter adjustment device based on cloud-edge collaboration, characterized in that, The cloud-edge collaborative large model parameter adjustment device, applied to cloud servers, includes: The acquisition module is used to acquire a general basic large model, a network security knowledge matrix, and cloud sample data in network security scenarios, and construct an initial low-rank matrix. The basic large model includes multiple initial processing layers, each of which includes a corresponding original parameter matrix. The shape of the original parameter matrix is ​​determined by the input space dimension vector and the output space dimension vector. Both the input space dimension vector and the output space dimension vector are high-order dense vectors. The shape of the initial low-rank matrix is ​​determined by the input space dimension and the output space dimension vector. The cloud data mapping and processing module is used to input the cloud sample data into the basic large model, and perform data mapping processing on the cloud sample data based on the original parameter matrix and the initial low-rank matrix to obtain the cloud target data. The target low-rank matrix determination module is used to determine the cloud task loss value based on the cloud sample data and the cloud target data, determine the knowledge regularization loss value based on the network security knowledge matrix and the initial low-rank matrix, and adjust the initial low-rank matrix according to the cloud task loss value and the knowledge regularization loss value to obtain the target low-rank matrix. The sending module is used to integrate the target low-rank matrix and the basic large model into a teacher model and send it to the edge device, so that the edge device can train the constructed student model based on the teacher model, and determine the target large model for cloud-edge collaborative parameter adjustment in the network security scenario based on the trained student model.

11. A large model parameter adjustment device based on cloud-edge collaboration, characterized in that, The large model parameter adjustment device based on cloud-edge collaboration, applied to edge-side devices, includes: A receiving module is configured to receive a teacher model sent from a cloud server according to any one of claims 1 to 5, and to acquire edge sample data. The teacher model includes a basic large model and a target low-rank matrix, wherein the basic large model is pre-acquired by the cloud server, and the target low-rank matrix is ​​obtained by the cloud server by adjusting a pre-constructed low-rank matrix. The sample data processing module is used to construct a student model based on the basic large model, and input the marginal sample data into the teacher model and the student model respectively to obtain teacher sample data and student sample data respectively. The target large model generation module is used to determine the distillation loss value based on the teacher sample data and the student sample data, adjust the student model parameters based on the distillation loss value, and perform structured pruning on the student model after parameter adjustment to obtain the target large model with cloud-edge collaborative parameter adjustment in the network security scenario.

12. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the large model parameter adjustment method based on cloud-edge collaboration as described in any one of claims 1 to 5, or implements the large model parameter adjustment method based on cloud-edge collaboration as described in any one of claims 6 to 9.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the large model parameter adjustment method based on cloud-edge collaboration as described in any one of claims 1 to 5, or the large model parameter adjustment method based on cloud-edge collaboration as described in any one of claims 6 to 9.

Images